Computer program, data archiving method, and recording medium

Abstract

To provide a real time data archive method and a system therefor based on a hybrid cloud.SOLUTION: The present invention is directed to a data archive method and a data archive apparatus arranged such that a remote near-line data archive function can be provided by receiving a call of a remote function from an object system storing data, responding to the call of the remote function, providing a first function of archiving at least a part of data stored in the object system to a storage system to the object system through a network, and providing a second function of inquiring data archived in the storage system to the object system through the network.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The following description relates to a method and system for real-time data archiving in a hybrid cloud infrastructure. [Background technology]

[0002] In recent years, the need for research into data archiving has increased due to stricter data regulations, the growth of the healthcare industry, the growing importance of patient data storage and management, and growing interest in corporate data management. For example, in terms of data regulations, data such as financial transaction data and medical information are required to be stored for years or even decades to protect consumer rights. Various data-related regulations require long-term data storage. Furthermore, in the healthcare industry, the growing reliance on diagnostic imaging has led to an increase in the volume of medical image data, driving demand for storage and backup systems for data storage and management, as well as the need for archive systems to manage the ever-increasing volume of data. In terms of corporate data management, the ability to store and manage the vast amounts of data transmitted and received by companies on servers, and the ability to restore and back up data in real time, as well as the ability to securely manage critical data, are becoming increasingly important. Furthermore, in terms of manufacturing automation, many traditionally labor-intensive manufacturing companies have relocated their factories to low-wage countries such as China and India. However, advances in robotics technology have led to the construction of integrated robot factories that combine high production efficiency and high quality, accelerating process automation.

[0003] With the advent of the Fourth Industrial Revolution, the big data field has come to the forefront, but as the current domestic big data field is mass-produced and concentrated on infrastructure investments such as servers, storage, and networks, future development of archive technology is required to decentralize infrastructure investment and expand development into the software and service fields. This has led to an increasing demand for the development of archive technology that has higher data compression rates and speeds than conventional technologies, while also enabling rapid data querying. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Korean Patent Publication No. 2014-0072929 Summary of the Invention [Problem to be solved by the invention]

[0005] A data archiving method and apparatus are provided that can provide remote near-line data archiving functionality by receiving a remote function call from a target system that stores data, responding to such remote function call, providing the target system via the network with a first function for archiving at least a portion of the data stored in the target system to a storage system, and providing the target system via the network with a second function for querying the data archived in the storage system. [Means for solving the problem]

[0006] A data archiving method executed by a computer device including at least one processor includes receiving, by the at least one processor, a remote function call from a target system that stores data; responding to the remote function call by the at least one processor, providing, to the target system via the network, a first function for archiving at least a portion of the data stored in the target system to a storage system; and providing, by the at least one processor, to the target system via the network, a second function for querying the data archived in the storage system.

[0007] According to one aspect, the step of providing the first function may be characterized by providing a function for compressing at least a portion of the data stored in a local database of the target system and archiving it in a table of the local database.

[0008] According to another aspect, the step of providing the first functionality may be characterized by providing functionality for compressing at least a portion of the data stored in a local database of the target system and archiving the data in a table in an external database of the target system.

[0009] According to another aspect, the step of providing the first function may be characterized by providing a function for compressing at least a portion of the data stored in the local database of the target system as a file and archiving the compressed file in a storage location included in a system external to the target system.

[0010] According to another aspect, the step of providing the first function may be characterized by providing a 1-1 function for controlling the target system to determine partitions for data records included in a table of a local database of the target system based on selection information for the data records; a 1-2 function for controlling the target system to compress the data records for each partition to generate compressed partitions; a 1-3 function for controlling the target system to associate the compressed partitions with a storage key that uniquely identifies the compressed partition and store them in a compression table; and a 1-4 function for controlling the target system to associate the storage key with the selection information and record them in an index table of the local database.

[0011] According to another aspect, the selection information may include an arbitrary field value of the corresponding data record, and functions 1-4 may be characterized by controlling the target system to associate the storage key with the arbitrary field value and record it in a group index table of the local database.

[0012] According to another aspect, the selection information may include information regarding the time of the corresponding data record, and the first to fourth functions may be characterized by controlling the target system to link the storage key with the information regarding the time and record them in a period index table.

[0013] According to another aspect, the step of providing the first function may further include providing a first to fifth function that controls the target system to record, for each data record included in the table, a primary key, key index information that is the position of the data record in a compressed partition that contains the data record, and a storage key corresponding to the compressed partition that contains the data record in a key index table in association with each other.

[0014] According to another aspect, functions 1-5 may be characterized by controlling the target system to search for a data record whose primary key is equal to that of a data record included in the table among the data records included in the second compressed partition, the data record being generated by compressing data records in a linked table linked to the table by the primary key, and to further record subindex information, which is a position within the second compressed partition, for the data record whose primary key is equal in the key index table for the searched data record.

[0015] According to another aspect, the step of providing the first function may further include providing a first to sixth function that controls the target system to delete the compressed data record from the table.

[0016] According to another aspect, the step of providing the first function may further include providing a first to seventh function that controls the target system to, in response to a restore request for the deleted data record, search the index table for a storage key associated with identification information included in the restore request, search the compression table for a compressed partition associated with the searched storage key, decompress the searched compressed partition to restore the deleted data record, and record the restored data record in the table based on the identification information.

[0017] According to another aspect, the first-second function may be characterized by controlling the target system to compress data records included in the determined partition as binary objects to generate the compressed partition.

[0018] According to another aspect, the step of providing the second function may include providing a second-first function that controls the target system to receive search criteria including data record selection information; a second-second function that controls the target system to search for a storage key associated with the selection information included in the search criteria from an index table that records the data record selection information in a local database of the target system in association with a storage key that uniquely identifies a compressed partition containing the corresponding data record; and a second-third function that controls the target system to search for a compressed partition associated with the searched storage key from a compression table that records the storage key in association with a compressed partition.

[0019] According to another aspect, the data archiving method may further include providing, via the network to the target system, a third function for managing the life cycle of data archived in the storage system.

[0020] According to yet another aspect, the step of providing the third function may be characterized by providing a 3-1 function for controlling the target system to archive and store data managed in a compressed state in a database table as a file when a storage period for the data archived in the storage system has expired, and a 3-2 function for controlling the target system to delete the data archived as a file.

[0021] A computer program is provided that is recorded on a computer-readable recording medium and that, when combined with a computer device, causes the computer device to execute the method.

[0022] A computer-readable recording medium is provided, on which a program for causing a computer device to execute the method is recorded.

[0023] A computer apparatus is provided, comprising at least one processor implemented to execute computer-readable instructions, wherein the at least one processor receives a remote function call from a target system storing data, responds to the remote function call, provides the target system, via the network, with a first function for archiving at least a portion of the data stored in the target system to a storage system, and provides the target system, via the network, with a second function for querying data archived in the storage system. [Effects of the Invention]

[0024] A remote near-line data archiving function can be provided by receiving a remote function call from a target system storing data, responding to such remote function call, providing the target system via the network with a first function for archiving at least a portion of the data stored in the target system to a storage system, and providing the target system via the network with a second function for querying the data archived in the storage system. [Brief explanation of the drawings]

[0025] [Figure 1] FIG. 1 illustrates an example of a network environment in accordance with an embodiment of the present invention. [Figure 2] FIG. 1 is a block diagram illustrating an example of a computing device according to an embodiment of the present invention. [Figure 3] FIG. 1 is a diagram generally illustrating a computer system for archiving in accordance with one embodiment of the present invention. [Figure 4] 1 is a flowchart illustrating an example method for archiving data in accordance with an embodiment of the present invention. [Figure 5] 10 is a flowchart illustrating an example of a process for archiving data according to a first function in accordance with an embodiment of the present invention. [Figure 6] FIG. 2 is a diagram showing a first example of the structure of a compression table in one embodiment of the present invention. [Figure 7] FIG. 10 is a diagram showing a second example of the structure of a compression table in one embodiment of the present invention. [Figure 8] FIG. 10 is a diagram showing an example of the structure of a period index table in one embodiment of the present invention. [Figure 9] FIG. 10 is a diagram showing an example of the structure of a group index table in one embodiment of the present invention. [Figure 10] FIG. 10 is a diagram showing a second example of the structure of a compression table in one embodiment of the present invention. [Figure 11]10 is a diagram illustrating an example of the structure of an index table in which a period index table and a group index table are combined in an embodiment of the present invention. FIG. [Figure 12] 10 is a flowchart illustrating another example of a process for archiving data according to a first function in accordance with an embodiment of the present invention. [Figure 13] 10A and 10B are diagrams showing examples of the structures of a compression table and a key index table in one embodiment of the present invention. [Figure 14] 10A and 10B are diagrams showing other examples of the structures of a compression table and a key index table in an embodiment of the present invention. [Figure 15] FIG. 10 illustrates an example of a process for searching archived data by a second function in an embodiment of the present invention. [Figure 16] FIG. 10 illustrates an example of searching archived data in accordance with an embodiment of the present invention. [Figure 17] FIG. 10 illustrates an example of searching archived data in accordance with an embodiment of the present invention. [Figure 18] FIG. 1 illustrates an example process for efficiently storing data in accordance with one embodiment of the present invention. [Figure 19] FIG. 1 illustrates an example method for de-identifying data in accordance with an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0026] The present invention may be modified in various ways and may have various embodiments, and specific embodiments will be described in detail in the following detailed description with reference to the drawings. However, this is not intended to limit the present invention to the specific embodiments, and it should be understood that the present invention includes all modifications, equivalents, and replacements within the spirit and technical scope of the present invention. In describing the various drawings, similar elements are designated by similar reference numerals.

[0027] Terms such as "first," "second," "A," and "B" are merely used to describe various components and should not be construed as limiting the scope of the present invention. These terms are merely intended to distinguish one component from another. For example, a first component may be designated as a second component, and similarly, a second component may be designated as a first component, without departing from the scope of the present invention. The term "and / or" includes a combination of multiple related listed items or any of multiple related listed items.

[0028] A description that a component is "coupled" or "connected" to another component should be understood to include not only a state in which the component is directly coupled or connected to the other component, but also a state in which there are other components between them. Conversely, a description that a component is "directly coupled" or "directly connected" to another component should be understood to include a state in which there are no other components between them.

[0029] The terms used in this application are merely for the purpose of describing particular embodiments and are not intended to limit the present invention. The singular terms include the plural terms unless otherwise clearly stated in the context. In this application, terms such as "comprise" or "have" are intended to specify the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, and should be understood not to preclude the presence or additional possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0030] Unless otherwise defined, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Commonly used, predefined terms should be interpreted to have a meaning consistent with the meaning described in the context of the relevant art, and should not be interpreted as idealized or overly formal unless expressly defined in this application.

[0031] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings.

[0032] A data archiving system according to an embodiment of the present invention may be realized by at least one computer device, and a data archiving method according to an embodiment of the present invention may be executed by at least one computer device included in the data archiving system. A computer program according to an embodiment of the present invention may be installed and implemented in the computer device, and the computer device may execute a data processing method according to an embodiment of the present invention under the control of the implemented computer program. The computer program may be recorded on a computer-readable recording medium in combination with the computer device to cause the computer device to execute the data archiving method.

[0033] FIG. 1 is a diagram showing an example of a network environment in one embodiment of the present invention. The network environment in FIG. 1 shows an example including multiple electronic devices 110, 120, 130, and 140, multiple servers 150 and 160, and a network 170. FIG. 1 is merely an example for explaining the invention, and the number of electronic devices and the number of servers are not limited to those shown in FIG. 1. Furthermore, the network environment in FIG. 1 is merely an example of an environment applicable to this embodiment, and environments applicable to this embodiment are not limited to the network environment in FIG. 1.

[0034] The electronic devices 110, 120, 130, and 140 may be fixed or mobile terminals implemented by computers. Examples of the electronic devices 110, 120, 130, and 140 include smartphones, mobile phones, navigation systems, personal computers (PCs), notebook PCs, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), and tablets. While FIG. 1 illustrates a smartphone as an example of the electronic device 110, in embodiments of the present invention, the electronic device 110 may represent one of a variety of physical computer devices capable of communicating with the other electronic devices 120, 130, and 140 and / or the servers 150 and 160 via the network 170 using a substantially wireless or wired communication method.

[0035] The communication method is not limited, and may include not only communication methods using communication networks (for example, a mobile communication network, a wired Internet, a wireless Internet, and a broadcast network) that can be included in network 170, but also short-range wireless communication between devices. For example, network 170 may include any one or more of networks such as a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), and the Internet. Furthermore, network 170 may include any one or more of network topologies including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or hierarchical network, etc.

[0036] Each of the servers 150, 160 may be implemented by one or more computing devices that communicate with multiple electronic devices 110, 120, 130, 140 via the network 170 to provide instructions, code, files, content, services, etc. For example, the server 150 may be a system that provides services (such as an archive service, a file distribution service, a map service, a content provision service, a group call service (or an audio conference service), a messaging service, an email service, a social networking service, a translation service, a financial service, a payment service, a search service, etc.) to multiple electronic devices 110, 120, 130, 140 connected via the network 170.

[0037] 2 is a block diagram showing an example of a computer device according to an embodiment of the present invention. Each of the electronic devices 110, 120, 130, and 140 and each of the servers 150 and 160 described above may be realized by a computer device 200 shown in FIG.

[0038] As shown in FIG. 2 , such a computer device 200 may include a memory 210, a processor 220, a communication interface 230, and an input / output interface 240. The memory 210 is a computer-readable recording medium and may include random access memory (RAM), read-only memory (ROM), and a persistent mass storage device such as a disk drive. Here, a persistent mass storage device such as a ROM or a disk drive may be included in the computer device 200 as a separate persistent storage device distinct from the memory 210. The memory 210 may also store an operating system and at least one program code. Such software components may be loaded into the memory 210 from a computer-readable recording medium separate from the memory 210. Such separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, or a memory card. In another embodiment, the software components may be loaded into the memory 210 through a communication interface 230, which is not a computer-readable recording medium. For example, the software components may be loaded into the memory 210 of the computing device 200 based on a computer program installed by a file received over the network 170 .

[0039] Processor 220 may be configured to process computer program instructions by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to processor 220 by memory 210 or by communication interface 230. For example, processor 220 may be configured to execute instructions received according to program code stored in a storage device such as memory 210.

[0040] The communication module 230 may provide a function for the computer device 200 to communicate with other devices (e.g., the above-mentioned storage device) via the network 170. For example, requests, instructions, data, files, etc. generated by the processor 220 of the computer device 200 in accordance with program code stored in a storage device such as the memory 210 may be transmitted to other devices via the network 170 under the control of the communication interface 230. Conversely, signals, instructions, data files, etc. from other devices may be received by the computer device 200 via the communication module 230 of the computer device 200 via the network 170. The signals, instructions, data, etc. received via the communication interface 230 may be transmitted to the processor 220 or the memory 210, and files, etc. may be recorded on a storage medium (e.g., the above-mentioned permanent storage device) that the computer device 200 may further include.

[0041] The input / output interface 240 may be a means for interfacing with the input / output device 250. For example, the input device may include a device such as a microphone, keyboard, or mouse, and the output device may include a device such as a display or speaker. As another example, the input / output interface 240 may be a means for interfacing with a device that integrates input and output functions into one, such as a touch screen. The input / output device 250 may be configured as a single device together with the computer device 200.

[0042] Also, in other embodiments, computing device 200 may include fewer or more components than those shown in Figure 2. However, most prior art components need not be explicitly shown in the figures. For example, computing device 200 may be implemented to include at least some of the input / output devices 250 described above, and may further include other components such as a transceiver, a database, etc.

[0043] FIG. 3 is a diagram generally illustrating a data archiving system according to one embodiment of the present invention.

[0044] The data archiving system 310 may be realized by a physical device such as the computer device 200 described in FIG. 2 or a combination of multiple physical devices, and may include a data compression module 311, a query module 312, a display and control module 313, and a near-line interface module 314, as shown in FIG. 3. Here, the data compression module 311, the query module 312, the display and control module 313, and the near-line interface module 314 may each represent a functional representation of an operation performed by the processor 220 of the computer device 200 implementing the data archiving system 310 in accordance with code of an archiving solution program installed in the data archiving system 310. As an example, the archiving solution program may include code for providing a data compression function, and the processor 220 may provide the data compression function based on such code. In this case, the term "data compression module 310" may be used as a functional representation of the operation(s) for providing the data compression function of the processor 220.

[0045] In other words, the data archiving system 310 may be realized by installing and executing an archiving solution program on the computer device 200. For example, the archiving solution program may be developed as a cloud SaaS (Software as a Service) type product, be installed in cloud systems of various cloud vendors, and provide archiving functions to the target system 320 described below. As another example, the data archiving system 310 may be realized in the form of an appliance server that integrates remote nearline data archiving technology (archiving solution program) and hardware. The appliance server allows for quick delivery and easy maintenance depending on the product form, and can provide consistent product quality and competitive pricing.

[0046] The target system 320 may also be realized by a physical device such as the computer device 200 described in Fig. 2 or a combination of multiple physical devices, and may include a database 321, a control module 322, and a near-line interface module 323 as shown in Fig. 3. In this case, the control module 322 and the near-line interface module 323 may also be functional representations of operations performed by the processor 220 of the computer device 200 that realizes the target system 320.

[0047] The data archive system 310 and the target system 320 may communicate with each other via a network (for example, the network 170 described in FIGS. 1 and 2), and the target system 320 may call functions provided by the data archive system 310 through the near-line interface module 323 under the control of the control module 322. In this case, the data archive system 310 may provide the functions called by the target system 320 to the target system 320. For example, the target system 320 may be an integrated information system for enterprise resource planning (ERP), and the near-line interface module 323 may be based on a remote function call (RFC) for SAP ERP, for example.

[0048] FIG. 4 is a flowchart illustrating an example of a data archiving method according to an embodiment of the present invention. The data archiving method according to this embodiment may be performed by a computer device 200 that implements the data archiving system 310 described above. In this case, the processor 220 of the computer device 200 may be configured to execute control instructions provided by operating system code and at least one computer program code stored in the memory 210. Here, the processor 220 may control the computer device 200 to perform steps 410 to 440 included in the method of FIG. 4 in accordance with the control instructions provided by the code stored in the computer device 200. Furthermore, the computer program may correspond to the archive solution program described above.

[0049] At step 410, computing device 200 may receive a remote function call from a target system that stores data, where the target system may correspond to target system 320 described in FIG. 3 and the remote function call may originate at near-line interface module 323 of target system 320.

[0050] At step 420, the computing device 200 may respond to the remote function call and provide the target system, via the network, with a first function for archiving at least a portion of the data stored on the target system to the storage system.

[0051] For example, referring to FIG. 3 , data archiving system 310 may provide a first function to target system 320 via a network for archiving at least a portion of data stored in database 321 of target system 320 to storage system 330 upon request from target system 320.

[0052] Here, depending on the embodiment, storage system 330 may be a local database included in target system 320 (for example, database 321), a database external to target system 320, and / or a storage location included in a system external to target system 320 (for example, a file server or cloud server).

[0053] For example, the data archiving system 310 may provide, as a first function, a function for compressing at least a portion of the data stored in the database 321 of the target system 320 and archiving the data in a table of the database 321. In this case, the compressed data is stored in a table of the database 321 of the target system 320, rather than in a file format, which increases the archiving speed and also improves the speed of data queries.

[0054] As another example, data archiving system 310 may provide, as a first function, a function for compressing at least a portion of the data stored in database 321 of target system 320 and archiving the compressed data in a table of a database external to target system 320. For example, assuming target system 320 is a client from the perspective of data archiving system 310, data archiving system 310 may store the compressed data in a table of a database included by another client.

[0055] As another example, data archiving system 310 may provide, as a first function, a function for compressing at least a portion of the data stored in database 321 of target system 320 as a file and archiving the compressed data in a storage location included in a system external to target system 320. For example, if data archiving system 310 is implemented in a cloud system, data archiving system 310 may store a file including the compressed data in a storage location of the cloud system.

[0056] As a more specific example, the data archive system 310 may provide a user interface via the display and control module 313 to the target system 320 for providing functions such as storage cycle management, archive configuration, archive execution, monitoring, data query, and data management for the archive service.

[0057] At this time, if archiving is requested via a user interface provided by the display and control module 313, the data archiving system 310 may provide the target system 320 with a first function for archiving at least a portion of the data stored in the database 321 of the target system 320 to the storage system 330 in accordance with the archiving configuration set by the data compression module 311. In other words, the target system 320 may archive at least a portion of the data stored in its own database 321 to the storage system 330 using the first function provided by the data archiving system 310.

[0058] At step 430, the computing device 200 may provide a second function to the target system over the network for querying data archived in the storage system, which may also be provided by a remote function call from the target system.

[0059] For example, referring to FIG. 3, data archiving system 310 may provide a second function to target system 320 via a network to query data archived in storage system 330 upon request from target system 320.

[0060] If a data query is requested via the user interface provided by display and control module 313, data archiving system 310 may provide target system 320 with a second function for querying data archived in storage system 330 via query module 312. In other words, target system 320 may query data archived in storage system 330 via the second function provided by data archiving system 310.

[0061] In this way, the target system 320 can archive the data stored in the database 321 by using the functions provided by the data archive system 310 without the need for additional hardware.

[0062] As described above, the first function provided by the data archive system 310 may include a function for storing compressed data in a table of a database (the database 321 of the target system 320 or an external database) or in file format. In this case, since archive data compressed and stored in a database table may also increase the database volume as data, the data archive system 310 may manage the data life cycle. For example, the data archive system 310 may manage the data life cycle in the order of "database -> data compression archive -> file archive -> archive dissipation." Here, "database" may refer to data managed in a state where it is stored in the database 321 of the target system 320. Furthermore, "data compression archive" may refer to data compressed and managed in a table of a database (the database 321 of the target system 320 or an external database). In addition, "file archiving" may mean archiving and storing data managed in a compressed state in a database table as a file after the storage period for the compressed data has elapsed, and "archive dissipation" may mean deleting data archived as a file that no longer has any meaning in storage.

[0063] The "file archiving" may be performed in a storage location of the target system 320 or in a storage location of a system external to the target system 320. As a more specific example, the data archiving system 310 may connect to the target system 320 after the extraction of the compressed data has been completed in order to transmit the archived data to a cloud system external to the target system 320. At this time, the data archiving system 310 may call the target system via the nearline interface module 314. Such a call may be based on an API call. Because the compressed data is stored in various types of storage, the API may be implemented to be connectable to various types of storage, such as databases, disks, files, in-memory, quantum memory, NoSQL, graph databases, blockchain databases, etc. Meanwhile, the data archiving system 310 may define transmission scenarios based on business types, such as finance, cost, production, sales, materials, quality, systems, etc. In some embodiments, the data archiving system 310 may generate a small group of transmission scenarios taking network bandwidth into consideration. The data archiving system 310 may also assign objects to the transmission scenarios. If a small group of transmission scenarios exists, the data archive system 310 may assign the extracted objects to the small group of transmission scenarios. The data archive system 310 may also convert the extracted objects into binary objects and create a transmission breakdown status table for the object capacity and quantity for each transmission scenario and / or each small group. The data archive system 310 may also perform a transmission simulation. In this case, the data archive system 310 may select a simulation target for each transmission scenario and / or each small group, perform a transmission simulation, and check the transmission time for each object, thereby predicting the optimal time for future transmission for each object data ratio.After the transmission simulation, the data archive system 310 may transmit actual data using scenario information, small group information, and / or object information. Based on the transmission simulation information, the data archive system 310 may optimize the overall completion time by arranging small groups and / or objects with short transmission times based on the small groups and / or objects with the longest transmission times. In this case, the data archive system 310 may store data storage locations according to the type of data and may check the number of transmissions and execution time in real time using a transmission status monitoring tool. The data archive system 310 may also update the extraction progress in a transmission execution map and, if an error occurs, may perform the extractions in the order of completion to maintain speed and completeness. Data transmission may be performed by selecting a streaming method or object-by-object transmission. The data archive system 310 may also check whether transmission of data to the storage system 330 for each scenario and / or small group transmitted by the target system 320 is complete, and then verify the archive data transmission process by comparing the object capacity and quantity status table with the transmitted data for each transmission scenario and / or small group. In this case, data may be transmitted in a 1:1 relationship or simultaneously to different servers in a 1:N relationship. In this case, a transmission details status table may be created for each server.

[0064] FIG. 5 is a flowchart illustrating an example of a process for archiving data using a first function according to an embodiment of the present invention. The process according to this embodiment may be performed by a computer device 200 implementing a target system 320 using a first function provided by a data archiving system 310. In this case, a processor 220 of the computer device 200 may be configured to execute control instructions provided by operating system code and at least one computer program code stored in a memory 210. Here, the processor 220 may control the computer device 200 to perform steps 510 to 550 included in the method of FIG. 5 according to the control instructions provided by the code recorded in the computer device 200. Here, the code may include code for the first function provided by the data archiving system 310.

[0065] In step 510, the computer device 200 may determine a partition for a data record included in a table of a database based on the selection information of the data record. Here, the database may correspond to the database 321 of the target system 320 described in FIG. 3. The selection information may include time information for the data record and / or any field value of the data record. The computer device 200 may determine a partition for the data record based on such time information and / or field value. A table is a basic structural unit for storing data in a database, and in step 510, may refer to a table that is to be archived to save space among multiple tables included in the database.

[0066] For example, the computer device 200 may select data records whose field values are within a predetermined range as one partition. In this case, the field value may be determined by the field value of the field that is most frequently searched in the table. This is because, when the archived data is later searched, index information generated based on the corresponding field value can be used to maximize search efficiency. As another example, the computer device may select data records whose information regarding time is within a predetermined range as one partition.

[0067] Furthermore, a partition may be composed of a set of data records selected from all data records included in a table. At least one partition may be generated, and if necessary, a partition may be generated that is limited to only some of the data records rather than the entire table. For example, data records from 2015 onwards may be excluded from the table, and a partition for archiving may be generated that targets only data records from before 2015.

[0068] On the other hand, the number of data records contained in one partition may be determined by comprehensively analyzing and considering the total number of records contained in the table, the performance of the computer that searches the in-memory database, and the search conditions that are frequently used in the database.

[0069] In another embodiment, if there is an excess partition among the selected partitions whose number of data records exceeds a threshold, the excess partition may be separated into multiple partitions whose number of records is equal to or less than the threshold. For example, suppose the threshold for the number of data records contained in a partition is set to 100,000. However, if the selected partition contains more data records than the threshold, this may cause computer overload and inefficiency, resulting in problems. Therefore, if a partition has more than 100,000 data records, the partition may be separated into multiple partitions of 100,000 each to generate multiple partitions. For example, if one partition has 250,000 data records, the computer device 200 may separate the excess partition into three partitions: two partitions with 100,000 data records each and one partition with 50,000 data records.

[0070] On the other hand, as described above, since the separated multiple partitions are classified according to the same classification criteria based on field values, there is no way to distinguish between these multiple partitions. Therefore, a serial number (e.g., 1, 2, 3, 4, ...) may be assigned to each of the separated multiple record groups and further stored in the serial number field of the index table. In this case, when searching archived data, it becomes possible to distinguish between each of the separated multiple partitions. Such serial numbers may correspond to the sequence described below.

[0071] In step 520, the computer device 200 may compress the data records for each partition to generate compressed partitions. As an example, the computer device 200 may compress the data records included in the determined partitions as binary objects to generate the compressed partitions.

[0072] As an example, to generate a compressed partition, the computer device 200 may first store the data records included in the compressed partition in a buffer. The size of the buffer in which the data records are stored may be determined based on the table structure (the number, type, and size of fields) and the threshold number of data records included in the compressed partition. For example, if a table includes three fields: DATE (8 characters), NAME (30 characters), and AGE (4-byte integer), and the threshold number of data records included in the compressed partition is 100,000, assuming one character is 2 bytes, the buffer size may be a minimum of 100,000 × (8 × 2 + 30 × 2 + 4) = 8 million bytes (approximately 8 megabytes). In this case, the computer device 200 may sequentially read all of the data records and their field values included in the compressed partition and store them in the buffer.

[0073] The computer device 200 may then compress the data stored in the buffer to generate a compressed partition. The compressed partition may be a binary object-shaped result generated by compressing the data stored in the buffer. At this time, a lossless compression algorithm such as ZIP, CTW, LZ77, LZW, gzip, bzip2, or DEFLATE may be used to prevent loss due to compression.

[0074] At this time, the computer device 200 may generate a storage key that is uniquely assigned to each of the generated compressed partitions.

[0075] In step 530, the computer device 200 may associate the compressed partitions with storage keys that uniquely identify the compressed partitions and store them in a compression table. The above description explains that compressed data is stored in a table in the database 321 of the target system 320 or in a table in an external database. The compression table may include a field for storing compressed partitions generated by compressing each partition and a field for storing a storage key uniquely assigned to each compressed partition. The storage key is a key containing a value uniquely assigned to each compressed partition, and the value of the storage key shared by each compressed partition may be stored in a field in the compression table corresponding to the storage key. Furthermore, there may be one or more fields corresponding to the storage key, and a unique storage key for each compressed partition may be formed when the values of the storage keys stored in a distributed manner in one or more fields are combined.

[0076] In step 540, the computer device 200 may associate the storage key with the selection information and store it in an index table of the database. For example, if the selection information includes any field value of the corresponding data record, the computer device 200 may associate the storage key with the any field value and store it in a group index table in step 440. The storage key and field value stored in the group index table may be used as an index for searching the compressed and stored data record using a search condition including the any field value. For another example, if the selection information includes information about the time of the data record, the computer device 200 may associate the storage key with the time information and store it in a period index table. The storage key and time information stored in the period index table may be used as an index for searching the compressed and stored data record using a search condition including the any time information. In other words, an index table including the group index table and / or the period index table may be used to obtain a storage key corresponding to the field value and / or time information included in the search condition, and the storage key may be used to obtain a compressed partition corresponding to the storage key in the compression table.

[0077] In step 550, the computer device 200 may delete the compressed data records from the table. Because the purpose of compressing and archiving a database is to save storage space in the database, the computer device 200 may save storage space in the database by deleting the archived data records from the table. However, in some embodiments, the compressed data records may not be deleted from the table immediately, but may be deleted from the table after a certain period of time has passed.

[0078] Meanwhile, deleted data records may be restored to the corresponding tables later. For example, in response to a restoration request for a deleted data record, the computer device 200 may search the index table for a storage key associated with the identification information included in the restoration request and search the compression table for a compressed partition associated with the searched storage key. The computer device 200 may then decompress the searched compressed partition to restore the deleted data record and record the restored data record in a table based on the identification information. In this case, information from a key index table (described below) may be used to identify the specific data record requested to be restored among the data records included in the compressed partition.

[0079] These steps 510 to 550 may be performed using a first function provided by the data archive system 310. In other words, the data archive system 310 may provide a first function including a function for controlling the target system 320 to perform steps 510 to 550.

[0080] FIG. 6 is a diagram showing a first example of the structure of a compressed table according to an embodiment of the present invention. Table 610 in FIG. 6 includes a Doc. No. field 611, a Date field 612 for time, and a Col1 field 613 for a specific attribute. In this case, computer device 200 may generate compressed partitions by classifying and compressing data records in table 610 based on the field value of Date field 612 or Col1 field 613 of table 610, which is time-related information, as selection information. Computer device 200 may generate compressed table 620 by storing a storage key for uniquely identifying the compressed partition and the corresponding compressed partition in association with each other. For example, compressed table 620 according to the embodiment of FIG. 6 may include an OBJECT ID field 621 having a storage key as a field value and a COMPRESSED DATA field 622 having a compressed partition as a field value.

[0081] FIG. 7 is a diagram showing a second example of the structure of a compression table in one embodiment of the present invention, FIG. 8 is a diagram showing an example of the structure of a period index table in one embodiment of the present invention, and FIG. 9 is a diagram showing an example of the structure of a group index table in one embodiment of the present invention.

[0082] 7 illustrates another embodiment for generating a compressed table 700 based on the table 610 illustrated in FIG. 6. For example, the computer device 200 may generate compressed partitions by classifying and compressing the data records of the table 610 based on the field value of the Date field 612 of the table 610, which is information related to time, as the selection information. The computer device 200 may also generate the compressed table 700 by storing the selection information and the corresponding compressed partitions in association with each other. For example, the compressed table 700 according to the embodiment of FIG. 7 may include a PERIOD field 710 having information related to time as a field value and a COMPRESSED DATA field 720 having a compressed partition as a field value.

[0083] 8 shows an example of a period index table 800 that can be generated and utilized when the compression table 620 includes compressed partitions generated by classifying and compressing data records based on the field value (time information) of the Date field 612. In this case, the period index table 800 may include a PERIOD field 810 having time information as a field value and an OBJECT ID field 820 having a storage key as a field value. For example, when the computer device 200 receives search criteria that include time information (e.g., "2020.01") as selection information, the computer device 200 may use the time information included in the search criteria to search for a corresponding storage key in the period index table 800 (e.g., storage key "O0001" corresponding to the time information "2020.01" in the period index table 800). The searched storage key can then be used to search for a compressed partition corresponding to the storage key in the compression table 620 (e.g., compressed partition "50,000 Rows" corresponding to the storage key "O0001" in the compression table 620).

[0084] 9 also shows an example of a group index table 900 that can be generated and utilized when the compression table 62 includes compressed partitions generated by sorting and compressing data records based on the field value of the Col1 field 613. In this case, the group index table 900 may include a PERIOD field 910 that has the field value of the Col1 field 613 as its field value, and an OBJECT ID field 920 that has the storage key as its field value. For example, when the computer device 200 receives a search condition that includes a field value of the Col1 field 613 (for example, "1000") as selection information, the computer device 200 may use the field value included in the search condition to search for a corresponding storage key (for example, the storage key "O0001" corresponding to the field value "1000" in the group index table 900), and then use the searched storage key to search for a compressed partition corresponding to the storage key in the compression table 620 (for example, the compressed partition of "50,000Rows" corresponding to the storage key "O0001" in the compression table 620).

[0085] FIG. 10 is a diagram showing a second example of the structure of a compression table in one embodiment of the present invention, and FIG. 11 is a diagram showing an example of the structure of an index table in a form in which a period index table and a group index table are combined in one embodiment of the present invention.

[0086] Fig. 10 illustrates another embodiment for generating a compressed table 1000 based on the table 610 illustrated in Fig. 6. For example, the computer device 200 may generate a compressed partition by classifying and compressing the data records of the table 610 based on two field values, namely, the field value of the Date field 612 and the field value of the Col1 field 613 of the table 610, which are time-related information.

[0087] As a more specific example, the computer device 200 compresses a data record in which the field value of the Data field 612 is “2002.01” but the field value of the Col1 field 613 is “1000” to create a first compressed partition, compresses a data record in which the field value of the Data field 612 is “2002.01” but the field value of the Col1 field 613 is “2000” to create a second compressed partition, and compresses a data record in which the field value of the Data field 612 is “2002.02” but the field value of the Col1 field 613 is “1000” to create a third compressed partition. The fourth compressed partition may be generated by compressing a data record in which the field value of the Data field 612 is "2002.02" but the field value of the Col1 field 613 is "2000." The fifth compressed partition may be generated by compressing a data record in which the field value of the Data field 612 is "2002.03" but the field value of the Col1 field 613 is "1000." The sixth compressed partition may be generated by compressing a data record in which the field value of the Data field 612 is "2002.03" but the field value of the Col1 field 613 is "2000."

[0088] In this case, the computer device 200 may generate the compressed table 1000 by storing a storage key for uniquely identifying the compressed partition and the corresponding compressed partition in association with each other. For example, the compressed table 1000 according to the embodiment of FIG. 10 may include an OBJECT ID field 1010 having the storage key as a field value and a COMPRESSED DATA field 1020 having the compressed partition as a field value.

[0089] 11 shows an example of an index table 1100 in which a period index table and a group index table are combined. In this case, the index table 1100 may include a PERIOD field 1110 having time-related information as a field value, a Col1 field 1120 having the field value of the Col1 field 613 as its own field value, and an OBJECT ID field 1130 having a storage key as a field value. For example, when the computer device 200 receives a search condition that includes time information (for example, "2020.02") and a field value of the Col1 field 613 (for example, "1000") as selection information, the computer device 200 may search the index table 1100 for a storage key that satisfies both the time information and the field value included in the search condition (for example, the storage key "O0003" in the index table 1100), and the searched storage key can be used to search the compression table 1000 for a compressed partition corresponding to the storage key (for example, the compressed partition of "30,000Rows" corresponding to the storage key "O0003" in the compression table 1000).

[0090] 12 is a flowchart illustrating another example of a process for archiving data according to the first function in accordance with an embodiment of the present invention. The process according to this embodiment may further include step 1210 after step 540 described in FIG. 5.

[0091] In step 1210, the computer device 200 may store, in a key index table, a primary key for each data record included in the table, key index information indicating the location of the data record in a compressed partition that contains the data record and is compressed, and a storage key corresponding to the compressed partition that contains the data record. Step 1210 may be performed using a first function provided by the data archiving system 310. In other words, the data archiving system 310 may provide a first function including a function for controlling the target system 320 to perform step 1210.

[0092] A primary key may refer to the value in a field that uniquely identifies each record in an in-memory database, and is also called a base key, primary key, or unique key. A table may have one or more primary keys. Key index information is information about the location in a compressed partition where a data record having a specific primary key value is stored. For example, the key index information may store information about the storage order, such as the 1,000th data record stored among information about 100,000 data records contained in the compressed partition.

[0093] Meanwhile, the reason why the primary key is stored in the key index table is to enable direct search of the table to be searched by this primary key, as well as by other field values and time information. That is, the key index table may be used when a user enters a specific primary key and searches a table for a data record having this primary key. More specifically, the computer device 200 may search the key index table for key index information and a storage key of a data record having a specific primary key. In this case, the computer device 200 may use the acquired storage key to acquire a compressed partition corresponding to the storage key in the compressed table, and use the key index information to search for a specific data record desired by the user in the compressed partition. As described above, the key index information in the key index table may be used to identify a data record meeting a specific condition from among the data records contained in the compressed partition when restoring the data record meeting a specific condition to a table.

[0094] FIG. 13 is a diagram showing an example of the structure of a compression table and a key index table in one embodiment of the present invention.

[0095] Compressed table 1310 may include an OBJECT ID field 1311 having a storage key as a field value, a SEQ field 1312 having a processing order (sequence) of the target table as a field value, and a COMPRESSED DATA field 1313 having a compressed partition as a field value. When a parent table and a child table exist, the sequence may define a processing order in which the parent table is extracted first and the child table is processed using the extracted data of the parent table.

[0096] As described above, key index table 1320 may include Doc.No. field 1321, which has a primary key as a field value, OBJECT ID field 1322, which has a storage key as a field value, and Key Location info. field 1323, which has key index information as a field value. For example, in key index information 1@1001, the "1" before the "@" may indicate a sequence corresponding to the field value of SEQ field 1312, and the "1001" after the "@" may indicate the 1001st data record among the data records contained in the corresponding compressed partition. As a more specific example, the first record in key index table 1320 may indicate that a data record with a primary key of "1" is included as the 1001st data record among the data records in a compressed partition with a storage key of "O0001" and a sequence of "1." Similarly, the second record in key index table 1320 may indicate that the data record with primary key “2” is included as the 2001st data record among the data records in the compressed partition with storage key “O0001” and sequence “2.”

[0097] In this way, the key index information may include information regarding the location of a particular data record within a compressed partition, and by utilizing a key index table (for example, key index table 1320 in Figure 13) that includes such key index information, a primary key, and a storage key, the number of data records that are queried based on a user's search criteria can be reduced.

[0098] In another embodiment, the computer device 200 may search for data records included in the second compressed partition, which is generated by compressing data records in a concatenated table linked to the first table (e.g., the table described in step 410) by a primary key, among the data records included in the second compressed partition, for data records having a primary key equal to that of the data record included in the first table. For the searched data records, subindex information indicating the location of the data records in the second compressed partition may be further stored in the key index table for the data records having the same primary key. The concatenated table is a table linked to the first table by a primary key. That is, the primary key exists in both the first table and the concatenated table. The second compressed partition may be data generated by compressing data records in the concatenated table when a concatenated table linked to the first table by a primary key exists. In this case, the second compressed partition may be generated in the same manner as the compressed partition described in FIG. 4 and may be stored in the compressed table with a unique storage key, similar to the compressed partition. The subindex information is information regarding the location of a data record having a specific primary key in a second compressed partition. For example, information about the storage order, such as the 1,000th data record stored among information about 100,000 data records included in the second compressed partition, may be stored as sub-index information. For example, a linked table linked to the first table by a primary key exists in the in-memory database, and a user may need field value information for a field that does not exist in the first table but does exist in the linked table. In this case, the computer device 200 may further store sub-index information for data records having the same primary key in the key index table so that the linked table can be searched later.

[0099] In yet another embodiment, when there are multiple concatenated tables for a first table, the computer device 200 may collect and compress subindex information for each concatenated table, and then store the collected information as new subindex information in the key index table. For example, the computer device 200 may collect all subindex information for positions in two or more second compressed partitions for data records having the same primary key in the concatenated tables, compress the collected values, and store the compressed values as new subindex information for the data records having the same primary key value in the key index table.

[0100] FIG. 14 is a diagram showing another example of the structure of the compression table and the key index table in one embodiment of the present invention.

[0101] Compressed table 1410 may include an OBJECT ID field 1411 having a storage key as a field value, a TABLE field 1412 having a table identifier as a field value, a SEQ field 1413 having a sequence as a field value, and a COMPRESSED DATA field 1414 having a compressed partition as a field value. TABLE field 1412 may include a table identifier as a field value, which may identify from which table the corresponding compressed partition contains data records extracted.

[0102] The key index table 1420 of this embodiment may include a Doc.No. field 1421 having a primary key as a field value, an OBJECT ID field 1422 having a storage key as a field value, a Key Location info. field 1423 having key index information as a field value, and a Sub Location info. field 1424 having subindex information as a field value.

[0103] For example, the first record in key index table 1420 may indicate that a data record with primary key "1" is included as the 10001st data record among the data records in the compressed partition with storage key "O0001" and sequence "1." In this case, the field value "TAB1@1001-2 / TAB2@2001-3" in Sub Location info. field 1424 indicates the location within the second compressed partition generated for the concatenated table of the data record with primary key "1." For example, in the field value "TAB1@1001-2 / TAB2@2001-3," "TAB1" and "TAB2" before "@" may refer to concatenated tables linked by equal primary keys, and "1001-2" after "@" indicates two data records (the 1001st data record (first data record) and the 1002nd data record (second data record)) from the 1001st data record among the data records contained in the second compressed partition for the concatenated table "TAB1." Furthermore, "2001-3" after "@" indicates three data records (the 2001st data record (third data record), the 2002nd data record (fourth data record), and the 2003rd data record (fifth data record)) from the 2001st data record among the data records contained in the second compressed partition for the concatenated table "TAB2." In this case, the first to fifth data records may all be identified by the same primary key.

[0104] 15 illustrates an example process for searching archived data using the second function according to an embodiment of the present invention. The process according to this embodiment may be performed by computer device 200 implementing target system 320 using the second function provided by data archiving system 310.

[0105] In step 1510, the computer device 200 may receive search criteria including selection information for data records. Such selection information may include information about any field values of the data records to be searched and / or time of the corresponding data records. The information about field values and / or time included in the selection information may be included in the form of a range.

[0106] In step 1520, the computer device 200 may search for a storage key associated with the selection information included in the search criteria from an index table that stores, in a database, the selection information of the data records in association with a storage key that uniquely identifies a compressed partition containing the corresponding data records. As described above, the index table may include a group index table and / or a period index table, where the group index table may store a specific field value in association with a storage key, and the period index table may store time-related information in association with a storage key. Thus, the computer device 200 may search the group index table and / or the period index table for a storage key corresponding to the field value and / or time information included in the selection information. For example, if the selection information includes an arbitrary field value of the data records, the computer device 200 may search for a storage key associated with the arbitrary field value included as the selection information of the search criteria from a group index table that stores the storage key in association with the arbitrary field value. As another example, if the selection information includes information about the time of the data record, the computer device 200 may search for a storage key associated with the time information included as the selection information of the search condition in a period index table that stores storage keys and time information in association with each other.

[0107] In step 1530, the computer device 200 may search for a compressed partition associated with the searched storage key in a non-volatile storage means that stores storage keys and compressed partitions in association with each other. As described above, the compression table stores compressed partitions in association with storage keys that uniquely identify the compressed partitions, so the computer device 200 can search for the corresponding compressed partition in the compression table using the storage key.

[0108] As described above, if a key index table is further utilized, a user may utilize a primary key for a search. As described above, the key index table may store, for each data record included in a table in the database, a primary key, key index information indicating the location of the data record within a compressed partition that contains the data record, and a storage key corresponding to the compressed partition that contains the data record, in association with each other. In this case, if the search criteria further includes a primary key of the data record, the computer device 200 may search the key index table for key index information and a storage key associated with the primary key further included in the search criteria. Then, the computer device 200 can search the compressed partition searched in step 1530 for a specific data record according to the search criteria based on the searched key index information and storage key.

[0109] In addition, if there is a concatenated table that is connected to an arbitrary table by a primary key, the key index table may further include subindex information, which is the position of the data record in the second compressed partition, for a second compressed partition generated by compressing the data record in the concatenated table. Therefore, if the search condition further includes a primary key, the computer device 200 can further search the key index table for subindex information associated with the primary key further included in the search condition, and further search the second compressed partition for the data record according to the search condition based on the second compressed partition and the subindex information, thereby obtaining not only the field value of the first table to be searched for a specific data record, but also the field value of the concatenated table that is connected to the first table by the primary key.

[0110] On the other hand, as described above, the compressed table may include a compressed table of a database of another computer device connected via a network to the computer device 200. In this case, in step 1530, the computer device 200 may search for a compressed partition associated with the storage key searched for in step 1520 from the compressed table of the database of the other computer device via the network.

[0111] These steps 1510 to 1530 may be performed using a second function provided by the data archive system 310. In other words, the data archive system 310 may provide a second function including a function for controlling the target system 320 to perform steps 1510 to 1530.

[0112] 16 and 17 are diagrams illustrating an example of searching archived data in one embodiment of the present invention.

[0113] FIG. 16 illustrates an example of searching archived data in a compression table 1620 according to a query 1610. In the embodiment of FIG. 16, the compression table 1620 is coupled to an index table and includes a PERIOD field 1621, a COL1 field 1622, a TABLE field 1623, an OBJECT ID field 1624, a SEQ field 1625, and a COMPRESSED DATA field 1626. In some embodiments, the PERIOD field 1621 and the COL1 field 1622 may reside in a separate index table. In this case, an OBJECT ID field 1624 may reside in each of the two tables to link the compression table 1620 and the index table. In some embodiments, the TABLE field 1623 and the SEQ field 1625 may also reside in the index table.

[0114] In this case, the query 1610 may represent an instruction to search the table "TAB1" for a data record in which the field value of the PERIOD field 1621 is "2002.01" and the field value of the COL1 field 1622 is "1000." In this case, the computer device 200 can confirm that the data record corresponding to the query 1610 in the compression table 1620 is a compressed partition stored in the COMPRESSED DATA field 1626 of the first record in the compression table 1620. Therefore, the computer device 200 can decompress the corresponding compressed partition and provide the data record (the "50,000 rows" data record) contained in the corresponding compressed partition as a search result.

[0115] 17 shows an example of searching for archived data in a compression table 1620 according to a query 1710. In the embodiment of FIG. 17, the query 1710 uses a primary key as a search condition, and therefore a key index table 1720 may be used. The key index table 1720 includes a Doc. No. field 1721, an OBJECT ID field 1722, a Key Location Info. field 1723, and a Sub Location Info. field 1724.

[0116] In this case, the query 1710 may represent an instruction to search for data records in which the field value of the Doc.No. field 1721 serving as the primary key is "1" from the tables "TAB1" and "TAB2." In this case, the computer device 200 can check the first record in the key index table 1720 in which the field value of the Doc.No. field 1721 is "1," and can search for a data record in the compression table 1620 in which the primary key is "1" based on the field value of the Sub Location Info. field 1724 of the first record. For example, the computer device 200 may extract a data record in the compression table 1620 in which the primary key is "1" based on the sub-index information "TAB1@1001-2 / TAB2@2001-3" of the first record in the key index table 1720 in which the field value of the Doc.No. field 1721 is "1." At this time, it is not necessary to search all the data records in the compressed partition based on the location included in the sub-index information, and it is possible to easily and quickly search for a data record having a specific primary key value.

[0117] 3 , the data archiving system 310 may utilize the data query logs to optimize the data in the target system 320 and the storage system 330 in an embodiment in which another system (e.g., a cloud storage system) external to the target system 320 includes the storage system 330. For example, the data archiving system 310 may continuously optimize the data capacity and user access speed between the target system 320 and the storage system 330 by analyzing at least one of (1) past table access logs for an on-premise (on-premise, a database owned and operated by a company or the like in its own facilities rather than in a cloud environment) database, (2) access volume predicted by machine learning based on the past table access logs, and (3) access logs since the data was transferred to the storage system 330.

[0118] FIG. 18 illustrates an example of a process for effectively recording data in an embodiment of the present invention. FIG. 18 illustrates a target system 320 and a cloud system 1810. In the embodiment of FIG. 18, both the storage system 330 and the data archive system 310 may be implemented on the cloud system 1810. To efficiently store data in a remote storage location (the storage system 330 implemented on the cloud system 1810), the data archive system 310 may differentially manage storage classes according to data usage. For example, the data archive system 310 may provide the target system 320 with a function for controlling the target system 320 to send data based on data usage in an on-premise database. In this case, the data archive system 310 may analyze the current data usage status of the target system 310 using this function, separate the data into classes, and then separate the data into classes before sending it to the cloud system 1810. In this case, the cloud system 1810 may also include class storage for each class, and data corresponding to a particular class may be stored in the class storage.

[0119] The data archiving system 310 may also monitor, separate, and store the current usage status of data sent to the cloud system 1810 by business object and period. For example, the data archiving system 310 may manage storage based on the data usage rate within the storage location of the cloud system 1810.

[0120] Meanwhile, the data archive system 310 may control the target system 320 to transmit the current data usage status to the cloud system 1810, analyze the data usage level using machine learning, and then store the data in each class of storage. For example, the data archive system 310 may control the target system 310 to transfer the current data usage status within the company to the cloud system 1810 for a certain period of time, and may predict the data usage rate by subjecting the transferred current data usage status to machine learning. Furthermore, the data archive system 310 may process the transfer of data between the target system 320 and the cloud system 1810 so that the data is optimized based on the predicted data usage rate. For example, data stored in a storage location (storage system 320) of the cloud system 1810 with a data usage rate equal to or greater than a first threshold may be transferred to a storage location (database 321) of the target system 320, and data stored in the storage location of the target system 320 with a data usage rate equal to or less than a second threshold may be transferred to a storage location of the cloud system 1810. The transfer of data may require data compression and decompression as described in the embodiments of Figures 3-17 above.

[0121] In this way, the data archive system 310 can perform continuous storage optimization tasks based on the current data usage status (past) of the target system 320, the current data usage status (present) of the cloud system, and the data usage rate (future) predicted by machine learning.

[0122] In another embodiment, the data archive system 310 may provide a function for optimizing the performance of the target system 320. For example, the target system 320 may be located in the form of an instance in a cloud environment. In this case, for example, after data deletion from the target system 320 (or after the storage space usage is reduced through the above-described continuous storage optimization operation), the data archive system 310 may monitor the overall performance (e.g., CPU, memory usage, system response speed) of the target system 320 located in the form of an instance in a cloud environment based on the database capacity of the target system 320. Based on the monitored performance, the specifications of the target system 320 may be changed to a server type that can reduce costs, thereby reducing the costs of the target system 320. For example, the data archive system 310 may provide an instance optimization function that considers both CPU and memory efficiency rather than data volume optimization. To this end, the data archive system 310 may consider the possibility of optimizing additional resources due to a reduction in data volume. As a more specific example, the data archive system 310 may measure the time for each flow by analyzing the technical bill of materials (BOM) and internal structure of a program that has been used frequently over a recent period (e.g., one year), and may reduce the specifications of the CPU, memory, etc. by reducing the processing time of database-related logic. The data archive system 310 may also change the instance for realizing the target system 320 to an instance that is one level lower in cost than the default instance class. In addition to the technical BOM and internal structure of the program, the time for each flow may also be measured using system response rate, CPU usage rate, processing time, database response time, etc.

[0123] In another embodiment, the data archiving system 310 may provide a data de-identification function. De-identification may be required at the time of data archive collection due to business and / or legal requirements. Alternatively, de-identification may be required to utilize data archived in the storage system 330 in a system other than the target system 320. Figure 19 illustrates an example method for de-identifying data in one embodiment of the present invention.

[0124] Thus, according to an embodiment of the present invention, a remote near-line data archiving function can be provided by receiving a remote function call from a target system that stores data, responding to such remote function call, providing the target system, via the network, with first functionality for archiving at least a portion of the data stored in the target system to a storage system, and providing the target system, via the network, with second functionality for querying the data archived in the storage system.

[0125] The above-described systems or devices may be realized using hardware components or a combination of hardware and software components. For example, the devices and components described in the embodiments may be realized using one or more general-purpose or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the OS. The processing device may also access, record, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, a single processing device may be described. However, those skilled in the art will understand that a processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing device may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0126] Software may include computer programs, codes, instructions, or a combination of one or more of these, which may configure a processing device to operate as desired or may independently or collectively instruct the processing device. The software and / or data may be embodied in any type of machine, component, physical device, virtual device, computer storage medium, or device to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed and stored or executed in a distributed manner on computer systems connected by a network. The software and data may be stored on one or more computer-readable storage media.

[0127] Methods according to embodiments may be embodied in the form of program instructions executable by various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The medium may continuously store a computer-executable program or temporarily store it for execution or download. The medium may also be a variety of storage means or storage means in the form of a single piece of hardware or multiple hardware components. It is not limited to media directly connected to a computer system, but may also be distributed over a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; ROM, RAM, flash memory, and the like, configured to store program instructions. Other examples of media include recording media or storage media managed by application stores, other software distribution sites, servers, and the like. Examples of program instructions include not only machine language code, such as that generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

[0128] Although the embodiments have been described above based on limited examples and drawings, those skilled in the art will appreciate that various modifications and variations may be made from the above description. For example, the described techniques may be performed in an order different from that described, and / or the described system, structure, device, circuit, or other element may be coupled or combined in a manner different from that described, or may be substituted or replaced by other elements or equivalents, and still achieve suitable results.

[0129] Therefore, different embodiments that are equivalent to the claims are within the scope of the appended claims. [Explanation of symbols]

[0130] 310: Data Archive System 311: Data compression module 312: Inquiry module 313: Display and Control Module 314: Nearline interface module 320: Target system 321: Database 322: Control module 323: Nearline interface module 330: Containment System

Claims

1. A computer program recorded on a computer-readable recording medium to be coupled with a computer device to cause the computer device to execute a data archiving method, The data archiving method includes: receiving a remote function call from a target system storing data; providing, to the target system over a network, a first function in response to the invocation of the remote function, for the target system to compress and archive at least a portion of the data stored on the target system to a storage system; and providing a second capability to the target system via the network for the target system to query data archived in the storage system; Including, The step of providing the first function comprises: providing functionality for compressing at least a portion of the data stored in the target system's local database and archiving it in a table in the local database; A computer program characterized by:

2. A computer program recorded on a computer-readable recording medium in combination with a computer device to cause the computer device to execute a data archiving method, The data archiving method includes: receiving a remote function call from a target system storing data; providing, to the target system over a network, a first function in response to the invocation of the remote function, for the target system to compress and archive at least a portion of the data stored on the target system to a storage system; and providing a second capability to the target system via the network for the target system to query data archived in the storage system; Including, The step of providing the first function comprises: providing functionality for compressing at least a portion of the data stored in the target system's local database and archiving it to a table in the target system's external database; A computer program characterized by:

3. A computer program recorded on a computer-readable recording medium in combination with a computer device to cause the computer device to execute a data archiving method, The data archiving method includes: receiving a remote function call from a target system storing data; providing, to the target system over a network, a first function in response to the invocation of the remote function, for the target system to compress and archive at least a portion of the data stored on the target system to a storage system; and providing a second capability to the target system via the network for the target system to query data archived in the storage system; Including, The step of providing the first function comprises: a first function for controlling the target system to determine partitions for data records included in tables of a local database of the target system based on selection information of the data records; a first-second function for controlling the target system to compress data records for each partition to generate a compressed partition; a first-third function for controlling the target system to associate the compressed partition with a storage key that uniquely identifies the compressed partition and store the associated storage key in a compression table; and a first-fourth function of controlling the target system to associate the storage key with the selection information and store them in an index table of the local database; A computer program characterized by providing:

4. The selection information includes any field value of the corresponding data record; The first-fourth function controls the target system to associate the storage key with the arbitrary field value and store them in a group index table of the local database.

4. The computer program of claim 3,

5. The selection information includes information about the time of the corresponding data record; The first to fourth functions control the target system to associate the storage key with the information about time and store them in a period index table.

4. The computer program of claim 3,

6. The step of providing the first function comprises: A first to fifth function of controlling the target system to associate, for each data record included in the table, a primary key, key index information which is the position of the corresponding data record in a compressed partition which includes the corresponding data record and is compressed, and a storage key corresponding to the compressed partition which includes the corresponding data record, and store the associated information in a key index table.

4. The computer program of claim 3, further comprising:

7. The first to fifth functions are: For a second compressed partition generated by compressing data records in a linked table linked to the table by the primary key, controlling the target system to search for a data record having the same primary key as a data record included in the table among the data records included in the second compressed partition, and to further store subindex information, which is a position within the second compressed partition, for the searched data record in the key index table for the data record having the same primary key; 7. A computer program according to claim 6, characterized in that:

8. The step of providing the first function comprises: Functions 1-6 control the target system to delete the compressed data record from the table.

4. The computer program of claim 3, further comprising:

9. The step of providing the first function comprises: a first-seventh function controlling the target system to, in response to a restore request for the deleted data record, retrieve from the index table a storage key associated with identification information included in the restore request, retrieve from the compression table a compressed partition associated with the retrieved storage key, decompress the retrieved compressed partition to restore the deleted data record, and record the restored data record in the table based on the identification information; 9. The computer program of claim 8, further comprising:

10. The first-second function controls the target system to compress data records included in the determined partition as binary objects to generate the compressed partition.

4. The computer program of claim 3,

11. A computer program recorded on a computer-readable recording medium in combination with a computer device to cause the computer device to execute a data archiving method, The data archiving method includes: receiving a remote function call from a target system storing data; providing, to the target system over a network, a first function in response to the invocation of the remote function, for the target system to compress and archive at least a portion of the data stored on the target system to a storage system; and providing a second capability to the target system via the network for the target system to query data archived in the storage system; Including, The step of providing the second function comprises: a second-1 function for controlling the target system to receive search criteria including data record selection information; A second-2 function controls the target system to search for a storage key associated with the selection information included in the search criteria from an index table that stores the selection information of the data record in association with a storage key that uniquely identifies a compressed partition containing the corresponding data record in a local database of the target system; and a second-third function of controlling the target system to search for a compressed partition associated with the searched storage key in a compression table that stores the storage key and the compressed partition in association with each other; A computer program characterized by providing:

12. A computer program recorded on a computer-readable recording medium in combination with a computer device to cause the computer device to execute a data archiving method, The data archiving method includes: receiving a remote function call from a target system storing data; providing, to the target system over a network, a first function in response to the invocation of the remote function, for the target system to compress and archive at least a portion of the data stored on the target system to a storage system; and providing a second capability to the target system via the network for the target system to query data archived in the storage system; Including, The data archiving method includes: providing a third function to the target system via the network for managing the life cycle of data archived in the storage system; A computer program further comprising:

13. The step of providing the third function includes: A third-1 function for controlling the target system to archive and store data managed in a compressed state in a database table as a file when a storage period for the data archived in the storage system has expired; and a third-second function for controlling the target system to delete the data archived as the file; 13. A computer program according to claim 12, characterized in that it provides:

14. A data archiving method executed by a computer device including at least one processor, comprising: receiving, by said at least one processor, a call for a remote function from a target system storing data; providing, by the at least one processor, a first function to the target system over a network in response to the invocation of the remote function, for the target system to compress at least a portion of the data stored on the target system and archive it on a storage system; and providing, by the at least one processor, a second function to the target system via the network for the target system to query data archived in the storage system. Including, The step of providing the first function comprises: a first function for controlling the target system to determine partitions for data records included in tables of a local database of the target system based on selection information of the data records; a first-second function for controlling the target system to compress data records for each partition to generate a compressed partition; a first-third function for controlling the target system to associate the compressed partition with a storage key that uniquely identifies the compressed partition and store the associated storage key in a compression table; and a first-fourth function of controlling the target system to associate the storage key with the selection information and store them in an index table of the local database; A data archiving method comprising:

15. A data archiving method executed by a computer device including at least one processor, comprising: receiving, by said at least one processor, a call for a remote function from a target system storing data; providing, by the at least one processor, a first function to the target system over a network in response to the invocation of the remote function, for the target system to compress at least a portion of the data stored on the target system and archive it on a storage system; and providing, by the at least one processor, a second function to the target system via the network for the target system to query data archived in the storage system. Including, The step of providing the second function comprises: a second-1 function for controlling the target system to receive search criteria including data record selection information; A second-2 function controls the target system to search for a storage key associated with the selection information included in the search criteria from an index table that stores the selection information of the data record in association with a storage key that uniquely identifies a compressed partition containing the corresponding data record in a local database of the target system; and a second-third function of controlling the target system to search for a compression partition associated with the searched storage key in a compression table that stores the storage key and the compression partition in association with each other; A data archiving method comprising:

16. A computer-readable recording medium having recorded thereon a computer program for causing a computer device to execute the method according to claim 14 or 15.

Images