COMPUTER PROGRAM, DATA PROCESSING METHOD, RECORDING MEDIUM, AND COMPUTER DEVICE

By compressing and archiving data from volatile in-memory databases with unique keys and sorting information, the method addresses capacity limits and data loss risks, enabling efficient retrieval from non-volatile storage.

JP7616559B2Active Publication Date: 2025-01-17ARMIQ CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2020175202
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2020-08-27
Filing Date
2020-10-19
Publication Date
2025-01-17
Estimated Expiration
2040-10-19

AI Technical Summary

Technical Problem

The increasing volume of data in volatile in-memory databases exceeds storage capacity limits, posing a risk of data loss due to volatility and high storage costs, necessitating efficient archiving techniques.

Method used

Data records in volatile in-memory databases are compressed into partitions and stored in non-volatile storage, with unique storage keys and sorting information used to quickly retrieve archived data through index tables.

Benefits of technology

This method effectively reduces storage requirements and minimizes data loss by allowing quick retrieval of compressed data from non-volatile storage, optimizing database capacity and management.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007616559000001
    Figure 0007616559000001
  • Figure 0007616559000002
    Figure 0007616559000002
  • Figure 0007616559000003
    Figure 0007616559000003
Patent Text Reader

Abstract

To provide a method and apparatus for processing data of an in-memory database.SOLUTION: Provided are a data processing method and apparatus capable of rapidly searching for data compressed and stored in non-volatile storage means by compressing a data record, stored in a volatile in-memory database, in a partition unit, storing the compressed data record in the non-volatile storage means, and storing, in a table of the volatile in-memory database, a storage key to uniquely identify each of partitions stored in the non-volatile storage means and identification information used to determine a partition for a data record.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The following description relates to a method and apparatus for processing data in an in-memory database. [Background technology]

[0002] With the widespread use of the Internet and the development of data communication technology, means of producing and consuming data, such as PCs, laptops, and smartphones, have become widely used, and the amount of data produced and consumed in recent years has increased exponentially.

[0003] In order to efficiently classify and utilize such data, the method of managing data in a database is widely used by various companies, organizations, schools, etc. However, as the amount of data increases exponentially, the capacity of the database soon reaches its limit, and the reality is that continuing to expand the capacity of the database is limited in terms of cost and management.

[0004] Therefore, there is growing interest in technology that can efficiently archive databases while saving capacity, and in particular in archiving databases using data compression technology and searching for archived data.

[0005] Meanwhile, with the advent of in-memory database technology, important data is increasingly stored in volatile data storage means that have a high risk of data loss in the event of a failure. Therefore, if the data in such in-memory databases can be archived to non-volatile data storage means quickly and safely with moderate storage costs, the risk of data loss in the volatile data storage means can be minimized. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Korean Patent Publication No. 2014-0072929 Summary of the Invention [Problem to be solved by the invention]

[0007] Provided is a data processing method and device in which data records stored in a volatile in-memory database are compressed on a partition-by-partition basis and stored in a non-volatile storage means, and by storing in a table of the volatile in-memory database a storage key that uniquely identifies each partition stored in the non-volatile storage means and selection information used to determine the partition for the data record, the data compressed and stored in the non-volatile storage means can be quickly searched. [Means for solving the problem]

[0008] The present invention provides a data processing method executed by a computer device including at least one processor, the data processing method including a step of determining, by the at least one processor, partitions for data records included in a table of a volatile in-memory database based on selection information of the data records; a step of compressing the data records for each partition to generate compressed partitions by the at least one processor; a step of storing, by the at least one processor, the compressed partitions and a storage key that uniquely identifies the compressed partitions in a non-volatile storage means; and a step of storing, by the at least one processor, the storage key and the selection information in an index table of the volatile in-memory database in association with each other.

[0009] According to one aspect, in the classifying step, the selection information may include an arbitrary field value of the corresponding data record, and in the storing in the index table, the storage key and the arbitrary field value may be associated with each other and stored in the group index table.

[0010] According to another aspect, the selection information may include information regarding a time of the corresponding data record, and the storing in the index table may be characterized by storing the storage key and the information regarding the time in association with each other in a period index table.

[0011] According to another aspect, the data processing method may further include storing, for each data record included in the table, a primary key, key index information which is a position of the data record in a compressed partition compressed to include the data record, and a storage key corresponding to the compressed partition compressed to include the data record in a key index table in association with each other.

[0012] According to another aspect, the storing in the key index table may include searching for a data record whose primary key is equal to that of a data record included in the table among the data records included in the second compressed partition, the data record being generated by compressing data records in a linked table linked to the table by the primary key, and further storing sub-index information, which is a position within the second compressed partition for the searched data record, for the data record whose primary key is equal in the key index table.

[0013] According to another aspect, the data processing method may further include deleting the compressed data record from the table.

[0014] According to another aspect, the data processing method may further include the steps of: in response to a restore request for the deleted data record, searching the index table for a storage key associated with identification information included in the restore request; searching the non-volatile storage means for a compressed partition associated with the searched storage key; decompressing the searched compressed partition to restore the deleted data record; and recording the restored data record in the table based on the identification information.

[0015] According to another aspect, generating the compressed partition may include compressing data records included in the determined partition as binary objects to generate the compressed partition.

[0016] According to yet another aspect, the step of storing in the non-volatile storage means may be characterized by storing the compressed partition and the storage key separately in a non-volatile storage means of another computer device connected to the computer device via a network.

[0017] The present invention provides a data processing method executed by a computer device including at least one processor, the data processing method including: a step of receiving, by the at least one processor, search conditions including selection information of data records; a step of searching, by the at least one processor, for a storage key associated with the selection information included in the search conditions from an index table in a volatile in-memory database that stores the selection information of the data records in association with a storage key that uniquely identifies a compressed partition including the relevant data record; and a step of searching, by the at least one processor, for a compressed partition associated with the searched storage key in a non-volatile storage means that stores storage keys and compressed partitions in association with each other.

[0018] According to one aspect, the selection information may include an arbitrary field value of a corresponding data record, and the step of searching for the storage key may include searching for a storage key associated with the arbitrary field value included as the selection information of the search condition in a group index table that stores the storage key and the arbitrary field value in association with each other.

[0019] According to another aspect, the selection information may include information regarding a time of the corresponding data record, and the step of searching for a compressed partition associated with the searched storage key may include searching for a storage key associated with the time information included as the selection information of the search condition in a period index table that stores the storage key in association with the time information.

[0020] According to another aspect, the data processing method may further include, if the search condition further includes a primary key of the data record, searching a key index table for key index information and a storage key associated with the primary key further included in the search condition, wherein the key index table stores, for each data record included in any table on the volatile in-memory database, a primary key, key index information which is a position of the data record in a compressed partition compressed to include the data record, and a storage key corresponding to the compressed partition compressed to include the data record, in association with each other.

[0021] According to another aspect, the step of searching for a compressed partition associated with the retrieved storage key may be characterized by searching for a data record according to a search condition in the retrieved compressed partition further based on the retrieved key index information and the storage key.

[0022] According to another aspect, the key index table may further include sub-index information, which is a position of the data record in the second compressed partition, for a second compressed partition generated by compressing the data record in the concatenated table if a concatenated table exists that is concatenated with the arbitrary table by the primary key, and the step of searching for the key index information and the storage key may further search for sub-index information associated with a primary key further included in the search condition in the key index table, and the step of searching for a compressed partition associated with the searched storage key may further search for the data record according to the search condition in the second compressed partition based on the second compressed partition and the sub-index information.

[0023] According to yet another aspect, the non-volatile storage means may include non-volatile storage means of another computer device connected to the computer device via a network, and the step of searching for a compressed partition associated with the retrieved storage key in the non-volatile storage means may be characterized as searching for a compressed partition associated with the retrieved storage key from the non-volatile storage means of the other computer device via the network.

[0024] A computer program is provided, recorded on a computer-readable recording medium, for coupling with a computer device to cause the computer device to execute the method.

[0025] A computer-readable recording medium is provided on which a program for causing a computer device to execute the method is recorded.

[0026] The present invention provides a computer device comprising at least one processor implemented to execute computer-readable instructions, the at least one processor determining partitions for data records included in a table of a volatile in-memory database based on selection information for the data records, compressing the data records for each partition to generate compressed partitions, storing the compressed partitions in a non-volatile storage means in association with a storage key that uniquely identifies the compressed partition, and storing the storage key in an index table of the volatile in-memory database in association with the selection information.

[0027] The present invention provides a computer device comprising at least one processor implemented to execute computer-readable instructions, the at least one processor receiving search criteria including data record selection information, searching for a storage key associated with the selection information included in the search criteria from an index table that stores the data record selection information in a volatile in-memory database in association with a storage key that uniquely identifies a compressed partition including the relevant data record, and searching for a compressed partition associated with the searched storage key in a non-volatile storage means that stores storage keys in association with compressed partitions. Effect of the Invention

[0028] Data records stored in a volatile in-memory database are compressed on a partition-by-partition basis and stored in a non-volatile storage means, and a storage key that uniquely identifies each partition stored in the non-volatile storage means and selection information used to determine the partition for the data record are stored in a table in the volatile in-memory database, thereby enabling rapid search of data compressed and stored in the non-volatile storage means. [Brief description of the drawings]

[0029] [Figure 1]FIG. 1 illustrates an example of a network environment in accordance with an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a computing device in accordance with an embodiment of the present invention. [Diagram 3] FIG. 1 is a diagram generally illustrating a computer system for archiving in accordance with one embodiment of the present invention. [Figure 4] 4 is a flow chart illustrating an example data processing method for archiving data in accordance with an embodiment of the present invention. [Diagram 5] 10 is a flow chart illustrating another exemplary data processing method for archiving data in accordance with an embodiment of the present invention. [Figure 6] FIG. 2 is a diagram showing a first example of a structure of a compression table in one embodiment of the present invention. [Figure 7] FIG. 13 is a diagram showing a second example of the structure of a compression table in one embodiment of the present invention. [Figure 8] FIG. 13 is a diagram showing an example of the structure of a period index table in one embodiment of the present invention. [Figure 9] FIG. 13 is a diagram showing an example of the structure of a group index table in one embodiment of the present invention. [Figure 10] FIG. 13 is a diagram showing a second example of the structure of a compression table in one embodiment of the present invention. [Figure 11] 11 is a diagram illustrating an example of the structure of an index table in which a period index table and a group index table are combined in an embodiment of the present invention. FIG. [Figure 12] 10 is a flow chart illustrating yet another example of a data processing method for archiving data in accordance with an embodiment of the present invention. [Figure 13] 1A and 1B are diagrams showing examples of the structures of a compression table and a key index table in one embodiment of the present invention. [Figure 14] 11A and 11B are diagrams showing other examples of the structures of a compression table and a key index table in an embodiment of the present invention. [Figure 15] FIG. 2 illustrates an example data processing method for searching archived data in accordance with an embodiment of the present invention. [Figure 16] FIG. 13 illustrates an example of searching archived data in accordance with an embodiment of the present invention. [Figure 17] FIG. 13 illustrates an example of searching archived data in accordance with an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] The present invention may have various modifications and various embodiments, and a specific embodiment will be described in detail in the following detailed description with reference to the drawings. However, this is not intended to limit the present invention to the specific embodiment, but should be understood to include all modifications, equivalents, or replacements within the spirit and technical scope of the present invention. In describing each drawing, similar elements are given similar reference numerals.

[0031] The terms "first", "second", "A", "B", etc. are merely used to describe various components, and the components should not be limited by such terms. The terms are merely intended to distinguish one component from another. For example, the first component may be named as the second component, and similarly, the second component may be named as the first component, without departing from the scope of the present invention. The term "and / or" includes a combination of multiple related described items or any of multiple related described items.

[0032] A description that a certain component is "coupled" or "connected" to another component should be understood to include a state in which the component is directly coupled or connected to the other component, as well as a state in which there is another component between them. On the other hand, when a certain component is described as being "directly coupled" or "directly connected" to another component, it should be understood to include a state in which there is no other component between them.

[0033] The terms used in this application are merely for describing certain embodiments and are not intended to limit the present invention. Singular expressions include plural expressions unless otherwise clearly explained in the context. In this application, terms such as "include" or "have" are intended to specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood not to preclude the presence or additional possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0034] Unless otherwise defined, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Commonly used and predefined terms should be interpreted to have a meaning consistent with the meaning described in the context of the relevant art, and should not be interpreted as being ideal or overly formal unless expressly defined in this application.

[0035] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings.

[0036] A data processing system according to an embodiment of the present invention may be realized by at least one computer device, and a data processing method according to an embodiment of the present invention may be executed by at least one computer device included in the data processing system. A computer program according to an embodiment of the present invention may be installed and executed in the computer device, and the computer device may execute the data processing method according to an embodiment of the present invention according to the control of the computer program. The above-mentioned computer program may be recorded in a computer-readable recording medium in combination with the computer device to cause the computer device to execute the data processing method.

[0037] Fig. 1 is a diagram showing an example of a network environment in one embodiment of the present invention. The network environment in Fig. 1 shows an example including a plurality of electronic devices 110, 120, 130, 140, a plurality of servers 150, 160, and a network 170. Such Fig. 1 is merely an example for explaining the invention, and the number of electronic devices and the number of servers are not limited to those shown in Fig. 1. Furthermore, the network environment in Fig. 1 is merely an example for explaining one example of environments applicable to this embodiment, and the environment applicable to this embodiment is not limited to the network environment in Fig. 1.

[0038] The electronic devices 110, 120, 130, and 140 may be fixed terminals or mobile terminals realized by computer devices. Examples of the electronic devices 110, 120, 130, and 140 include smartphones, mobile phones, navigation systems, personal computers (PCs), notebook PCs, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), and tablets. As an example, FIG. 1 shows a smartphone as an example of the electronic device 110, but in the embodiment of the present invention, the electronic device 110 may mean one of various physical computer devices that can communicate with other electronic devices 120, 130, and 140 and / or servers 150 and 160 via a network 170 using a substantially wireless or wired communication method.

[0039] The communication method is not limited, and may include not only a communication method using a communication network (for example, a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network) that the network 170 can include, but also short-range wireless communication between devices. For example, the network 170 may include any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, and the like. Furthermore, the network 170 may include any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or hierarchical network, and the like.

[0040] Each of the servers 150, 160 may be realized by one or more computing devices that communicate with the multiple electronic devices 110, 120, 130, 140 via the network 170 to provide instructions, code, files, content, services, etc. For example, the server 150 may be a system that provides services (such as an archive service, a file distribution service, a map service, a content provision service, a group calling service (or an audio conference service), a messaging service, an email service, a social networking service, a translation service, a financial service, a payment service, a search service, etc.) to the multiple electronic devices 110, 120, 130, 140 connected via the network 170.

[0041] 2 is a block diagram showing an example of a computer device according to an embodiment of the present invention. Each of the electronic devices 110, 120, 130, and 140 and the servers 150 and 160 described above may be realized by a computer device 200 shown in FIG.

[0042] Such a computer device 200 may include a memory 210, a processor 220, a communication interface 230, and an input / output interface 240, as shown in FIG. 2. The memory 210 is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. Here, a permanent mass storage device such as a ROM or a disk drive may be included in the computer device 200 as a separate permanent storage device separate from the memory 210. The memory 210 may also store an operating system and at least one program code. Such software components may be loaded into the memory 210 from a computer-readable recording medium separate from the memory 210. Such separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. In another embodiment, the software components may be loaded into the memory 210 through a communication interface 230 that is not a computer-readable recording medium. For example, the software components may be loaded into the memory 210 of the computing device 200 based on a computer program installed by a file received over the network 170 .

[0043] The processor 220 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to the processor 220 by the memory 210 or the communication interface 230. For example, the processor 220 may be configured to execute instructions received according to program code stored in a storage device such as the memory 210.

[0044] The communication module 230 may provide a function for the computer device 200 to communicate with other devices (for example, the above-mentioned storage device) via the network 170. For example, requests, instructions, data, files, etc. generated by the processor 220 of the computer device 200 according to program codes recorded in a storage device such as the memory 210 may be transmitted to other devices via the network 170 under the control of the communication interface 230. Conversely, signals, instructions, data files, etc. from other devices may be received by the computer device 200 via the communication module 230 of the computer device 200 via the network 170. The signals, instructions, data, etc. received via the communication interface 230 may be transmitted to the processor 220 or the memory 210, and files, etc. may be recorded in a storage medium (the above-mentioned permanent storage device) that the computer device 200 may further include.

[0045] The input / output interface 240 may be a means for interfacing with the input / output device 250. For example, the input device may include a device such as a microphone, a keyboard, or a mouse, and the output device may include a device such as a display and a speaker. As another example, the input / output interface 240 may be a means for interfacing with a device in which functions for input and output are integrated into one, such as a touch screen. The input / output device 250 may be configured as a single device together with the computer device 200.

[0046] Also, in other embodiments, the computing device 200 may include fewer or more components than those of Figure 2. However, most of the prior art components need not be explicitly shown in the figures. For example, the computing device 200 may be implemented to include at least some of the input / output devices 250 described above, and may further include other components such as a transceiver, a database, etc.

[0047] Fig. 3 is a diagram generally illustrating a computer system for archiving according to an embodiment of the present invention. The computer system 300 of Fig. 3 may be implemented by the computer device 200 described above and may include an in-memory database 310, a non-volatile storage means 320, an archive component 330, and an interface component 340.

[0048] Data stored in the in-memory database 310 may be archived to the non-volatile storage means 320 by the processing of the archiving component 330. Additionally, data archived to the non-volatile storage means 320 may be restored to the in-memory database 310 or retrieved by the processing of the archiving component 330.

[0049] In some embodiments, non-volatile storage means 320 may exist outside computer system 300. For example, non-volatile storage means 320 may be included in another computer device that is connected to computer system 300 via a network. The other computer device may exist independently or may be part of a cloud system.

[0050] In addition, at least one of the functions for archiving data, restoring data, and searching data by the archive component 330 according to the embodiment may be provided by an external service. For example, the archive component 330 may archive, restore, and / or search data based on an API call by utilizing the functions provided by the external service.

[0051] FIG. 4 is a flow chart showing an example of a data processing method for archiving data according to an embodiment of the present invention. The data processing method according to this embodiment may be executed by a computer device 200 realizing the computer system 300 described in FIG. 3. In this case, the processor 220 of the computer device 200 may be realized to execute control instructions according to the operating system code and at least one computer program code included in the memory 210. Here, the processor 220 may control the computer device 200 so that the computer device 200 executes steps 410 to 440 included in the method of FIG. 4 according to the control instructions provided by the code recorded in the computer device 200.

[0052] In step 410, the computer device 200 may determine a partition for a data record included in a table of the volatile in-memory database based on the selection information of the data record. Here, the volatile in-memory database may correspond to the in-memory database 310 described in FIG. 3. The selection information may include time information of the data record and / or any field value of the data record. The computer device 200 may determine a partition for the data record based on such time information and / or field value. A table is a unit forming a basic structure for storing data in an in-memory database, and in step 410, may refer to a table to be archived to save space among multiple tables included in the in-memory database.

[0053] For example, the computer device 200 may select data records whose field values ​​are within a predetermined range as one partition. In this case, the field value may be determined by the field value of the field that is most frequently searched in the table. This is because, when the archived data is searched later, the index information generated based on the corresponding field value can be used to maximize search efficiency. As another example, the computer device may select data records whose information regarding time is within a predetermined range as one partition.

[0054] Also, a partition may be composed of a set of data records selected from all the data records included in a table. At least one partition may be generated, and if necessary, a partition may be generated by limiting the target to only some of the data records rather than the entire table. For example, data records after 2015 may be excluded from the table, and a partition for archiving may be generated by targeting only data records before 2015.

[0055] On the other hand, the number of data records contained in one partition may be determined by comprehensively analyzing and considering the total number of records contained in the table, the performance of the computer that searches the in-memory database, and frequently used search conditions in the in-memory database.

[0056] In another embodiment, if there is an excess partition among the selected partitions in which the number of data records exceeds a threshold, the excess partition may be separated into multiple partitions in which the number of records is equal to or less than the threshold. For example, the threshold, which is the number of data records contained in one partition, is set to 100,000. However, if the selected partition contains a number of data records that exceeds the threshold, this may cause computer overload and inefficiency, which may cause problems. Therefore, if one partition has more than 100,000 data records, it may be separated into multiple partitions of 100,000 units to generate multiple partitions. For example, if one partition has 250,000 data records, the computer device 200 may separate the excess partition into a total of three partitions, with two partitions each having 100,000 data records and one partition having 50,000 data records.

[0057] On the other hand, as described above, since the multiple partitions separated are classified according to the same classification criteria based on field values, there is no way to distinguish between such multiple partitions. Therefore, a sequence number (e.g., 1, 2, 3, 4, ...) may be assigned to each of the multiple separated record groups and further stored in the sequence number field of the index table. In this case, when searching archived data, it becomes possible to distinguish between each of the multiple separated partitions. Such a sequence number may correspond to the sequence described below.

[0058] In step 420, the computer device 200 may compress the data records for each partition to generate compressed partitions. As an example, the computer device 200 may compress the data records included in the determined partitions as binary objects to generate the compressed partitions.

[0059] As an example, the computer device 200 may first store the data records included in the compressed partition in a buffer in order to generate a compressed partition. The size of the buffer in which the data records are stored may be determined based on the structure of the table (the number, type, and size of the fields) and the threshold of the data records included in the compressed partition. For example, if the table includes a total of three fields, DATE (8 characters), NAME (30 characters), and AGE (4-byte integer), and the threshold for the number of data records included in the compressed partition is 100,000, when calculating one character as 2 bytes, the size of the buffer may be a minimum of 100,000 x (8 x 2 + 30 x 2 + 4) = 8 million bytes (about 8 megabytes). At this time, the computer device 200 may sequentially read all the data records included in the compressed partition and the field values ​​of the data records, and sequentially store them in the buffer.

[0060] Thereafter, the computer device 200 may compress the data stored in the buffer to generate a compressed partition. The compressed partition may be a resultant product in the form of a binary object generated by compressing the data stored in the buffer. At this time, a lossless compression algorithm such as ZIP, CTW, LZ77, LZW, gzip, bzip2, DEFLATE, etc. may be used to prevent loss due to compression.

[0061] At this time, the computer device 200 may generate a storage key that is uniquely assigned to each of the generated compressed partitions.

[0062] In step 430, the computer device 200 may store the compressed partitions and the storage keys that uniquely identify the compressed partitions in a non-volatile storage means in association with each other. As described above, the non-volatile storage means may be included in the computer device 200, or may be included in another computer device connected to the computer device 200 via a network. For example, the compressed partitions may be stored in a compression table in which data compressed and archived in partition units is stored in the non-volatile storage means. The compression table may include a field for storing the compressed partitions generated by compressing each partition, and a field for storing a storage key uniquely assigned to correspond to the corresponding compressed partition. The storage key is a key including a value uniquely assigned to each compressed partition, and the value of the storage key shared for each compressed partition may be stored in a field of the compression table corresponding to the storage key. Also, there may be one or more fields corresponding to the storage key, and a unique storage key may be formed for each compressed partition when the values ​​of the storage keys stored in a distributed manner in the one or more fields are combined.

[0063] In step 440, the computer device 200 may store the storage key and the selection information in an index table of the volatile in-memory database in association with each other. As an example, if the selection information includes an arbitrary field value of the corresponding data record, the computer device 200 may store the storage key and the arbitrary field value in association with each other in a group index table in step 440. The storage key and the field value stored in the group index table may be used as an index for searching the compressed and stored data record according to a search condition including the arbitrary field value. As another example, if the selection information includes information regarding the time of the data record, the computer device 200 may store the storage key and the information regarding the time in association with each other in a period index table. The storage key and the information regarding the time stored in the period index table may be used as an index for searching the compressed and stored data record according to a search condition including the arbitrary time information. In other words, an index table including a group index table and / or a period index table may be used to obtain a storage key corresponding to the field values ​​and / or time information contained in the search criteria, and the storage key may be used to obtain a compressed partition corresponding to the storage key in (a compressed table contained in) the non-volatile storage means.

[0064] 5 is a flowchart showing another example of a data processing method for archiving data according to an embodiment of the present invention. The data processing method according to this embodiment may further include step 510 after step 440 described in FIG.

[0065] In step 510, the computer device 200 may delete the compressed data records from the table. Since the purpose of compressing and archiving the database is to save storage space in the database, the computer device 200 may save storage space in the in-memory database by deleting the archived data records from the table. However, in some embodiments, the compressed data records may not be deleted from the table immediately, but may be deleted from the table after a certain period of time has passed.

[0066] Meanwhile, the deleted data record may be restored to the corresponding table later. For example, in response to a restoration request for the deleted data record, the computer device 200 may search an index table for a storage key associated with the identification information included in the restoration request, and search a compressed partition associated with the searched storage key in the non-volatile storage means (or a compression table included in the non-volatile storage means). Thereafter, the computer device 200 may decompress the searched compressed partition to restore the deleted data record, and record the restored data record in a table based on the identification information. At this time, information in a key index table described below may be used to identify a specific data record requested to be restored among the data records included in the compressed partition.

[0067] FIG. 6 is a diagram showing a first example of the structure of a compressed table in an embodiment of the present invention. Table 610 in FIG. 6 includes Doc.No. field 611, Date field 612 for time, and Col1 field 613 for a specific attribute. In this case, computer device 200 may generate compressed partitions by classifying and compressing data records of table 610 based on the field value of Date field 612 or Col1 field 613 of table 610, which is information related to time, as selection information. In this case, computer device 200 may generate compressed table 620 by storing a storage key for uniquely identifying a compressed partition and the corresponding compressed partition in association with each other. For example, compressed table 620 according to the embodiment of FIG. 6 may include OBJECT ID field 621 having a storage key as a field value, and COMPRESSED DATA field 622 having a compressed partition as a field value.

[0068] FIG. 7 is a diagram showing a second example of the structure of a compression table in one embodiment of the present invention, FIG. 8 is a diagram showing an example of the structure of a period index table in one embodiment of the present invention, and FIG. 9 is a diagram showing an example of the structure of a group index table in one embodiment of the present invention.

[0069] FIG. 7 illustrates another embodiment of generating a compressed table 700 based on the table 610 illustrated in FIG. 6. For example, the computer device 200 may generate a compressed partition by classifying and compressing the data records of the table 610 based on the field value of the Date field 612 of the table 610, which is information related to time, as the selection information. The computer device 200 may also generate the compressed table 700 by storing the selection information and the corresponding compressed partition in association with each other. For example, the compressed table 700 according to the embodiment of FIG. 7 may include a PERIOD field 710 having information related to time as a field value, and a COMPRESSED DATA field 720 having a compressed partition as a field value.

[0070] FIG. 8 shows an example of a period index table 800 that can be generated and utilized when the compression table 620 includes compressed partitions generated by classifying and compressing data records based on the field value (information about time) of the Date field 612. In this case, the period index table 800 may include a PERIOD field 810 having information about time as a field value, and an OBJECT ID field 820 having a storage key as a field value. For example, when the computer device 200 receives a search condition including information about time (for example, "2020.01") as selection information, the computer device 200 may search for a corresponding storage key (for example, a storage key "O0001" corresponding to the information about time "2020.01" in the period index table 800) using the information about time included in the search condition, and the searched storage key can be used to search for a compressed partition corresponding to the storage key in the compression table 620 (for example, a compressed partition of "50,000Rows" corresponding to the storage key "O0001" in the compression table 620).

[0071] 9 also illustrates an example of a group index table 900 that may be generated and utilized when the compressed table 620 includes compressed partitions generated by sorting and compressing data records based on the field value of the Col1 field 613. In this case, the group index table 900 may include a PERIOD field 910 that has the field value of the Col1 field 613 as its field value, and an OBJECT ID field 920 that has the storage key as its field value. For example, when the computer device 200 receives search criteria that include a field value of the Col1 field 613 (for example, "1000") as selection information, the computer device 200 may use the field value included in the search criteria to search for a corresponding storage key (for example, storage key "O0001" corresponding to the field value "1000" in the group index table 900), and then use the searched storage key to search for a compressed partition corresponding to the storage key in the compression table 620 (for example, compressed partition of "50,000Rows" corresponding to storage key "O0001" in the compression table 620).

[0072] FIG. 10 is a diagram showing a second example of the structure of a compression table in one embodiment of the present invention, and FIG. 11 is a diagram showing an example of the structure of an index table in a form in which a period index table and a group index table are combined in one embodiment of the present invention.

[0073] Fig. 10 illustrates another embodiment for generating a compressed table 1000 based on the table 610 illustrated in Fig. 6. For example, the computer device 200 may generate a compressed partition by classifying and compressing the data records of the table 610 based on two field values, that is, the field value of the Date field 612 and the field value of the Col1 field 613 of the table 610, which are information related to time.

[0074] As a more specific example, the computer device 200 compresses a data record in which the field value of the Data field 612 is “2002.01” but the field value of the Col1 field 613 is “1000” to generate a first compressed partition, compresses a data record in which the field value of the Data field 612 is “2002.01” but the field value of the Col1 field 613 is “2000” to generate a second compressed partition, and compresses a data record in which the field value of the Data field 612 is “2002.02” but the field value of the Col1 field 613 is “1000” to generate a third compressed partition. The fourth compressed partition may be generated by compressing a data record in which the field value of the Data field 612 is "2002.02" but the field value of the Col1 field 613 is "2000", the fifth compressed partition may be generated by compressing a data record in which the field value of the Data field 612 is "2002.03" but the field value of the Col1 field 613 is "1000", and the sixth compressed partition may be generated by compressing a data record in which the field value of the Data field 612 is "2002.03" but the field value of the Col1 field 613 is "2000".

[0075] In this case, the computer device 200 may generate the compressed table 1000 by storing a storage key for uniquely identifying the compressed partition and the corresponding compressed partition in association with each other. For example, the compressed table 1000 according to the embodiment of FIG. 10 may include an OBJECT ID field 1010 having the storage key as a field value, and a COMPRESSED DATA field 1020 having the compressed partition as a field value.

[0076] 11 shows an example of an index table 1100 in which a period index table and a group index table are combined. In this case, the index table 1100 may include a PERIOD field 1110 having information related to time as a field value, a Col1 field 1120 having the field value of the Col1 field 613 as its own field value, and an OBJECT ID field 1130 having a storage key as a field value. For example, when the computer device 200 receives search criteria that include time-related information (for example, "2020.02") and a field value of the Col1 field 613 (for example, "1000") as selection information, the computer device 200 may search the index table 1100 for a storage key (for example, storage key "O0003" in the index table 1100) that satisfies both the time-related information and the field value included in the search criteria, and the searched storage key can be used to search for a compressed partition corresponding to the storage key in the compression table 1000 (for example, a compressed partition of "30,000Rows" corresponding to the storage key "O0003" in the compression table 1000).

[0077] As described above, the data processing method according to an embodiment of the present invention has an effect of significantly reducing the capacity of the in-memory database by archiving the in-memory database through a compression procedure. Also, the index table included in the in-memory database is connected to an original table in a non-volatile storage means in which compressed data is stored, and the archived data is later restored or searched, thereby maximizing search efficiency.

[0078] On the other hand, the index table included in the volatile in-memory database may be backed up in a separate non-volatile storage means since there is a risk of data damage due to the characteristics of the in-memory database. The non-volatile storage means in which the index table is backed up may be the same as the non-volatile storage means in which the compressed partition is stored, or may be different. For example, it is conceivable that the compressed partition is stored in a non-volatile storage means included in an external system of the computer device 200, and the index table is backed up in a local storage of the computer device 200.

[0079] 12 is a flowchart illustrating another example of a data processing method for archiving data according to an embodiment of the present invention. The data processing method according to this embodiment may further include step 1210 after step 440 described in FIG.

[0080] In step 1210, the computer device 200 may store in a key index table, for each data record included in the table, a primary key, key index information which is the position of the corresponding data record in a compressed partition which contains the corresponding data record and is compressed, and a storage key corresponding to the compressed partition which contains the corresponding data record, in association with each other.

[0081] A primary key may refer to a value in a field that has a value that uniquely identifies each record in an in-memory database, and is also called a base key, primary key, or unique key. A table may have one or more primary keys. Key index information is information about where a data record having a specific primary key value is stored in a compressed partition. For example, information about the storage order, such as the data record being the 1000th data record stored among information about 100,000 data records contained in the compressed partition, may be stored in the key index information.

[0082] Meanwhile, the reason why the primary key is stored in the key index table is to enable the table to be searched to be searched to be directly searched by the primary key as well as by other field values ​​and time information. That is, the key index table may be used when a user inputs a specific primary key and searches a table for a data record having the specific primary key. More specifically, the computer device 200 may search the key index table for key index information and storage key of a data record having a specific primary key. In this case, the computer device 200 may use the acquired storage key to acquire a compressed partition corresponding to the storage key in the non-volatile storage means (or a compressed table included in the non-volatile storage means), and use the key index information to search for a specific data record desired by the user in the compressed partition. As described above, the key index information of such a key index table may be utilized to identify a data record having a specific condition from among the data records included in the compressed partition when restoring the data record having a specific condition to a table.

[0083] FIG. 13 is a diagram showing an example of the structure of a compression table and a key index table in one embodiment of the present invention.

[0084] Compressed table 1310 may include an OBJECT ID field 1311 having a storage key as a field value, a SEQ field 1312 having a processing order (sequence) of the target table as a field value, and a COMPRESSED DATA field 1313 having a compressed partition as a field value. When a parent table and a child table exist, the sequence may define a processing order in which the parent table is extracted first and the child table is processed using the extracted data of the parent table.

[0085] As described above, the key index table 1320 may include a Doc.No. field 1321 having a primary key as a field value, an OBJECT ID field 1322 having a storage key as a field value, and a Key Location info. field 1323 having key index information as a field value. For example, in the key index information 1@1001, the "1" before the "@" may indicate a sequence corresponding to the field value of the SEQ field 1312, and the "1001" after the "@" may indicate the 1001st data record among the data records contained in the corresponding compressed partition. As a more specific example, the first record in the key index table 1320 may indicate that a data record with a primary key of "1" is included as the 1001st data record among the data records of a compressed partition with a storage key of "O0001" and a sequence of "1". Similarly, the second record in key index table 1320 may indicate that the data record with primary key “2” is included as the 2001st data record among the data records in the compressed partition with storage key “O0001” and sequence “2.”

[0086] In this manner, the key index information may include information regarding the location of a particular data record within a compressed partition, and by utilizing a key index table (such as key index table 1320 in FIG. 13) that includes such key index information, a primary key, and a storage key, the number of data records that are queried based on a user's search criteria can be reduced.

[0087] In another embodiment, the computer device 200 may search for a data record having a primary key equal to that of a data record included in the first table among the data records included in the second compressed partition for a second compressed partition generated by compressing data records in a linked table linked to the first table (the table described in step 410 as an example) by a primary key, and may further store sub-index information, which is a position in the second compressed partition, for the data record having the equal primary key in the key index table for the searched data record. The linked table is a table linked to the first table by a primary key. That is, the primary key exists in both the first table and the linked table. The second compressed partition may be data generated by compressing data records in the linked table when a linked table linked to the first table by a primary key exists. In this case, the second compressed partition may be generated in the same manner as the compressed partition described in FIG. 4, and may be stored in a non-volatile storage means (or a compressed table included in the non-volatile storage means) together with a unique storage key, like the compressed partition. The sub-index information is information about where in the second compressed partition a data record having a specific primary key is stored. For example, information about the storage order, such as the data record being the 1000th stored data record among information about 100,000 data records included in the second compressed partition, may be stored as the sub-index information. For example, a linked table linked to the first table by a primary key exists in the in-memory database, and a user may require field value information for a field that does not exist in the first table but does exist in the linked table. In this case, the computer device 200 may further store sub-index information for data records having the same primary key in the key index table so that the linked table can be searched later.

[0088] In another embodiment, when there are multiple concatenated tables for the first table, the computer device 200 may collect and compress the sub-index information for each concatenated table, and then store it as new sub-index information in the key index table. For example, the computer device 200 may collect all sub-index information for positions in two or more second compressed partitions for data records having the same primary key in the concatenated table, compress the collected values, and store them as new sub-index information for the data records including the same primary key value in the key index table.

[0089] FIG. 14 is a diagram showing another example of the structure of the compression table and the key index table in one embodiment of the present invention.

[0090] Compressed table 1410 may include an OBJECT ID field 1411 having a storage key as a field value, a TABLE field 1412 having a table identifier as a field value, a SEQ field 1413 having a sequence as a field value, and a COMPRESSED DATA field 1414 having a compressed partition as a field value. TABLE field 1412 may include a table identifier as a field value, which may identify from which table the corresponding compressed partition contains data records extracted.

[0091] The key index table 1420 of this embodiment may include a Doc.No. field 1421 having a primary key as a field value, an OBJECT ID field 1422 having a storage key as a field value, a Key Location info. field 1423 having key index information as a field value, and a Sub Location info. field 1424 having sub index information as a field value.

[0092] For example, the first record of key index table 1420 may indicate that a data record with primary key "1" is included as the 10001st data record among the data records in a compressed partition with storage key "O0001" and sequence "1." In this case, the field value "TAB1@1001-2 / TAB2@2001-3" of Sub Location info. field 1424 indicates the location in the second compressed partition generated for the concatenated table of the data record with primary key "1." For example, in the field value "TAB1@1001-2 / TAB2@2001-3", "TAB1" and "TAB2" before "@" may mean concatenated tables linked by equal primary keys, and "1001-2" after "@" indicates two data records (the 1001st data record (first data record) and the 1002nd data record (second data record)) from the 1001st data record among the data records included in the second compressed partition for the concatenated table "TAB1". Furthermore, "2001-3" after "@" indicates three data records (the 2001st data record (third data record), the 2002nd data record (fourth data record), and the 2003rd data record (fifth data record)) from the 2001st data record among the data records included in the second compressed partition for the concatenated table "TAB2". In this case, the first to fifth data records may all be identified by the same primary key.

[0093] 15 is a diagram illustrating an example of a data processing method for searching archived data according to an embodiment of the present invention. The data processing method according to the present embodiment may also be performed by the computing device 200.

[0094] In step 1510, the computer device 200 may receive search criteria including selection information for data records. Such selection information may include information about any field values ​​of the data records to be searched and / or the time of the corresponding data records. The field values ​​and / or time information included in the selection information may be included in the form of a range.

[0095] In step 1520, the computer device 200 may search for a storage key associated with the selection information included in the search condition from an index table that stores the selection information of the data record in a volatile in-memory database in association with a storage key that uniquely identifies a compressed partition including the corresponding data record. As described above, the index table may include a group index table and / or a period index table, and the group index table may store a specific field value in association with a storage key, and the period index table stores information related to time in association with a storage key. Thus, the computer device 200 may search for a storage key corresponding to the field value and / or information related to time included in the selection information in the group index table and / or the period index table. For example, when the selection information includes an arbitrary field value of the data record, the computer device 200 may search for a storage key associated with an arbitrary field value included as the selection information of the search condition in a group index table that stores a storage key in association with an arbitrary field value. As another example, if the selection information includes information regarding the time of the data record, the computer device 200 may search for a storage key associated with the time information included as the selection information of the search condition in a period index table that stores storage keys and time information in association with each other.

[0096] In step 1530, the computer device 200 may search for a compressed partition associated with the searched storage key in a non-volatile storage means that stores storage keys and compressed partitions in association with each other. As described above, the non-volatile storage means (or the compression table included in the non-volatile storage means) stores compressed partitions in association with storage keys that uniquely identify the compressed partitions, so that the computer device 200 can search for the corresponding compressed partition in such a non-volatile storage means by the storage key.

[0097] As described above, when the key index table is further utilized, the user may utilize the primary key for the search. As described above, the key index table may store, for each data record included in any table on the volatile in-memory database, a primary key, key index information that is the position of the corresponding data record in a compressed partition that contains the corresponding data record and is compressed, and a storage key corresponding to the compressed partition that contains the corresponding data record, in association with each other. In this case, if the search condition further includes a primary key of the data record, the computer device 200 may search the key index information and storage key associated with the primary key further included in the search condition in the key index table. After this, the computer device 200 can search for a specific data record according to the search condition from the compressed partition searched in step 1530 based on the searched key index information and storage key.

[0098] In addition, if there is a concatenated table that is connected to an arbitrary table by a primary key, the key index table may further include sub-index information that is the position of the data record in the second compressed partition for the second compressed partition generated by compressing the data record in the concatenated table. Therefore, if the search condition further includes a primary key, the computer device 200 may further search for sub-index information associated with the primary key further included in the search condition in the key index table, and further search for the data record according to the search condition in the second compressed partition based on the second compressed partition and the sub-index information, thereby obtaining not only the field value of the first table to be searched for a specific data record, but also the field value of the concatenated table that is connected to the first table by the primary key.

[0099] Meanwhile, as described above, the non-volatile storage means may include a non-volatile storage means of another computer device connected via a network to the computer device 200. In this case, in step 1530, the computer device 200 may retrieve a compressed partition associated with the storage key retrieved in step 1520 from the non-volatile storage means of the other computer device via the network.

[0100] 16 and 17 are diagrams illustrating an example of searching archived data in one embodiment of the present invention.

[0101] FIG. 16 illustrates an example of searching for archived data in a compression table 1620 according to a query 1610. In the embodiment of FIG. 16, the compression table 1620 includes a PERIOD field 1621, a COL1 field 1622, a TABLE field 1623, an OBJECT ID field 1624, a SEQ field 1625, and a COMPRESSED DATA field 1626 in combination with an index table. In some embodiments, the PERIOD field 1621 and the COL1 field 1622 may be in a separate index table. In this case, an OBJECT ID field 1624 may be present in each of the compression table 1620 and the index table to link the two tables. In some embodiments, the TABLE field 1623 and the SEQ field 1625 may also be present in the index table.

[0102] In this case, the query 1610 may represent an instruction to search the table "TAB1" for a data record in which the field value of the PERIOD field 1621 is "2002.01" and the field value of the COL1 field 1622 is "1000". In this case, the computer device 200 can confirm that the data record corresponding to the query 1610 in the compression table 1620 is a compressed partition stored in the COMPRESSED DATA field 1626 of the first record in the compression table 1620. Therefore, the computer device 200 can decompress the corresponding compressed partition and provide the data record contained in the corresponding compressed partition (the data record of "50,000 rows") as a search result.

[0103] 17 shows an example of searching for archived data in a compression table 1620 according to a query 1710. In the embodiment of FIG. 17, the query 1710 utilizes a primary key as a search condition, so a key index table 1720 may be utilized. The key index table 1720 includes a Doc.No. field 1721, an OBJECT ID field 1722, a Key Location Info. field 1723, and a Sub Location Info. field 1724.

[0104] In this case, the query 1710 may mean an instruction to search for a data record having a field value of "1" in the Doc.No. field 1721 as a primary key from the table "TAB1" and the table "TAB2". In this case, the computer device 200 can confirm the first record having a field value of "1" in the Doc.No. field 1721 in the key index table 1720, and can search for a data record having a primary key of "1" in the compression table 1620 based on the field value of the Sub Location Info. field 1724 of the first record. For example, the computer device 200 may extract a data record having a primary key of "1" in the compression table 1620 based on the sub-index information "TAB1@1001-2 / TAB2@2001-3" of the first record in the key index table 1720 having a field value of "1" in the Doc.No. field 1721. At this time, it is not necessary to search all the data records in the compressed partition based on the location included in the sub-index information, and it is possible to easily and quickly search for a data record having a specific primary key value.

[0105] Thus, according to an embodiment of the present invention, data records stored in a volatile in-memory database are compressed on a partition-by-partition basis and stored in a non-volatile storage means, and by storing a storage key that uniquely identifies each partition stored in the non-volatile storage means and the selection information used to determine the partition for the data record in a table in the volatile in-memory database, data compressed and stored in the non-volatile storage means can be quickly searched.

[0106] The above-described systems or devices may be realized by hardware components or a combination of hardware and software components. For example, the devices and components described in the embodiments may be realized using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications that run on the OS. The processing device may also respond to the execution of the software to access, record, manipulate, process, and generate data. For convenience of understanding, a single processing device may be described as being used, but those skilled in the art will understand that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0107] The software may include computer programs, codes, instructions, or a combination of one or more of these, and may configure a processing device to operate as desired or instruct the processing device, either independently or collectively. The software and / or data may be embodied in any type of machine, component, physical device, virtual device, computer storage medium, or device to be interpreted based on the processing device or to provide instructions or data to the processing device. The software may be distributed and stored or executed in a distributed manner on computer systems connected by a network. The software and data may be stored on one or more computer-readable storage media.

[0108] The method according to the embodiment may be realized in the form of program instructions executable by various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The medium may be one that continuously records a computer-executable program, or one that temporarily records it for execution or download. The medium may be one of various storage means or storage means in the form of a single or multiple hardware devices combined, and is not limited to a medium directly connected to a certain computer system, but may be one that is distributed over a network. Examples of the medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and ROMs, RAMs, flash memories, and the like, which are configured to record program instructions. Other examples of the medium include recording media or storage media managed by application stores that distribute applications, or other sites that supply or distribute various software, servers, and the like. Examples of the program instructions include not only machine language codes such as those generated by compilers, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0109] Although the embodiments have been described based on the limited embodiments and drawings as above, those skilled in the art will be able to make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from that described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a manner different from that described, or may be counterbalanced or replaced by other components or equivalents, and still achieve suitable results.

[0110] Therefore, different embodiments are within the scope of the appended claims provided they are equivalent to the claims. [Explanation of symbols]

[0111] 300: Computer Systems 310: In-memory database 320: Non-volatile storage means 330: Archive Component 340: Interface Components 350:Client

Claims

1. A computer program recorded on a computer-readable recording medium for causing the computer to execute a data processing method in combination with a computer device, The data processing method includes: determining partitions for data records included in a table of the volatile in-memory database based on the selection information of the data records; compressing the data records for each partition to generate a compressed partition; storing in a non-volatile storage means in association with said compressed partition a storage key that uniquely identifies said compressed partition; storing the storage key and the selection information in an index table of the volatile in-memory database in association with each other; A computer program comprising:

2. The selection information includes any field values ​​of the corresponding data record; The step of storing in the index table comprises: storing said storage key and said arbitrary field value in a group index table in association with each other; 2. A computer program according to claim 1, characterized in that

3. The selection information includes information regarding the time of the corresponding data record, The step of storing in the index table comprises: storing said storage key and said information related to time in a period index table in association with each other; 2. A computer program according to claim 1, characterized in that

4. The data processing method includes: storing, in a key index table, a primary key for each data record included in the table, key index information which is a position of the corresponding data record in a compressed partition which includes the corresponding data record and is compressed, and a storage key corresponding to the compressed partition which includes the corresponding data record, in association with each other; 2. The computer program of claim 1, further comprising:

5. The step of storing in the key index table comprises: For a second compressed partition generated by compressing data records in a linked table linked to the table by the primary key, searching for a data record, the primary key of which is equal to that of a data record included in the table, among the data records included in the second compressed partition; and further storing sub-index information, which is a position in the second compressed partition for the searched data record, for the data record having the same primary key in the key index table; 5. A computer program according to claim 4, comprising:

6. The data processing method includes: removing the compressed data record from the table.

2. The computer program of claim 1, further comprising:

7. The step of generating a compressed partition comprises: compressing the data records included in the determined partition as binary objects to generate the compressed partition.

2. A computer program according to claim 1, characterized in that

8. The step of storing in the non-volatile storage means comprises: storing said compressed partition and said storage key separately in a non-volatile storage means of another computer device connected to said computer device via a network; 2. A computer program according to claim 1, characterized in that

9. A computer program recorded on a computer-readable recording medium for causing the computer to execute a data processing method in combination with a computer device, The data processing method includes: receiving search criteria including selection information for the data records; searching for a storage key associated with the selection information included in the search condition from an index table that stores the selection information of the data record in association with a storage key that uniquely identifies a compressed partition including the corresponding data record in a volatile in-memory database; searching for a compressed partition associated with the searched storage key in a non-volatile storage means for storing the storage key and the compressed partition in association with each other; A computer program comprising:

10. The selection information includes any field values ​​of the corresponding data record; The step of retrieving the stored key comprises: searching for a storage key associated with an arbitrary field value included as selection information of the search condition in a group index table that stores the storage key and the arbitrary field value in association with each other; 10. A computer program according to claim 9, characterized in that

11. The selection information includes information regarding the time of the corresponding data record, The step of retrieving a compressed partition associated with the retrieved storage key comprises: Searching for a storage key associated with information related to time included as selection information of the search condition in a period index table that stores the storage key and the information related to time in association with each other.

10. A computer program according to claim 9, characterized in that

12. The data processing method includes: searching for key index information and a storage key associated with the primary key further included in the search condition in a key index table if the search condition further includes a primary key of the data record; Further comprising: The key index table stores, for each data record included in an arbitrary table on the volatile in-memory database, a primary key, key index information which is the position of the corresponding data record in a compressed partition which contains the corresponding data record and is compressed, and a storage key which corresponds to the compressed partition which contains the corresponding data record, in association with each other.

10. A computer program according to claim 9, characterized in that

13. The step of retrieving a compressed partition associated with the retrieved storage key comprises: searching the retrieved compressed partition for a data record according to a search criterion, further based on the retrieved key index information and a storage key; 10. A computer program according to claim 9, characterized in that

14. the non-volatile storage means includes a non-volatile storage means of another computer device connected to the computer device via a network, retrieving a compressed partition associated with the retrieved storage key in the non-volatile storage means, retrieving a compressed partition associated with the retrieved storage key from a non-volatile storage means of the other computer device via the network.

10. A computer program according to claim 9, characterized in that

15. 1. A data processing method executed by a computing device including at least one processor, comprising: determining, by the at least one processor, partitions for data records included in a table of the volatile in-memory database based on the selection information for the data records; compressing, by the at least one processor, the data records for each of the partitions to generate a compressed partition; storing, by said at least one processor, in a non-volatile storage means, in association with said compressed partition a storage key that uniquely identifies said compressed partition; storing, by said at least one processor, said storage key and said selection information in an index table of said volatile in-memory database in association with each other. (c) a method for processing data,

16. 1. A data processing method executed by a computing device including at least one processor, comprising: receiving, by the at least one processor, search criteria including data record selection information; searching, by the at least one processor, for a storage key associated with the selection information included in the search condition from an index table that stores, in a volatile in-memory database, the selection information of the data record in association with a storage key that uniquely identifies a compressed partition including the corresponding data record; and searching, by said at least one processor, for a compressed partition associated with said retrieved storage key in a non-volatile storage means for storing storage keys and compressed partitions in association with each other. (c) a method for processing data,

17. A computer-readable recording medium having a computer program recorded thereon for causing a computer device to execute the method according to claim 15 or 16.

18. At least one processor implemented to execute computer readable instructions Including, by said at least one processor, determining partitions for data records included in a table of the volatile in-memory database based on the selection information of the data records; compressing the data records for each of the partitions to generate a compressed partition; storing said compressed partition and a storage key that uniquely identifies said compressed partition in a non-volatile storage means in association with each other; storing the storage key and the selection information in an index table of the volatile in-memory database in association with each other; A computer device comprising:

19. At least one processor implemented to execute computer readable instructions Including, by said at least one processor, receiving search criteria including selection information for the data records; searching for a storage key associated with the selection information included in the search criteria from an index table that stores the selection information of the data record in association with a storage key that uniquely identifies a compressed partition including the corresponding data record on a volatile in-memory database; searching for a compressed partition associated with the searched storage key in a non-volatile storage means for storing the storage key and the compressed partition in association with each other; A computer device comprising:

Citation Information

Patent Citations

  • Managing data storage for range-based searches

    JP2014524090A

  • Design and implementation of a clustered in-memory database

    JP2017504874A

  • Method and apparatus for archiving a database that generates index information, and method and apparatus for searching an archived database that includes index information

    JP2019512124A

  • KR2014-0072929

  • Database Partition Management System

    US20200167330A1