Method and system for integrating inverted indexes in database and medium

Abstract

The invention relates to a method and system for integrating inverted indexes in a database and a medium, and the method comprises the steps: setting a corresponding inverted index for each Level File data partition in a preset database; in the process that data in a memory buffer area of a preset database is written into a Level File data partition, a data index of the data is added into a reverse index corresponding to the Level File data partition, and the reverse index is stored into the Level File data partition; and when the preset database receives a query request, performing data retrieval through the inverted index. According to the method and the device, a full-quantity construction mode of the reverse index is changed into an increment construction mode, so that the construction mode of the reverse index is matched with a data writing mode of the database, the maintenance of the reverse index is simplified, and the problem of how to integrate the reverse index into the database is solved.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a method, system and medium for integrating an inverted index into a database. Background Art

[0002] The most common implementation method for a database that supports inverted indexes is to maintain an independent inverted index outside the database, and use data synchronization tools to maintain data consistency between the database and the inverted index. The disadvantage of this implementation method is that it cannot achieve strong data consistency, and the text data needs to be maintained in the database and in the inverted index, resulting in data redundancy and wasted storage space.

[0003] Integrating the inverted index into the database can ensure strong data consistency and store only one copy of the text data. However, difficulties still exist, such as the mismatch between the database data writing mode and the inverted index construction mode.

[0004] Currently, no effective solution has been proposed for the problem of how to integrate the inverted index into the database in the related technology. Summary of the invention

[0005] The embodiments of the present application provide a method, system and medium for integrating an inverted index into a database, so as to at least solve the problem of how to integrate an inverted index into a database in the related art.

[0006] In a first aspect, an embodiment of the present application provides a method for integrating an inverted index into a database, the method comprising: Set a corresponding inverted index for each Level File data partition in the preset database; In the process of writing the data in the memory buffer of the preset database into the Level File data partition, adding the data index of the data to the inverted index corresponding to the Level File data partition, and storing the inverted index in the Level File data partition; When the preset database receives a query request, data retrieval is performed through the inverted index.

[0007] In some embodiments, setting a corresponding inverted index for each Level File data partition in a preset database includes: For each Level File data partition in the preset database, a corresponding inverted index is set respectively, that is, each inverted index is only responsible for retrieving and querying the data in the corresponding Level File data partition; No inverted index is set for the memory buffer of the preset database, that is, the data in the memory buffer is not searched and queried through the inverted index.

[0008] In some embodiments, during the process of writing data in the memory buffer of the preset database into the Level File data partition, adding the data index of the data to the inverted index corresponding to the Level File data partition comprises: Writing the data to be written in the memory buffer of the preset database into the Level File data partition; In the process of the data to be written being written into the Level File data partition in sequence, the data index of the data to be written is synchronously added to the inverted index corresponding to the Level File data partition, so that the write row number RowID of the data to be written is the same as the index ID of the corresponding data index.

[0009] In some embodiments, the method comprises: When the Level File data partitions in the preset database are merged, the inverted indexes corresponding to the merged Level File data partitions are synchronously merged; By presetting the row number mapping rule, the row number RowID of the data written in the merged Level File data partition is made consistent with the index ID of the merged inverted index.

[0010] In some embodiments, when the preset database receives a query request, performing data retrieval through the inverted index includes: Based on the query request received by the preset database, searching the data in the memory buffer of the preset database to obtain the first target data; Determine the target LevelFile data partition where the second target data is located based on the query request received by the preset database; Performing data retrieval through the inverted index stored in the target Level File data partition, directly using the characteristic that the index ID in the inverted index is the same as the write row number RowID of the second target data, to obtain the second target data; The first target data and the second target data are combined to obtain final target data of the query request.

[0011] In some embodiments, based on the query request received by the preset database, searching the data in the memory buffer of the preset database to obtain the first target data includes: Based on the query request received by the preset database, the data in the memory buffer of the preset database is traversed and retrieved through a string search algorithm to obtain the first target data.

[0012] In some embodiments, writing the to-be-written data in the memory buffer of the preset database into the Level File data partition includes: The data cache amount in the memory buffer of the preset database is detected, and if the data cache amount in the memory buffer reaches a preset threshold, the format of the data to be written in the memory buffer is converted into a Level File storage format; The data to be written in the Level File storage format is written into the Level File data partition.

[0013] In some embodiments, the method comprises: The preset database is a DolphinDB database, and the Level File data partition is a data storage unit in a storage engine PKEY based on an LSM-Tree architecture.

[0014] In a second aspect, an embodiment of the present application provides a system for integrating an inverted index in a database, the system being used for the method described in the first aspect above, the system comprising an inverted index setting module, an inverted index generating module and a data retrieval module; The inverted index setting module is used to set a corresponding inverted index for each Level File data partition in the preset database; The inverted index generation module is used to add the data index of the data to the inverted index corresponding to the Level File data partition during the process of the data in the memory buffer of the preset database being written into the Level File data partition, and store the inverted index in the Level File data partition; The data retrieval module is used to perform data retrieval through the inverted index when the preset database receives a query request.

[0015] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any one of the first aspects above.

[0016] Compared with the related art, the embodiments of the present application provide a method, system and medium for integrating an inverted index in a database, wherein the method sets a corresponding inverted index for each Level File data partition in a preset database; in the process of data in the memory buffer of the preset database being written into the Level File data partition, the data index of the data is added to the inverted index corresponding to the Level File data partition, and the inverted index is stored in the Level File data partition; when the preset database receives a query request, data is retrieved through the inverted index, thereby realizing the construction of multiple inverted indexes for each Level File data partition in the database, that is, the full construction mode of the inverted index is changed to an incremental construction mode, so that the construction mode of the inverted index matches the data writing mode of the database, and the inverted index can be stored inside the Level File, which simplifies the maintenance of the inverted index and solves the problem of how to integrate the inverted index inside the database. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a flowchart of the steps of a method for integrating an inverted index into a database according to an embodiment of the present application; Figure 2 is a flow chart of a method for integrating an inverted index into a database according to an embodiment of the present application; Figure 3 It is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0019] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0020] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0021] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantity limitation, and may indicate the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0022] The present application embodiment provides a method for integrating an inverted index into a database. Figure 1 is a flowchart of the steps of the method for integrating an inverted index into a database according to an embodiment of the present application, such as Figure 1As shown, the method comprises the following steps: Step S102, setting a corresponding inverted index for each Level File data partition in the preset database; Step S102 specifically includes the following steps: Step S1021, a corresponding inverted index is set for each Level File data partition in the preset database, that is, each inverted index is only responsible for retrieving and querying the data in the corresponding Level File data partition; It should be noted that in step S1021, a single inverted index is not used to index the full amount of data, but a multiple inverted index solution is adopted in combination with the characteristics of storing data in data partitions of the preset database (such as the Level File of DolphinDB PKEY). In other words, an inverted index is set for each Level File data partition, and each inverted index is only responsible for indexing the data in its Level File data partition. The advantage of this design is that it changes the full amount of inverted index construction mode into incremental construction, which solves the problem of inconsistency between the inverted index construction mode and the data writing mode of the preset database, and also allows the inverted index to be stored inside the Level File data partition, simplifying the maintenance of the inverted index.

[0023] Step S1022: No inverted index is set for the memory buffer of the preset database, that is, the data in the memory buffer is not searched and queried through the inverted index.

[0024] It should be noted that since the construction of an inverted index is a computationally intensive task that takes a certain amount of time, in step S1021 and step S1022, only the corresponding inverted index is constructed for the Level File data partition, and no inverted index is constructed for the data still in the memory buffer. The reason for this design is that the data in the memory buffer is written synchronously from the outside. If an inverted index is constructed for this part of the data, it will greatly affect the write throughput, while the construction of the Level File data partition is asynchronous. Although it takes a certain amount of time to synchronously construct an inverted index during the construction of the Level File data partition, it will not have a significant impact on the external write throughput. This design can achieve high-performance writing of data in the preset database.

[0025] Step S104, in the process of writing the data in the memory buffer of the preset database into the Level File data partition, adding the data index of the data to the inverted index corresponding to the Level File data partition, and storing the inverted index in the Level File data partition; Step S104 specifically includes the following steps: Step S1041, writing the data to be written in the memory buffer of the preset database into the Level File data partition; Specifically, step S1041 detects the amount of data cache in the memory buffer of the preset database. If the amount of data cache in the memory buffer reaches a preset threshold, the format of the data to be written in the memory buffer is converted into the LevelFile storage format; the data to be written in the Level File storage format is written into the Level File data partition.

[0026] Step S1042, while the data to be written is being written into the Level File data partition in sequence, the data index of the data to be written is synchronously added to the inverted index corresponding to the Level File data partition, so that the write row number RowID of the data to be written is the same as the index ID of the corresponding data index.

[0027] It should be noted that, since the construction of the Level File data partition is asynchronous, the asynchronous construction of the inverted index is realized through step S1041 and step S1042. In the preset database, when the size of the memory buffer reaches the preset threshold, the background thread will convert the data in the memory buffer into the storage format of the Level File data partition. Therefore, step S1042 sets the construction time of the inverted index when the Level File data partition is generated. When the data to be written is written into the Level File data partition in sequence, its data index is added to the inverted index at the same time, so that the writing order of the text data is consistent with the order of adding it in the inverted index. Therefore, the writing row number RowID of the data to be written in the Level File data partition is its ID in the inverted index. Thanks to this design, when the inverted index needs to access the original data, its internal index ID can be directly used as the RowID of the Level File to obtain the original data from the Level File. This makes it unnecessary to maintain a copy of the text data in the inverted index, and only the index itself needs to be maintained, which greatly saves storage space. At the same time, this design also greatly simplifies the query steps during text retrieval.

[0028] Step S106: After receiving the query request, the preset database performs data retrieval through the inverted index.

[0029] Step S106 specifically includes the following steps: Step S1061, based on the query request received by the preset database, searching the data in the memory buffer of the preset database to obtain the first target data; Specifically, step S1061 is to traverse and search the data in the memory buffer of the preset database through a string search algorithm based on the query request received by the preset database to obtain the first target data.

[0030] Step S1062, based on the query request received by the preset database, determining the target Level File data partition where the second target data is located; Step S1063, performing data retrieval through the inverted index stored in the target Level File data partition, directly using the characteristic that the index ID in the inverted index is the same as the write row number RowID of the second target data, to obtain the second target data; Step S1064: merge the first target data and the second target data to obtain the final target data of the query request.

[0031] It should be noted that after receiving the query request, the preset database will search the memory buffer and the Level File data partition respectively, merge the results and return. First, search the data in the memory buffer (if it exists). Since no inverted index is established for the data in the memory buffer, the conventional string search algorithm is used for traversal retrieval; secondly, search the data in the Level File data partition. Since each Level File has its own inverted index, it is necessary to traverse all Level File data partitions, load the inverted index inside it, and search on the inverted index to obtain the search results that meet the conditions; finally, merge the results in the first two steps as the final return result.

[0032] It can be seen that since the amount of data in the memory buffer is usually small, good performance can be achieved without the inverted index, and for the massive data in the Level File data partition, high-performance query speed is obtained with the help of the inverted index. The above steps S1061 to S1064 respectively adopt different query strategies for the data in the memory cache area and the Level File data partition, so that high-performance text retrieval is achieved without affecting the write throughput.

[0033] Through the steps in the embodiments of the present application, multiple inverted index construction for each Level File data partition in the database is realized, that is, the full construction mode of the inverted index is changed to an incremental construction mode, so that the construction mode of the inverted index matches the data writing mode of the database. The inverted index can be stored inside the Level File, which simplifies the maintenance of the inverted index and solves the problem of how to integrate the inverted index inside the database.

[0034] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0035] In some embodiments, the method for integrating an inverted index in the database further includes: When the Level File data partitions in the preset database are merged, the inverted indexes corresponding to the merged Level File data partitions are merged synchronously; By presetting the row number mapping rule, the row number RowID of the data written in the merged Level File data partition is made consistent with the index ID of the merged inverted index.

[0036] It should be noted that the Level File data partitions of the preset database will be merged regularly to ensure the query performance and efficiency of each vector index. Under the premise of the "multiple inverted indexes" design of the above embodiment, when the Level File data partitions are merged, the inverted indexes inside them also need to be merged synchronously. Since the row number of the data in the Level File data partition will change before and after the merger, in order to ensure that the row number RowID in the Level File data partition always corresponds to the ID in the inverted index, the mapping row number RowID of the index is specified when the inverted index is merged through the preset row number mapping rule. Therefore, when the Level File is merged, the index ID of the inverted index can be kept consistent with the row number RowID of the Level File data partition only through the row number mapping rule, avoiding the reconstruction of the corresponding inverted index when merging the Level File data partitions, and greatly improving the merging efficiency of the Level File data partitions containing the inverted index.

[0037] In some of the embodiments, the preset database in the above embodiments is preferably a DolphinDB database, and the Level File data partition is preferably a data storage unit in a storage engine PKEY based on an LSM-Tree architecture.

[0038] It should be noted that PKEY (Partition Key) is mainly used to define how to partition and store data. Through PKEY, data can be partitioned according to one or more fields (such as time, geographic location, etc.), and multi-level partitioning is supported, that is, multiple partition keys can be set on the same table to form a hierarchical partition structure, for example: first-level partitioning by year, second-level partitioning by month, third-level partitioning by region, etc. Each data partition can be managed independently, which not only helps to improve the writing speed, but also significantly speeds up the query response time.

[0039] It should be further explained that Figure 2 is a flow chart of a method for integrating an inverted index into a database according to an embodiment of the present application. Figure 2 As shown in the figure, the Level File in DolphinDB is an important part of its storage engine, which is used to efficiently manage and store data. DolphinDB uses a hierarchical (Level) file system to organize and manage data. Data is divided into multiple layers or levels (Level) according to time or other dimensions. Each Level contains a series of data files, which are usually divided according to a certain time window or data volume. As the Level increases, the time span or data volume of the file will also increase accordingly. Among them, Level 0 is usually the most recent data and may contain small files that are not fully merged. Over time, data will migrate from lower levels to higher levels. During this process, the data will be merged, compressed and optimized to reduce disk usage and improve read performance.

[0040] The embodiment of the present application provides a system for integrating an inverted index in a database, the system comprising an inverted index setting module, an inverted index generating module and a data retrieval module; An inverted index setting module is used to set a corresponding inverted index for each Level File data partition in a preset database; An inverted index generation module, used for adding the data index of the data to the inverted index corresponding to the Level File data partition during the process of the data in the memory buffer of the preset database being written into the Level File data partition, and storing the inverted index in the Level File data partition; The data retrieval module is used to retrieve data through the inverted index when the preset database receives a query request.

[0041] Through the inverted index setting module, inverted index generation module and data retrieval module in the embodiments of the present application, multiple inverted index construction for each Level File data partition in the database is implemented, that is, the full construction mode of the inverted index is changed to an incremental construction mode, so that the construction mode of the inverted index matches the data writing mode of the database. The inverted index can be stored inside the Level File, which simplifies the maintenance of the inverted index and solves the problem of how to integrate the inverted index inside the database.

[0042] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0043] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0044] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0045] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0046] In addition, in combination with the method of integrating an inverted index in a database in the above embodiment, the embodiment of the present application can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any one of the methods of integrating an inverted index in a database in the above embodiment is implemented.

[0047] In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for integrating an inverted index in a database is implemented. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball, or touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0048] In one embodiment, Figure 3 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, such as Figure 3 As shown, an electronic device is provided, which may be a server, and its internal structure diagram may be as shown in Figure 3 As shown. The electronic device includes a processor, a network interface, an internal memory and a non-volatile memory connected through an internal bus, wherein the non-volatile memory stores an operating system, a computer program and a database. The processor is used to provide computing and control capabilities, the network interface is used to communicate with an external terminal through a network connection, the internal memory is used to provide an environment for the operation of the operating system and the computer program, the computer program is executed by the processor to implement a method for integrating an inverted index in a database, and the database is used to store data.

[0049] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0050] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0051] Those skilled in the art should understand that the technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0052] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for integrating an inverted index in a database, characterized in that: The method comprises: Set a corresponding inverted index for each Level File data partition in the preset database; In the process of writing the data in the memory buffer of the preset database into the Level File data partition, adding the data index of the data to the inverted index corresponding to the Level File data partition, and storing the inverted index in the Level File data partition; When the preset database receives a query request, data retrieval is performed through the inverted index.

2. The method according to claim 1, characterized in that For each Level File data partition in the preset database, set the corresponding inverted index separately, including: For each Level File data partition in the preset database, a corresponding inverted index is set respectively, that is, each inverted index is only responsible for retrieving and querying the data in the corresponding Level File data partition; No inverted index is set for the memory buffer of the preset database, that is, the data in the memory buffer is not searched and queried through the inverted index.

3. The method according to claim 1, characterized in that In the process of writing the data in the memory buffer of the preset database into the Level File data partition, adding the data index of the data to the inverted index corresponding to the Level File data partition comprises: Writing the data to be written in the memory buffer of the preset database into the Level File data partition; In the process of the data to be written being written into the Level File data partition in sequence, the data index of the data to be written is synchronously added to the inverted index corresponding to the Level File data partition, so that the write row number RowID of the data to be written is the same as the index ID of the corresponding data index.

4. The method according to claim 3, characterized in that The method comprises: When the Level File data partitions in the preset database are merged, the inverted indexes corresponding to the merged Level File data partitions are synchronously merged; By presetting the row number mapping rule, the row number RowID of the data written in the merged Level File data partition is made consistent with the index ID of the merged inverted index.

5. The method according to claim 3, characterized in that: When the preset database receives a query request, performing data retrieval through the inverted index includes: Based on the query request received by the preset database, searching the data in the memory buffer of the preset database to obtain the first target data; Determine, based on the query request received by the preset database, a target Level File data partition where the second target data is located; Performing data retrieval through the inverted index stored in the target Level File data partition, directly using the characteristic that the index ID in the inverted index is the same as the write row number RowID of the second target data, to obtain the second target data; The first target data and the second target data are combined to obtain final target data of the query request.

6. The method according to claim 5, characterized in that Based on the query request received by the preset database, searching the data in the memory buffer of the preset database to obtain the first target data includes: Based on the query request received by the preset database, the data in the memory buffer of the preset database is traversed and retrieved through a string search algorithm to obtain the first target data.

7. The method according to claim 3, characterized in that Writing the data to be written in the memory buffer of the preset database into the Level File data partition includes: The data cache amount in the memory buffer of the preset database is detected, and if the data cache amount in the memory buffer reaches a preset threshold, the format of the data to be written in the memory buffer is converted into a Level File storage format; The data to be written in the Level File storage format is written into the Level File data partition.

8. The method according to claim 1, characterized in that The method comprises: The preset database is a DolphinDB database, and the Level File data partition is a data storage unit in a storage engine PKEY based on an LSM-Tree architecture.

9. A system integrating an inverted index in a database, characterized in that: The system is used to execute the method according to any one of claims 1 to 8, and the system comprises an inverted index setting module, an inverted index generating module and a data retrieval module; The inverted index setting module is used to set a corresponding inverted index for each Level File data partition in the preset database; The inverted index generation module is used to add the data index of the data to the inverted index corresponding to the Level File data partition during the process of the data in the memory buffer of the preset database being written into the Level File data partition, and store the inverted index in the Level File data partition; The data retrieval module is used to perform data retrieval through the inverted index when the preset database receives a query request.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images