Index prefix compression processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to an index prefix compression processing method and device, computer equipment, a computer readable storage medium and a computer program product, and relates to the technical field of databases. According to the method, intelligent compression of the index prefix can be realized, the compression efficiency is improved, the space occupation is reduced, and the storage overhead of the index is reduced. The method comprises the following steps: generating a prefix symbol for simplifying and representing a corresponding repeated prefix part according to the repeated prefix part of each original index through a symbol generation model; the symbol generation model generates a prefix symbol corresponding to the original index based on the use condition of a historical index containing a historical prefix symbol and / or the use condition of an index sample containing a prefix symbol sample; registering each prefix symbol corresponding to each repeated prefix part in a prefix symbol table; and for the target index, replacing the repeated prefix part of the target index with the corresponding prefix symbol according to the prefix symbol registered in the prefix symbol table.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to an index prefix compression processing method, device, computer equipment, computer-readable storage medium and computer program product. Background Art

[0002] In the context of data-intensive applications, one of the core challenges of Database Management System (DBMS) is how to effectively manage the growing storage requirements while ensuring efficient query response.

[0003] Currently, B+ tree, as a widely used index structure, occupies a core position in large-scale data storage and retrieval with its excellent query performance and range scanning capabilities. However, in scenarios containing a large number of datasets with similar prefixes, especially in environments with limited memory and disk resources, traditional B+ tree indexes may cause significant storage overhead. Summary of the invention

[0004] Based on this, it is necessary to provide an index prefix compression processing method, device, computer equipment, computer-readable storage medium and computer program product to address the above technical problems.

[0005] In a first aspect, the present application provides an index prefix compression processing method, comprising:

[0006] Generate a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index through a symbol generation model; wherein the symbol generation model generates the prefix symbol corresponding to the original index based on the usage of the historical index containing the historical prefix symbol and / or the usage of the index sample containing the prefix symbol sample;

[0007] registering each of the prefix symbols corresponding to each of the repeated prefix parts in a prefix symbol table;

[0008] For a target index, a repeated prefix portion of the target index is replaced with a corresponding prefix symbol according to the prefix symbol registered in the prefix symbol table.

[0009] In one of the embodiments, the symbol generation model generates a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index, including: if the time period condition is met, the symbol generation model generates a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index.

[0010] In one of the embodiments, after generating a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index through a symbol generation model, the method further includes: determining the original index set for which the prefix symbol needs to be regenerated according to the usage of each prefix symbol; regenerating a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index in the original index set through a symbol generation model; and registering each regenerated prefix symbol corresponding to the repeated prefix part of each original index in the original index set in the prefix symbol table.

[0011] In one of the embodiments, the symbol generation model generates a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index, including: according to the index insertion operation, when the compression resource condition is met, the symbol generation model generates a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index; wherein each original index includes an inserted index and an existing index in the index page before the insertion; for the target index, according to the prefix symbol registered in the prefix symbol table, replacing the repeated prefix part of the target index with the corresponding prefix symbol, including: for the inserted index, according to the prefix symbol registered in the prefix symbol table, replacing the repeated prefix part of the inserted index with the corresponding prefix symbol.

[0012] In one of the embodiments, the method further includes: according to the index insertion operation, if the index page meets a page fullness threshold and the released space of the index page is insufficient, determining that the compression resource condition is met.

[0013] In one of the embodiments, the compressed index in the index page is provided with an identification bit; the identification bit is used to indicate that the index has been compressed; the compressed index includes the inserted index and the updated index in the existing index before the insertion; the uncompressed index in the index page includes the unupdated index in the existing index before the insertion.

[0014] In a second aspect, the present application also provides an index prefix compression processing device, comprising:

[0015] A symbol generation model is used to generate a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index; wherein the symbol generation model generates the prefix symbol corresponding to the original index based on the usage of the historical index containing the historical prefix symbol and / or the usage of the index sample containing the prefix symbol sample; each prefix symbol corresponding to each repeated prefix part is registered in a prefix symbol table; for the target index, the repeated prefix part of the target index is replaced with the corresponding prefix symbol according to the prefix symbol registered in the prefix symbol table.

[0016] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0017] A symbol generation model is used to generate a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index; wherein the symbol generation model generates the prefix symbol corresponding to the original index based on the usage of the historical index containing the historical prefix symbol and / or the usage of the index sample containing the prefix symbol sample; each prefix symbol corresponding to each repeated prefix part is registered in a prefix symbol table; for the target index, the repeated prefix part of the target index is replaced with the corresponding prefix symbol according to the prefix symbol registered in the prefix symbol table.

[0018] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0019] A symbol generation model is used to generate a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index; wherein the symbol generation model generates the prefix symbol corresponding to the original index based on the usage of the historical index containing the historical prefix symbol and / or the usage of the index sample containing the prefix symbol sample; each prefix symbol corresponding to each repeated prefix part is registered in a prefix symbol table; for the target index, the repeated prefix part of the target index is replaced with the corresponding prefix symbol according to the prefix symbol registered in the prefix symbol table.

[0020] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0021] A symbol generation model is used to generate a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index; wherein the symbol generation model generates the prefix symbol corresponding to the original index based on the usage of the historical index containing the historical prefix symbol and / or the usage of the index sample containing the prefix symbol sample; each prefix symbol corresponding to each repeated prefix part is registered in a prefix symbol table; for the target index, the repeated prefix part of the target index is replaced with the corresponding prefix symbol according to the prefix symbol registered in the prefix symbol table.

[0022] The index prefix compression processing method, device, computer equipment, computer-readable storage medium and computer program product generate prefix symbols for simplifying the representation of the corresponding repeated prefix parts according to the repeated prefix parts of each original index through a symbol generation model; the symbol generation model generates prefix symbols corresponding to the original index based on the usage of historical indexes containing historical prefix symbols and / or the usage of index samples containing prefix symbol samples; registers each prefix symbol corresponding to each repeated prefix part in a prefix symbol table; and for a target index, replaces the repeated prefix part of the target index with the corresponding prefix symbol according to the prefix symbol registered in the prefix symbol table. This scheme can intelligently generate prefix symbols for simplifying the representation of the corresponding repeated prefix parts for each original index, realize intelligent compression of index prefixes, improve compression efficiency, reduce its space occupancy, and reduce the storage overhead of the index. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 An application environment diagram of an index prefix compression processing method in an embodiment;

[0025] Figure 2 A schematic diagram of a flow chart of an index prefix compression processing method in one embodiment;

[0026] Figure 3 A schematic diagram showing a comparison of effects of an index prefix compression processing method in one embodiment;

[0027] Figure 4 is a structural block diagram of an index prefix compression processing device in one embodiment;

[0028] Figure 5FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0030] The index prefix compression processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the application environment may include a terminal and a server, and the terminal may communicate with the server through a network. The data storage system may store data that the server needs to process. The data storage system may be integrated on the server, or may be placed on the cloud or other network servers. Among them, the terminal may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0031] In an exemplary embodiment, Figure 2 As shown, a method for index prefix compression processing is provided, which can be applied to Figure 1 The method may include the following steps:

[0032] Step S201 : generating a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index through a symbol generation model.

[0033] The original index may be the original index contained in the index page before compression. The repeated prefix part of the original index may be the repeated prefix part of each original index in the index page, and the repeated prefix part may be the entire original index. For example, if the original index in the index page includes several indexes str012305, the repeated prefix part may be the entire index str012305.

[0034] The prefix symbol is a symbol used to simplify and replace the corresponding repeated prefix part, and may include characters, numbers, etc. As an example, the prefix symbol of the index str012305 may be SP1, etc.

[0035] In this step, a prefix symbol for simplifying the representation of the corresponding repeated prefix part can be generated by a symbol generation model according to the repeated prefix part of each original index contained in the index page, thereby compressing the database index space.

[0036] The symbol generation model generates a prefix symbol corresponding to the original index based on the usage of the historical index containing the historical prefix symbol and / or the usage of the index sample containing the prefix symbol sample. The historical prefix symbol may include prefix symbols used in other index pages, or may include prefix symbols used before the index page. The historical index is the index corresponding to the historical prefix symbol, and its usage may be the usage of the historical prefix symbols by the relevant users, such as the number of times used, whether it is recommended for use, etc. The prefix symbol sample may include the prefix symbol sample set by the relevant user for a specific index sample, which may be actually used or set according to the usage habits of the relevant user. When setting the prefix symbol sample of the index sample, the usage of the prefix symbol sample may be set at the same time, such as whether it is recommended for use, etc. Based on this, the usage of the historical index containing the historical prefix symbol and / or the usage of the index sample containing the prefix symbol sample may be used to train the symbol generation model (which may be an artificial intelligence model, a neural network, etc.), and the trained symbol generation model may be used to generate prefix symbols for simplifying the representation of the corresponding repeated prefix parts according to the repeated prefix parts of each original index in the index page.

[0037] Step S202: register each prefix symbol corresponding to each repeated prefix part in the prefix symbol table.

[0038] In this step, the prefix symbol table may be set in the index page, and the prefix symbol corresponding to each repeated prefix part may be registered in the prefix symbol table.

[0039] Step S203 , for the target index, according to the prefix symbol registered in the prefix symbol table, replace the repeated prefix part of the target index with the corresponding prefix symbol.

[0040] In this step, for the target index that needs to be index prefix compressed, the repeated prefix part of the target index can be replaced with the corresponding prefix symbol according to the prefix symbol registered in the prefix symbol table, and the target index after the prefix compression is recorded in the index page. Therefore, when writing the index, the repeated prefix part can be replaced with the prefix symbol reference according to the prefix symbol table, and reverse parsing can be performed when reading.

[0041] like Figure 3The comparison of the index page before and after compression is shown. The target index may include all the indexes contained in the index page before compression, and the repeated prefix part is mapped to a prefix symbol for simplifying the representation of the corresponding repeated prefix part by the symbol generation model, and then the prefix symbol corresponding to each repeated prefix part is registered in the prefix symbol table provided in the index page. For all the indexes contained in the index page before compression, the repeated prefix part can be replaced with the corresponding prefix symbol, and the indexes after prefix compression can still be arranged in a certain order in the index page, and the compressed indexes can be displayed in a certain order following the prefix symbol table. Therefore, based on the intelligent shared prefix storage mechanism, through the integration and function of the prefix symbol table, frequently occurring prefixes can be stored in a static form. By centrally storing the repeated prefix information and retaining it only once, and storing the rest as independent suffixes, the index space is greatly saved and the parsing time is reduced. It can not only reduce duplicate data, but also simplify the query process and improve efficiency. It can also be combined with deep learning to generate prefix symbols suitable for each index in the current index page, improve compression efficiency, reduce space occupation, and based on the sorting characteristics of the index, through the division of data in static and dynamic forms, the compression of the database index space is achieved, while maintaining or improving the timeliness of query processing.

[0042] The index prefix compression processing method of the present embodiment generates a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index through a symbol generation model; the symbol generation model generates a prefix symbol corresponding to the original index based on the usage of the historical index containing the historical prefix symbol and / or the usage of the index sample containing the prefix symbol sample; each prefix symbol corresponding to each repeated prefix part is registered in a prefix symbol table; for the target index, the repeated prefix part of the target index is replaced with the corresponding prefix symbol according to the prefix symbol registered in the prefix symbol table, and a prefix symbol for simplifying the representation of the corresponding repeated prefix part of each original index can be intelligently generated, thereby realizing intelligent compression of the index prefix, improving compression efficiency, reducing its space occupancy, and reducing the storage overhead of the index.

[0043] In an exemplary embodiment, the step S201 of generating a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index by using a symbol generation model may include:

[0044] If the time period condition is met, a prefix symbol for simplifying the representation of the corresponding repeated prefix part is generated according to the repeated prefix part of each original index through a symbol generation model.

[0045] In this embodiment, the time period condition may be a time period such as one day, one week, or one month set by the relevant user. If the current time satisfies the time period condition, a prefix symbol for simplifying the representation of the corresponding repeated prefix part may be generated according to the repeated prefix part of each original index through a symbol generation model. The original indexes may include the original indexes contained in the index page at the current time. In this way, the original indexes contained in the index page may be compressed according to a certain time period, and the simplified representation of the corresponding repeated prefix part of each original index in the index page may be continuously optimized.

[0046] In an exemplary embodiment, after generating a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index by the symbol generation model in step S201, the method may further include:

[0047] According to the usage of each prefix symbol, determine the original index set that needs to regenerate the prefix symbol; regenerate the prefix symbol used to simplify the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index in the original index set through the symbol generation model; register each regenerated prefix symbol corresponding to the repeated prefix part of each original index in the original index set in the prefix symbol table.

[0048] In this embodiment, the original index set for which prefix symbols need to be regenerated in each original index contained in the index page can be determined based on the usage of each prefix symbol corresponding to each original index contained in the index page, such as the number of times used, whether it is recommended for use, etc. Among them, the original index set for which prefix symbols need to be regenerated can be, for example, an original index whose number of uses does not meet the requirements. At this time, prefix symbols can be regenerated for it according to the user's usage habits. The symbol generation model is used to regenerate prefix symbols for simplifying the representation of the corresponding repeated prefix parts according to the repeated prefix parts of each original index in the original index set. The regenerated prefix symbol should be different from the prefix symbol corresponding to the original index whose number of uses does not meet the requirements. If they are the same, they can be regenerated again. After the prefix symbol is regenerated, each regenerated prefix symbol corresponding to the repeated prefix part of each original index in the original index set can be registered in the prefix symbol table. In this way, the prefix symbol can be regenerated for it when the usage of the prefix symbol does not meet the user's usage requirements, thereby improving the adaptability of the prefix symbol.

[0049] In an exemplary embodiment, generating a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index through a symbol generation model in step S201 may include: according to the index insertion operation, while satisfying the compression resource condition, generating a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index through a symbol generation model; wherein each original index includes an inserted index and an existing index in the index page before the insertion.

[0050] During the transaction commit phase, when the index is converted from the in-memory structure to the persistent structure, the compression logic can be intelligently inserted. This process not only takes into account the atomicity of the transaction, but also uses the transaction lock mechanism to ensure the consistency and integrity of the data during the conversion and compression process.

[0051] In this embodiment, according to the index insertion operation, when the compression resource conditions are met, for example, when one or more compression resource conditions such as the current system load, memory status, etc. are met, a prefix symbol for simplifying the representation of the corresponding repeated prefix part is generated according to the repeated prefix part of each original index through the symbol generation model. At this time, each original index can include the inserted index and the index page (or index page) that existed before the insertion. That is, when inserting the index, if the compression resource conditions are currently met, the compression processing of all the indexes in the index page that existed before the insertion and the newly inserted index can be triggered. In this way, an active or conservative compression strategy can be adopted according to the compression resource conditions. For example, when resources are sufficient and index updates are not frequent, a more active compression strategy is adopted, otherwise a conservative strategy is adopted to reduce CPU and I / O overhead to ensure the stability of the overall system performance.

[0052] Compression can also be performed outside of the index management operation (SMO), avoiding expensive index locks and multiple page locks, reducing the blocking of concurrent operations, and improving the concurrent processing capabilities of the system. At the same time, compression decisions are made only at the page level, without relying on global information, simplifying the complexity of the compression algorithm.

[0053] Based on this, in step S203, for the target index, replacing the repeated prefix part of the target index with the corresponding prefix symbol according to the prefix symbol registered in the prefix symbol table may include:

[0054] For the inserted index, the repeated prefix part of the inserted index is replaced with the corresponding prefix symbol according to the prefix symbol registered in the prefix symbol table.

[0055] In this embodiment, after triggering compression processing of all existing indexes in the index page before insertion and the newly inserted index, the inserted index can have its repeated prefix parts replaced with corresponding prefix symbols according to the prefix symbols registered in the prefix symbol table.

[0056] In an exemplary embodiment, the method of the present application may further include the following steps: according to the index insertion operation, if the index page meets the page fullness threshold and the released space of the index page is insufficient, it is determined that the compression resource condition is met.

[0057] In this embodiment, a page fill threshold can be set. When the index page meets the page fill threshold (close to full load) and cannot release enough space through, for example, page reorganization (insufficient released space of the index page), it is determined that the compression resource condition is met, that is, only then is the batch compression of the data of the entire index page triggered. The solution of this embodiment can effectively disperse the overhead of the compression operation, avoid the high cost of compression for each insertion operation, reduce frequent input and output (IO) operations through batch processing, smooth the usage curve of system resources, especially in high-concurrency write scenarios, and significantly improve the stability and response speed of the system.

[0058] In an exemplary embodiment, a compressed index in an index page is provided with an identification bit; the identification bit is used to indicate that the index has been compressed; the compressed index includes the inserted index and the updated index in the existing index before the insertion; the uncompressed index in the index page includes the unupdated index in the existing index before the insertion.

[0059] In this embodiment, a flag bit can be set for the compressed index in the index page. The flag bit is used to indicate that the index in the index page has been compressed (it can be by adding a specific bit in each index header to identify whether it has been compressed), so that the compressed and uncompressed indexes can be quickly distinguished, which is convenient for the system to process on demand. Among them, the compressed index can include the newly inserted index, the updated index in the existing index before the insertion (the updated index). The uncompressed index in the index page can include the non-updated index in the existing index before the insertion, which can be understood as the index in the old version of the database.

[0060] In the scheme of this embodiment, the universal identification bit can ensure the data consistency during the database upgrade or migration process, and the identification bit fully considers the forward and backward compatibility. Among them, for the index of the old version of the database, the newly added identification bit can be disabled by default to ensure that the existing query logic is not affected. For the newly inserted or updated index, the identification bit can be set as needed, so as to gradually transition to the new storage mode without destroying the historical data. Therefore, in the same index page, compressed indexes and non-compressed indexes are allowed to coexist. Through the identification bit system, different types of indexes can be dynamically identified and processed. This hybrid storage mode makes full use of the page space, especially in the scenario where data is frequently updated. The strategy of coexistence of new and old indexes helps to smoothly transition to a more efficient compressed storage format. Based on this, during the query operation, the system can quickly distinguish the index type according to the identification bit and select the appropriate decompression path. The update operation is dynamically adjusted according to the state of the index, which may involve decompression, updating, and recompression of the record, or directly performing the update in the compressed state to minimize resource consumption. Through the design of the index and the hybrid storage mode, the database system can achieve efficient use of storage space and flexible storage strategy adjustment without sacrificing query efficiency. This not only enhances the scalability of the system, but also provides strong support for responding to ever-changing data processing needs.

[0061] In this regard, in specific applications, efficient decompression logic and data access optimization can be implemented to ensure data access performance. Among them, an identification bit can be used to mark whether the index is in a compressed state. In order to ensure the seamlessness of index access, the index access can be adjusted so that it can recognize and correctly handle compressed indexes. This can include introducing the parsing of compressed metadata in the decompression logic, and processing compressed fields when obtaining index offsets, to ensure that the index is accurately located in a compressed state. Among them, the best decompression path can be selected according to the query conditions and operation types. For simple read operations, only the required fields can be decompressed. For complex queries or update operations, the index may need to be fully decompressed. Flexible strategies can reduce unnecessary data decompression and significantly improve processing speed. For column attribute information that may be lost in compressed indexes, a metadata mapping table corresponding to the compressed index can be maintained to quickly restore column attributes during comparison or conversion to ensure the integrity of the index and the accuracy of the operation. Among them, it is also possible to dynamically decide whether to decompress the index immediately, to what extent to decompress, or to perform comparison operations directly in the compressed domain based on the context of the current operation (such as query conditions, execution plan stage, etc.), thereby minimizing the overhead of data decompression while ensuring data consistency. Based on this, it can ensure that when processing compressed data, it can maintain high data access speed and effectively utilize storage resources, providing strong performance support for modern database management systems.

[0062] On the whole, the index prefix compression processing method of the present application realizes efficient reuse and storage compression of prefix information through the design of semi-static prefix strategy and symbol table, significantly reduces the storage space required for index, especially when processing highly repetitive data, the effect is more prominent, thereby reducing the hardware resource requirements and operation and maintenance costs, thereby improving storage efficiency and resource utilization. It can also intelligently adjust the compression timing and method dynamically according to information such as system load, avoid the negative impact of compression operation on system performance, and ensure the efficiency of data access, which is conducive to optimizing the full link of decompression logic in the application, ensuring a seamless experience of data access and reducing query delay. Secondly, the introduction of identification bits and hybrid storage modes not only ensures the smooth transition and compatibility of new and old data formats, but also improves the scalability and flexibility of the system, and can switch freely between different record formats to adapt to diverse business scenarios and data processing requirements. Multi-level decompression strategies and intelligent selection mechanisms can be applied, and the degree of decompression can be dynamically adjusted according to query requirements, reducing unnecessary data processing, and improving query performance. It can be used to solve the problem of missing column attributes and ensure the accuracy and integrity of data processing. Therefore, the index prefix compression processing method of the present application can provide technical support for database management and data-intensive applications, and solve the storage and performance bottleneck problems encountered when processing large-scale data.

[0063] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0064] Based on the same inventive concept, the embodiment of the present application also provides an index prefix compression processing device for implementing the above-mentioned index prefix compression processing method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in the one or more index prefix compression processing device embodiments provided below can refer to the limitations of the index prefix compression processing method above, and will not be repeated here.

[0065] In an exemplary embodiment, Figure 4 As shown, an index prefix compression processing device is provided, and the device 400 may include:

[0066] A symbol generation module 401 is used to generate a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index through a symbol generation model; wherein the symbol generation model generates the prefix symbol corresponding to the original index based on the usage of the historical index containing the historical prefix symbol and / or the usage of the index sample containing the prefix symbol sample;

[0067] A symbol registration module 402, configured to register each of the prefix symbols corresponding to each of the repeated prefix parts in a prefix symbol table;

[0068] The prefix replacement module 403 is used to replace the repeated prefix part of the target index with a corresponding prefix symbol according to the prefix symbol registered in the prefix symbol table.

[0069] In an exemplary embodiment, the symbol generation module 401 is used to generate a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index through a symbol generation model if the time period condition is met.

[0070] In an exemplary embodiment, the symbol generation module 401 is also used to determine the original index set of the prefix symbols that need to be regenerated according to the usage of each prefix symbol; regenerate the prefix symbol used to simplify the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index in the original index set through the symbol generation model; and register each regenerated prefix symbol corresponding to the repeated prefix part of each original index in the original index set in the prefix symbol table.

[0071] In an exemplary embodiment, the symbol generation module 401 is used to generate a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index through a symbol generation model according to the index insertion operation and when the compression resource conditions are met; wherein the original indexes include the inserted index and the existing index in the index page before the insertion; the prefix replacement module 403 is used to replace the repeated prefix part of the inserted index with a corresponding prefix symbol according to the prefix symbol registered in the prefix symbol table.

[0072] In an exemplary embodiment, the symbol generation module 401 is further configured to determine, based on the index insertion operation, that the compression resource condition is met if the index page meets a page fill threshold and the released space of the index page is insufficient.

[0073] In an exemplary embodiment, the compressed index in the index page is provided with an identification bit; the identification bit is used to indicate that the index has been compressed; the compressed index includes the inserted index and the updated index in the existing index before the insertion; the uncompressed index in the index page includes the unupdated index in the existing index before the insertion.

[0074] Each module in the above-mentioned index prefix compression processing device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0075] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store index data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an index prefix compression processing method is implemented.

[0076] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0077] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0078] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0079] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0080] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0081] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0082] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0083] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for index prefix compression processing, characterized in that: The method comprises: Generate a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index through a symbol generation model; wherein the symbol generation model generates the prefix symbol corresponding to the original index based on the usage of the historical index containing the historical prefix symbol and / or the usage of the index sample containing the prefix symbol sample; registering each of the prefix symbols corresponding to each of the repeated prefix parts in a prefix symbol table; For a target index, a repeated prefix portion of the target index is replaced with a corresponding prefix symbol according to the prefix symbol registered in the prefix symbol table.

2. The method according to claim 1, characterized in that The generating, according to the repeated prefix part of each original index, a prefix symbol for simplifying the representation of the corresponding repeated prefix part by the symbol generating model comprises: If the time period condition is met, a prefix symbol for simplifying the representation of the corresponding repeated prefix part is generated according to the repeated prefix part of each original index through a symbol generation model.

3. The method according to claim 1, characterized in that After generating a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index by the symbol generation model, the method further includes: Determining, according to the usage of each of the prefix symbols, an original index set of the prefix symbols that needs to be regenerated; Regenerate, by a symbol generation model, a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index in the original index set; The regenerated prefix symbols corresponding to the repeated prefix parts of the original indexes in the original index set are registered in the prefix symbol table.

4. The method according to claim 1, characterized in that The generating, according to the repeated prefix part of each original index, a prefix symbol for simplifying the representation of the corresponding repeated prefix part by the symbol generating model comprises: According to the index insertion operation, in the case where the compression resource condition is met, a prefix symbol for simplifying the representation of the corresponding repeated prefix part is generated according to the repeated prefix part of each original index through a symbol generation model; wherein each original index includes the inserted index and the index existing in the index page before the insertion; The step of replacing a repeated prefix portion of the target index with a corresponding prefix symbol according to a prefix symbol registered in the prefix symbol table for the target index includes: For the inserted index, according to the prefix symbol registered in the prefix symbol table, the repeated prefix part of the inserted index is replaced with the corresponding prefix symbol.

5. The method according to claim 4, characterized in that The method further comprises: According to the index insertion operation, if the index page meets a page fullness threshold and the released space of the index page is insufficient, it is determined that the compression resource condition is met.

6. The method according to claim 4, characterized in that The compressed index in the index page is provided with an identification bit; the identification bit is used to indicate that the index has been compressed; the compressed index includes the inserted index and the updated index in the existing index before the insertion; the uncompressed index in the index page includes the unupdated index in the existing index before the insertion.

7. An index prefix compression processing device, characterized in that: The device comprises: A symbol generation module, configured to generate a prefix symbol for simplifying the representation of the corresponding repeated prefix part according to the repeated prefix part of each original index through a symbol generation model; wherein the symbol generation model generates the prefix symbol corresponding to the original index based on the usage of the historical index containing the historical prefix symbol and / or the usage of the index sample containing the prefix symbol sample; A symbol registration module, used for registering each of the prefix symbols corresponding to each of the repeated prefix parts in a prefix symbol table; The prefix replacement module is used to replace the repeated prefix part of the target index with the corresponding prefix symbol according to the prefix symbol registered in the prefix symbol table.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images