Index creation method, device and equipment, readable storage medium and program product

Abstract

The embodiment of the invention provides an index creation method and device, equipment, a readable storage medium and a program product, and can be applied to the fields or scenes of cloud technology, artificial intelligence, database management and the like, and the method comprises the following steps: in response to an index creation request for a target field in a database, obtaining an index creation starting condition and an index creation pause condition; if it is determined that an index creation starting condition is met according to the first state data associated with the database, index creation processing is conducted on a data operation record associated with a target field in the log data; if it is determined that an index creation pause condition is met according to second state data associated with the database, index creation processing is paused, and the processing resources are used for conducting service processing on the database; if it is determined that an index creation starting condition is met according to third state data associated with the database, index creation processing is restarted to obtain index data of the target field. According to the embodiment of the invention, the index creation efficiency and automation degree can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an index creation method, an index creation device, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In the process of informatization and digital transformation, a large amount of data is stored in the database. Faced with such a large amount of data, indexes can be created to improve data query efficiency. At present, indexes are created manually, such as first creating a new table and setting indexes for the new table, and then copying the data in the original table to the new table to make the set index effective. The above index creation method has a low degree of automation and the operation process is cumbersome, which makes the efficiency of index creation low. Summary of the invention

[0003] The present application provides an index creation method, apparatus, device, readable storage medium and program product, which can improve the efficiency and automation of index creation.

[0004] In a first aspect, the present application provides an index creation method, the method comprising:

[0005] In response to an index creation request for a target field in a database, obtaining index creation conditions, wherein the index creation conditions include an index creation start condition and an index creation pause condition;

[0006] If it is determined according to the first state data associated with the database that the index creation start condition is met, index creation processing is performed on the data operation record associated with the target field in the log data; the log data record contains the data operation record associated with each field in the database;

[0007] If it is determined according to the second state data associated with the database that the index creation suspension condition is met, suspending the index creation process so as to use the processing resources of the index creation process for business processing on the database, the business processing being different from the index creation process;

[0008] If it is determined according to the third state data associated with the database that the index creation start condition is met, the index creation process is restarted to obtain the index data of the target field.

[0009] In a second aspect, the present application provides an index creation device, the device comprising:

[0010] An acquisition module, configured to acquire index creation conditions in response to an index creation request for a target field in a database, wherein the index creation conditions include an index creation start condition and an index creation pause condition;

[0011] A processing module, configured to perform index creation processing on the data operation record associated with the target field in the log data if it is determined according to the first state data associated with the database that the index creation start condition is met; the log data record includes data operation records associated with each field in the database;

[0012] The processing module is further configured to suspend the index creation process if it is determined according to the second state data associated with the database that the index creation suspension condition is met, so as to use the processing resources of the index creation process for business processing on the database, the business processing being different from the index creation process;

[0013] The processing module is further configured to restart the index creation process to obtain the index data of the target field if it is determined that the index creation start condition is met according to the third state data associated with the database.

[0014] In a third aspect, the present application provides a computer device comprising: a processor, a storage device and a communication interface, wherein the processor, the communication interface and the storage device are interconnected, wherein the storage device stores an executable program code, and the processor is used to call the executable program code to implement the above-mentioned index creation method.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and wherein the program instructions are executed by a processor to implement the index creation method as described above.

[0016] In a fifth aspect, the present application provides a computer program product, which includes a computer program or computer instructions, and the computer program or computer instructions are executed by a processor to implement the above-mentioned index creation method.

[0017] The embodiment of the present application responds to an index creation request for a target field in a database, obtains index creation conditions, and the index creation conditions include index creation start conditions and index creation pause conditions; when the index creation start conditions are met, index creation processing is performed on the data operation records associated with the target field in the log data; when the index creation pause conditions are met, the index creation processing is paused; when the index creation start conditions are met again, the index creation processing is restarted to obtain the index data of the target field. The above method can automatically determine the start time and pause time of index creation, avoids creating indexes manually, thereby realizing automated, dynamic and flexible index creation, and improving the efficiency of index creation. Moreover, in the process of suspending the index creation process, the processing resources of the index creation process are used for business processing on the database that is different from the index creation process, which can improve resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 It is a schematic diagram of the architecture of an index creation system provided by an exemplary embodiment of the present application;

[0020] Figure 2 It is a flowchart of an index creation method provided by an exemplary embodiment of the present application;

[0021] Figure 3 It is a flowchart of another index creation method provided by an exemplary embodiment of the present application;

[0022] Figure 4A is a flow chart of a data query provided by an exemplary embodiment of the present application;

[0023] Figure 4B is a flowchart of creating an elastic index provided by an exemplary embodiment of the present application;

[0024] Figure 5 is a schematic block diagram of an index creation device provided by an exemplary embodiment of the present application;

[0025] Figure 6 It is a schematic block diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] It should be noted that the descriptions of "first", "second", etc. involved in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the technical features defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0028] In order to better understand the solutions of the embodiments of the present application, the following first introduces the relevant terms and concepts that may be involved in the embodiments of the present application:

[0029] Database index: The main purpose of a database index is to speed up the search and lookup of data rows in a specific table. An index provides a faster access path for one or more columns in a table, allowing the database to retrieve relevant data faster while reducing disk reads. When there is no index, the database must scan the entire table row by row to find data that meets the query criteria. Such a full table scan consumes a lot of resources and time, reducing the availability and high concurrency of the application system.

[0030] Elastic index: It is an automated, flexible, dynamic, low-cost index creation method and mechanism proposed in the embodiment of this application. It realizes dynamic and elastic index creation by intelligently judging the index creation start conditions and index creation suspension conditions, and the index creation process will not affect other business processes of the database, which makes users unaware when using the database. However, the traditional index creation method has a greater impact on applications and data. When creating an index for a large amount of data, it may cause the database to lock for a short time, causing the system to crash.

[0031] DDL: (Data definition language) is a data definition language. The main commands are CREAT, ALTER, DROP, etc. DDL is mainly used to define or change the structure of a table (TABLE), data type, add configuration indexes, links and constraints between tables, and other initialization work. They are mostly used when creating tables.

[0032] The embodiments of the present application can be applied to various fields or scenarios such as cloud computing, cloud Internet of Things, cloud games, artificial intelligence, in-vehicle scenarios, smart transportation, assisted driving, database management, etc. Several typical fields or scenarios will be introduced below.

[0033] Cloud computing refers to the delivery and use mode of IT infrastructure, which means obtaining the required resources through the network in an on-demand and easily scalable manner; in a broad sense, cloud computing refers to the delivery and use mode of services, which means obtaining the required services through the network in an on-demand and easily scalable manner. This service can be related to IT and software, the Internet, or other services. Cloud computing is the product of the development and integration of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing. With the development of the Internet, real-time data streams, and the diversification of connected devices, as well as the promotion of search services, social networks, mobile commerce, and open collaboration, cloud computing has developed rapidly. Different from previous parallel distributed computing, the emergence of cloud computing will promote revolutionary changes in the entire Internet model and enterprise management model in terms of concept. This application can store index creation conditions, data related to the target field, etc. on the cloud server. When the above different data need to be used, it can be directly obtained on the cloud server, which greatly improves the data acquisition speed.

[0034] Cloud IOT aims to connect the information sensed and instructions received by sensor devices in traditional IoT to the Internet, truly realize networking, and achieve massive data storage and computing through cloud computing technology. Due to the characteristics of IoT, objects are connected to each other and the current operating status of each "object" is perceived in real time. A large amount of data information will be generated in this process. How to aggregate this information and how to filter out useful information from the massive information to support decision-making for subsequent development have become key issues affecting the development of IoT. IoT cloud based on cloud computing and cloud storage technology has therefore become a strong support for IoT technology and applications.

[0035] Cloud gaming, also known as gaming on demand, is an online gaming technology based on cloud computing technology. Cloud gaming technology enables thin clients with relatively limited graphics processing and data computing capabilities to run high-quality games. In a cloud gaming scenario, the game is not played on the player's game terminal, but on a cloud server, which renders the game scene into a video and audio stream and transmits it to the player's game terminal over the network. The player's game terminal does not need to have powerful graphics computing and data processing capabilities, but only needs to have basic streaming media playback capabilities and the ability to obtain player input commands and send them to the cloud server.

[0036] The application scenario of the index creation method provided by the embodiment of the present application will be described below: In the process of informatization and digital transformation, a large amount of data of enterprises and government departments needs to be digitized and services are online. Driven by the construction of big data applications, data storage of users and business data is becoming more and more centralized, the amount of data is becoming more and more huge, and the data service objects are becoming more and more extensive, which makes the demand for high concurrency, high performance and high availability of business systems more and more intense. At the same time, the business scenarios of data services are mostly data query services, which makes it very important to improve data query efficiency and performance. Creating indexes manually can improve the query efficiency of data, which has a good effect without changing the business, but the above method has a low degree of automation and a cumbersome operation process, which makes the efficiency of creating indexes low. In addition, a large storage space is required to copy and migrate tables, and DDL (database definition language) operations need to be performed on data online, and once an operation error occurs, there will be a greater risk.

[0037] Based on this, the embodiment of the present application proposes to obtain index creation conditions in response to an index creation request for a target field in a database, and the index creation conditions include index creation start conditions and index creation pause conditions; when the index creation start conditions are met, index creation processing is performed on the data operation records associated with the target field in the log data; when the index creation pause conditions are met, the index creation processing is paused; when the index creation start conditions are met again, the index creation processing is restarted to obtain the index data of the target field. The above method can automatically determine the start time and pause time of index creation, avoids creating indexes manually, thereby realizing automated, dynamic and flexible index creation, and improving the efficiency of index creation. Moreover, in the process of suspending the index creation process, the processing resources of the index creation process are used to perform business processing on the database that is different from the index creation process, which can improve resource utilization.

[0038] In addition, the embodiment of the present application proposes to define the index creation start condition and the index creation pause condition according to the system load and the number of connections in the database connection pool, so that the processing device corresponding to the index creation process can use the idle processing resources to perform the index creation process, and the index creation process will not occupy too many IO and CPU resources. This ensures that the index creation process will not affect the business processing (such as the online business of the database, including data addition, deletion, and modification), which can greatly reduce the risk and difficulty of online DDL operations, making online index generation more convenient and safer.

[0039] It can be understood that in the specific implementation of the present application, related data such as index creation conditions and data related to the target field are involved. When the above embodiments of the present application are applied to specific products or technologies, the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of the relevant regions.

[0040] This application will be specifically described by the following embodiments:

[0041] See also Figure 1 , which is a schematic diagram of the architecture of an index creation system provided by an exemplary embodiment of the present application. The index creation system may specifically include a processing device 101 and a database 102. The processing device 101 and the database 102 are connected via a network, such as a local area network, a wide area network, a mobile Internet, etc.

[0042] The processing device 101 may refer to a terminal device or a server. The terminal device is also referred to as a terminal, user equipment (UE), access terminal, user unit, mobile device, user terminal, wireless communication device, user agent or user device. The terminal device may be a smart home appliance, a handheld device with wireless communication function (such as a smart phone, a tablet computer), a computing device (such as a personal computer (PC)), a vehicle terminal, an intelligent voice interaction device, a wearable device or other smart device, but is not limited thereto.

[0043] The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), as well as big data and artificial intelligence platforms.

[0044] The database 102 is used to store data, which can be recorded in different database tables in the database. For example, the database can be a database located in the server, that is, it can be a database built into or carried by the server; the database can also be an external database connected to the server, such as a cloud database (i.e., a database deployed in the cloud), which can be based on any deployment of private cloud, public cloud, hybrid cloud, edge cloud, etc., so that the cloud database focuses on different functions.

[0045] In one embodiment, the processing device 101 can respond to an index creation request for a target field in the database 102 (the index creation request can be sent to the processing device 101 by a terminal device where a database administrator is located, or can be automatically generated by the processing device 101 when it is detected that there is no index data of the target field in the database 102), and obtain an index creation condition, wherein the index creation condition includes an index creation start condition and an index creation pause condition; if the processing device 101 determines that the index creation start condition is met according to the first state data associated with the database 102, the index creation process is performed for the data operation record associated with the target field in the log data; the log data record contains the data operation record associated with each field in the database 102; if the processing device 101 determines that the index creation pause condition is met according to the second state data associated with the database 102, the index creation process is paused so that the processing resources of the index creation process are used for business processing for the database 102, which is different from the index creation process; if the processing device 101 determines that the index creation start condition is met according to the third state data associated with the database 102, the index creation process is restarted to obtain the index data of the target field. Finally, the processing device 101 may update the index data of the target field into the database 102 so that data query processing is performed according to the index data of the target field.

[0046] It is understandable that the system architecture diagram described in the embodiment of the present application is to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided by the embodiment of the present application. For example, the index creation method provided in the embodiment of the present application can be executed by the processing device 101, and can also be executed by other servers or server clusters that are different from the processing device 101 and can communicate with the terminal device and / or the server. It is known to those skilled in the art that Figure 1 The number of processing devices and databases in the embodiment is only illustrative. According to the business implementation needs, any number of processing devices and databases can be configured. In addition, with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems. In the subsequent embodiments, the processing device will refer to the above-mentioned processing device 101, and the database will refer to the above-mentioned database 102, which will not be repeated in the subsequent embodiments.

[0047] See also Figure 2 , which is a flow chart of an index creation method provided by an exemplary embodiment of the present application, and is described by taking the method applied to a processing device as an example, the method may include the following steps:

[0048] S201. In response to an index creation request for a target field in a database, obtain index creation conditions, where the index creation conditions include index creation start conditions and index creation pause conditions.

[0049] In the embodiment of the present application, the target field in the database refers to a specific field in the database that needs to create an index to improve query efficiency. The processing device responds to the index creation request and first obtains the index creation condition, which includes the index creation start condition and the index creation pause condition, which are used to determine whether to start or pause the index creation process.

[0050] In one embodiment, assuming that there is a target field (field A) in the database, when the processing device receives an index creation request for field A, the processing device obtains the index creation conditions, and parses and judges the index creation start conditions and index creation pause conditions therein, so as to decide when to start or pause the index creation process.

[0051] S202: If it is determined according to the first state data associated with the database that the index creation start condition is met, index creation processing is performed on the data operation record associated with the target field in the log data.

[0052] In an embodiment of the present application, the first state data associated with the database is used to indicate the state data associated with the database at the first moment, which may include the state data of the database (such as the number of connections in the connection pool of the database) and the state data of the processing device where the database is located (such as the system load of the processing device, the system load includes CPU load, memory load, IO load, etc.). The index creation start condition is used to determine whether to start the index creation process, which is determined by the first state data associated with the database. For example, the index creation start condition may refer to: the system load is less than the first load threshold, and the number of connections in the connection pool of the database is less than the first connection threshold. Data operation records refer to records of operations such as adding, deleting, and modifying data in the database, such as records corresponding to operations such as INSERT, UPDATE, and DELETE, which can be used to adjust the data in the database. Then, the data operation record associated with the target field refers to the data operation record corresponding to the target field, such as the record of adding, deleting, and modifying operations for field A. Exemplarily, in an order table, the record of inserting and updating operations on the order number field is the data operation record.

[0053] When the processing device determines that the conditions for starting index creation are met, it means that the system has a lot of idle resources that can be used for index creation processing. Therefore, index creation processing can be performed on the data operation records associated with the target field in the log data (index creation processing will not affect the business processing of the database) to obtain the index data of the target field, so as to facilitate subsequent data query requests for the target field based on the index data of the target field, thereby improving data query efficiency.

[0054] In one embodiment, the log data records include data operation records associated with each field in the database. The processing device needs to obtain the data operation records associated with the target field from the log data. The data operation records generally include multiple data operation records. The processing device performs index creation processing on each data operation record associated with the target field in sequence according to the operation occurrence sequence of each data operation record associated with the target field, thereby ensuring the accuracy of the generated index.

[0055] In one embodiment, after determining that the index creation start condition is met, the processing device performs index creation processing on the data operation record associated with the target field in the log data. During the index creation process, the processing device detects the state data associated with the database in real time. If it is detected at a certain moment that the state data associated with the database meets the index creation suspension condition (i.e., it is determined that the index creation suspension condition is met according to the second state data associated with the database), the index creation process is immediately suspended and step S203 is executed.

[0056] S203: If it is determined according to the second state data associated with the database that the index creation suspension condition is met, suspend the index creation process so that the processing resources of the index creation process are used for business processing on the database.

[0057] In an embodiment of the present application, the second state data associated with the database is used to indicate the state data associated with the database at the second moment, which may include the state data of the database (such as the number of connections in the connection pool of the database) and the state data of the processing device where the database is located (such as the system load of the processing device, the system load includes CPU load, memory load, IO load). The index creation pause condition is used to determine whether to suspend the index creation process, which is determined by the second state data associated with the database. For example, the index creation pause condition may refer to: the system load is greater than or equal to the second load threshold, or the number of connections in the connection pool of the database is greater than or equal to the second connection threshold.

[0058] The processing resources for index creation processing refer to the resources in the processing device used to perform index creation processing, such as CPU, memory, disk space, etc. During the index creation process, an appropriate amount of processing resources needs to be allocated to perform index calculation and storage operations, so as to ensure the efficiency and performance of index creation.

[0059] Business processing is different from index creation processing. For example, business processing can be query, transaction processing, data backup, etc. In order to ensure the normal operation of the database and meet business needs, it is necessary to suspend the index creation process and then use the processing resources to perform business processing. For example, during peak hours, in order to handle a large number of query requests, you can suspend index creation to release computing resources for query operations, so that users are unaware when using the database, ensuring that index creation does not affect online business.

[0060] When the processing device determines that the conditions for suspending index creation are met, it means that the system has fewer idle resources that can be used for index creation processing at this time. Therefore, the index creation processing can be suspended, and the processing resources originally used for index creation can be used for business processing on the database to ensure the normal operation of the database and the execution of the business.

[0061] S204: If it is determined according to the third state data associated with the database that the index creation start condition is met, restart the index creation process to obtain the index data of the target field.

[0062] In the embodiment of the present application, the third state data associated with the database is used to indicate the state data associated with the database at the third moment. When the processing device re-determines that the index creation start condition is met, it means that the system has more idle resources available for index creation processing at this time, and can support the index creation processing, so the index creation processing can be restarted. Through the above method, the start time and pause time of index creation can be automatically determined, avoiding the manual creation of indexes, thereby realizing automatic, dynamic and flexible index creation.

[0063] Based on the above embodiments, the beneficial effects of the present application are: the embodiments of the present application respond to the index creation request for the target field in the database, obtain the index creation conditions, and the index creation conditions include the index creation start conditions and the index creation pause conditions; when the index creation start conditions are met, the index creation process is performed for the data operation records associated with the target field in the log data; when the index creation pause conditions are met, the index creation process is paused; when the index creation start conditions are met again, the index creation process is restarted to obtain the index data of the target field. The above method can automatically determine the start time and pause time of index creation, avoid creating indexes manually, thereby realizing automated, dynamic and elastic index creation, and improving the efficiency of index creation. Moreover, in the process of suspending the index creation process, the processing resources of the index creation process are used for business processing different from the index creation process for the database, which can improve resource utilization.

[0064] See also Figure 3, which is a flow chart of another index creation method provided by an exemplary embodiment of the present application, and is described by taking the method applied to a processing device as an example, the method may include the following steps:

[0065] S301. In response to an index creation request for a target field in a database, obtain index creation conditions, where the index creation conditions include index creation start conditions and index creation pause conditions.

[0066] The specific implementation of step S301 is described in detail in the above-mentioned embodiment. The implementation of step S201 is described below through steps S302-S303:

[0067] S302: If it is determined according to the first state data associated with the database that the index creation start condition is met, data parsing is performed on the data operation record associated with the target field in the log data to obtain a parsed data field.

[0068] Among them, log data is used to record the data operation records associated with each field in the database. Log data can refer to the binlog of the database. Binlog is a log file that records database change operations. It is stored in binary format and contains every data operation in the database, such as insert, update, delete, etc. Binlog can be automatically generated and maintained by the database management system, and is mainly used for data recovery, data backup, data analysis, etc.

[0069] In one embodiment, a data operation record may be shown in Table 1 below:

[0070] Table 1

[0071]

[0072] Among them, Log_name refers to the name of the binlog log file, indicating the binlog file where the data operation record is located. Pos refers to the starting position of the data operation record in the binlog file. End_pos refers to the ending position of the data operation record in the binlog file. Info refers to the specific information of the data operation record, including the type of data event and related operation information. insert into tinc(c)values('abc') means inserting a piece of data into the table tinc. The inserted column is column A, and the value is 'abc'. The field corresponding to column A can refer to the target field in the embodiment of the present application.

[0073] When the processing device determines that the index creation start condition is met, the processing device performs data parsing processing on the data operation record associated with the target field in the log data to obtain one or more parsed data fields. Exemplarily, the processing device can perform data parsing processing on the data operation record in Table 1 to obtain one or more parsed data fields, and the parsed data fields can include the table name, field name and field value in the INSERT statement, as follows:

[0074] Table name: tinc;

[0075] Field name: A;

[0076] Field value: abc.

[0077] It should be noted that binlog records are sequential records of database operations. Therefore, the processing device needs to create indexes in sequence according to the records in the binlog to ensure the correctness and consistency of the created indexes.

[0078] In one embodiment, if it is determined that the index creation start condition is met according to the first state data associated with the database, it can be implemented according to the following steps:

[0079] (1) Acquire first status data associated with a database; the first status data includes a first system load and a first number of connections in a connection pool of the database.

[0080] The database connection pool is a buffer created to reuse database connections. Applications can obtain connections from the connection pool to perform database operations and return the connections to the connection pool after use so that other requests can reuse the connection. The first connection number of the database connection pool refers to the number of database connections currently allocated in the database connection pool and used by applications.

[0081] System load can include the load of the database server, such as CPU usage, memory usage, disk read and write latency, network traffic, etc. At the same time, it can also include the load of the processing device that processes database requests, such as the CPU and memory usage of the processing device, the response time of request processing, etc.

[0082] (2) If the first system load is less than the first load threshold and the first connection quantity is less than the first connection threshold, it is determined that the index creation start condition is met.

[0083] In an embodiment of the present application, the processing device determines whether the system is in an idle state through the number of connections in the connection pool and the system load. The number of connections in the connection pool is an indicator related to the database connection and is used to determine the number of requests, while the system load is used to comprehensively consider the resource usage of the entire system, including the load of components such as the database and the processing device.

[0084] Exemplarily, the first system load and the first number of connections are state data corresponding to the first moment. The first load threshold may be 0.5, and the first connection threshold may be 10. When the first system load is less than 0.5 and the first number of connections is less than 10, the processing device considers that the index creation start condition is met.

[0085] S303: Generate an index creation statement according to the parsed data field, and perform index creation processing according to the index creation statement.

[0086] The processing device may generate an index creation statement based on the parsed data fields obtained by parsing Table 1. For example, the index creation statement may be as follows:

[0087] ALTER TABLE tinc ADDINDEX A_index(A)

[0088] The index creation statement indicates that an index named A_index is created on field A of table tinc. By parsing the binlog record using the above method, the target table of the operation, the field name and field value information for the index to be created can be extracted, and then the corresponding SQL statement is used to create the index. When there is a large amount of data that needs to be indexed, it takes a long time to directly perform the index creation operation on the entire table, and the efficiency of index creation can be improved by the method in the embodiment of the present application. It should be noted that the processing device can also create indexes for multiple target fields, which will not be repeated here.

[0089] S304: If it is determined according to the second state data associated with the database that the index creation suspension condition is met, suspend the index creation process so that the processing resources of the index creation process are used for business processing on the database.

[0090] In one embodiment, if it is determined that the index creation suspension condition is met according to the second state data associated with the database, the following steps can be used to implement the above:

[0091] (1) Acquire second status data associated with the database; the second status data includes a second system load and a second number of connections in a connection pool of the database.

[0092] (2) If the second system load is greater than or equal to the second load threshold, or the second number of connections is greater than or equal to the second connection threshold, it is determined that the index creation suspension condition is met.

[0093] In steps (1)-(2), the second system load and the second number of connections are state data corresponding to the second moment. The second load threshold value may be 1, and the second connection threshold value may be 50. When the second system load is greater than or equal to 1, and the second number of connections is greater than or equal to 50, the processing device considers that the index creation suspension condition is met.

[0094] It should be noted that in the actual application of the method provided in the embodiment of the present application, the settings of the first load threshold, the first connection threshold, the second load threshold, and the second connection threshold should be adjusted and optimized according to the specific circumstances. At the same time, other factors can also be comprehensively considered, such as the availability of system resources, the performance requirements of the database, etc., so as to ensure the accuracy of the index creation conditions.

[0095] S305: If it is determined according to the third state data associated with the database that the index creation start condition is met, the index creation process is restarted to obtain the index data of the target field.

[0096] In the embodiment of the present application, the third state data associated with the database is used to indicate the state data associated with the database at the third moment. When the processing device re-determines that the index creation start condition is met, it means that the system has more idle resources available for index creation processing at this time, and can support the index creation processing, so the index creation processing can be restarted. Through the above method, the start time and pause time of index creation can be automatically determined, avoiding the manual creation of indexes, thereby realizing automatic, dynamic and flexible index creation.

[0097] In one embodiment, in order to further ensure the accuracy of the index creation conditions, the processing device can generate the index creation conditions by comprehensively considering other indicators, such as the average load of the system, the response time of processing requests, etc., so as to more accurately judge the actual load of the system. In addition, the index creation conditions can be dynamically adjusted based on the real-time load conditions, so as to more accurately adjust the creation and stopping of threads according to the actual load conditions of the system, so as to achieve better system performance and resource utilization.

[0098] S306 : In response to a second data query request for a target field in a database, query the database to see whether there is index data of the target field.

[0099] In an embodiment of the present application, since the index data of the target field has been obtained before, at this time, if the processing device obtains a second data query request for the target field in the database, the processing device can query whether there is index data for the target field from the database, such as by querying index metadata, database table structure, table index, etc., the index information of the target field can be obtained, and the index information may include index name, index type, index field, etc.

[0100] S307: If the index data of the target field exists in the database, determine the key value according to the index data of the target field.

[0101] In the embodiment of the present application, if the processing device determines that there is index data of the target field in the database, the key value, that is, the key information in the index, can be determined based on the index data of the target field. The key value corresponds to the value of the target field and can be used for subsequent query operations.

[0102] Exemplarily, taking the target field as a mobile phone number as an example, it is determined through step S306 that index data for the mobile phone number already exists, then the key value in the index data, such as the specific value of a mobile phone number, can be determined.

[0103] S308. Query the database for data matching the key value.

[0104] In the embodiment of the present application, by using the key value as the query condition, a query is performed in the database to obtain data records matching the key value, and the specific data corresponding to the target field can be obtained through the query operation. Through the above method, the queried data can be accurately located, the query scope can be effectively reduced and the query efficiency can be improved, unnecessary data scanning can be reduced, and the data access speed and system responsiveness can be improved.

[0105] In one embodiment, before executing step S306, the server may further execute the following steps:

[0106] (1) In response to a first data query request for a target field in a database, query the database to see whether there is index data of the target field.

[0107] (2) If the index data of the target field does not exist in the database, an index creation request for the target field in the database is generated, and the step of obtaining the index creation condition and subsequent steps are executed in response to the index creation request for the target field in the database.

[0108] In steps (1)-(2), when performing the first data query on the target field, if the target field in the database does not have a corresponding index, the processing device can automatically generate an index creation request and execute the index creation method in the aforementioned embodiment, thereby realizing automatic addition of the index when the target field has no index. This process does not require the operation and maintenance personnel to actively perform index creation operations, further improving the degree of automation of index creation.

[0109] It should be noted that the first data query request and the second data query request may be the same data query request or different data query requests, and this embodiment of the present application is not limited to this.

[0110] In one embodiment, the server may further perform the following steps:

[0111] (1) Determine the number of cores of the processor of the processing device corresponding to the index creation process.

[0112] (2) Determine the target number of threads according to the number of cores.

[0113] In steps (1)-(2), it is necessary to determine the number of processor cores of the processing device used to execute the index creation process, and then determine the number of threads (i.e., the target number) used for the index creation process according to the number of processor cores. This is to make full use of the computing resources of the processing device, reasonably allocate tasks and schedule threads, which helps to improve the efficiency and performance of index creation. Among them, the target number of threads can refer to a single thread (i.e., the target number is 1), and the target number of threads can also refer to multiple threads (i.e., the target number is n, where n>1).

[0114] In one embodiment, assuming that the number of cores of the processor of the processing device is m, then the processing device can use k*m as the target number (0<k<1, such as k = 0.5). In this way, it can ensure that there are enough threads to execute the index creation process (which can be executed either single-threaded or multi-threaded), make full use of the computing power of the processing device, speed up the index creation speed, and thus improve the efficiency of index creation and the performance of the overall system. At the same time, it also ensures that the core business processing of the processing device (processing other than the index creation process) is not affected, does not cause blockage to the system, does not compete for resources with the main business, and ensures the stability of the system operation.

[0115] Based on this, the processing device can perform index creation processing on the data operation records associated with the target field in the log data according to the following steps:

[0116] (1) Start the target number of threads.

[0117] In one embodiment, according to the previously determined target number, the server will create the corresponding number of threads and allocate corresponding resources to each thread to prepare for the corresponding index creation process.

[0118] (2) Use the target number of threads to sequentially perform index creation processing on each data operation record associated with the target field to obtain the index data of the target field; the log data is updated in real time according to the data operations on the database, and the order of the index creation processing corresponding to each data operation record matches the order of the data operations corresponding to each data operation record.

[0119] In one embodiment, the processing device uses a target number of threads to perform index creation processing on each data operation record associated with the target field in a preset order, and finally obtains the index data of the target field. During the processing, it is necessary to ensure that the order of index creation matches the order in which the data operations occur to ensure data consistency.

[0120] For example, assume that the processing device starts 4 threads to process the index creation operation. During the index creation process, the processing device will process the data operation records associated with the target field in the order of the data operation records. For example, if an INSERT operation record is the first line in the log, the corresponding index creation process will be executed in the first thread; and if another UPDATE operation record is the second line in the log, the corresponding index creation process will be executed in the second thread. By processing the data operation records in sequence, the index data of the target field is finally generated.

[0121] It should be noted that the log data is updated in real time based on the data operations on the database, that is, the database data operations (such as business queries, new additions, etc.) and the database index creation processing are performed separately. Whenever a new database data operation occurs, it will be recorded in the log data accordingly, and the database index creation processing will sequentially obtain the data operation records from the real-time updated log data for processing. This allows the database data operations and index creation processing to be performed synchronously during the elastic index creation process, without locking the database table, and will not cause system obstruction. At the same time, it can respond to business requests in a timely and fast manner.

[0122] By using threads to create indexes, single-threaded or concurrent processing can be implemented, thereby improving the efficiency and speed of index creation. Threads can independently execute index creation tasks without being interfered with by other tasks, and have good concurrency and parallelism. At the same time, threads can read the binlog information of the database and create indexes in sequence, ensuring data consistency and accuracy. By reasonably controlling the start and stop conditions of threads, dynamic adjustments can be made according to the status of the system to implement a flexible index creation mechanism.

[0123] In addition, when the processing device is not in an idle state (that is, when the index creation suspension condition is met), the created thread is usually paused, suspended or in a waiting state to avoid unnecessary resource occupation. When the system is judged to be in an idle state again (that is, when the index creation start condition is met), there is no need to re-create the thread for index creation, but the previously created thread can continue to resume the index creation task, thereby avoiding unnecessary thread creation and destruction overhead.

[0124] In one embodiment, the server may further perform the following steps:

[0125] (1) Determine the maximum number of threads and the number of idle threads supported by the processing device corresponding to the index creation process.

[0126] (2) Determine the updated number of threads based on the maximum number of threads and the number of idle threads.

[0127] (3) Using the updated number of threads, index creation processing is performed on each data operation record associated with the target field in turn to obtain index data of the target field.

[0128] In the above steps (1) and (3), the maximum number of threads and the number of idle threads can be used to evaluate the current processing capacity of the processing device. The processing device will calculate and determine the updated number of threads that can be used for index creation processing based on the previously determined maximum number of threads and the number of idle threads, and then use the updated number of threads to perform index creation processing on each data operation record associated with the target field, thereby realizing adaptive adjustment of the number of threads, more reasonable allocation of tasks and scheduling of threads, and helping to improve the efficiency and performance of index creation.

[0129] In one embodiment, the processing device may further perform the following steps: when the data update condition is met, the index data of the target field is updated to the database, so that a data query operation for the target field is performed according to the index data of the target field. The data update condition is an index creation suspension condition, or the data update condition is that the index creation processing for each data operation record associated with the target field has been completed, thereby improving the flexibility of index creation.

[0130] In one embodiment, the processing device may also perform index adjustment based on the performance detection data of the database, and the index adjustment may include index optimization and index deletion. The performance detection data may include query performance, response time, index usage, etc. The existing indexes are optimized through the performance detection data, specifically including adjusting the structure and configuration of the existing indexes to improve query performance and reduce disk space usage. The indexes that are no longer needed or redundant are deleted through the performance detection data to reduce the maintenance cost of the index and improve the overall performance of the database.

[0131] The following is an explanation of the data query process: Figure 4A , which is a flow chart of a data query provided by an embodiment of the present application, and the main steps are as follows:

[0132] (1.1) The user business application issues a query request to retrieve the query field A.

[0133] (1.2) The query request is parsed by the SQL DQL (Data Query Language) parser to determine whether field A needs to be queried. Then, the database table structure is checked to see whether there is an index for field A.

[0134] (1.3) If you find that field A has no index after querying the table structure, then perform a full table scan query directly.

[0135] (1.4) If you find that an index has been created for field A after querying the table structure, then query the data through the index of field A.

[0136] The above method may perform a full table scan when querying, which has a significant impact on system performance and overhead, increases system load, affects the response performance of other business modules, and takes a long time.

[0137] The following is an explanation of the process of creating an elastic index in the embodiment of the present application: Figure 4B , which is a flowchart of creating an elastic index provided by an embodiment of the present application, and the main steps are as follows:

[0138] (2.1) The database administrator (DBA) sends an index creation request for field A to the processing device.

[0139] (2.2) The automatic elastic index module identifies the index creation request and reads the elastic mechanism and threshold of the elastic index in the configuration file of the database (such as the index creation conditions, including the index creation start condition and the index creation pause condition).

[0140] For example, the elastic mechanism and threshold of the elastic index are configured in the my.cnf configuration file. The configuration content may be as shown in Table 2 below:

[0141] Table 2

[0142]

[0143] Among them, Elastic_index_thread can refer to the target number of threads used for index creation processing (such as the target number is 1), Elastic_index_start_Conn_pool_num can refer to the first number of connections, Elastic_index_start_System_Load can refer to the first system load, Elastic_index_stop_Conn_pool_num can refer to the second number of connections, and Elastic_index_stop_System_Load can refer to the second system load.

[0144] The index creation start condition can be set as follows: the number of connections in the database connection pool is less than 10, and the system load is less than 0.5. Based on these two indicators, it can be determined that the current system is idle, and a thread (single thread or multi-thread) can be started to create the index. If the number of connections in the connection pool increases to more than 50 during the index creation process, or the system load is greater than or equal to 1 (i.e., the index creation suspension condition is met), the index creation is stopped and the next system idle period is waited for. The above process is automatically performed, and the index creation is continuously performed through elastic dynamics until the index creation is completed.

[0145] (2.3) Start the thread to perform index creation processing.

[0146] (2.4) During the continuous index creation process, read the data operation records corresponding to field A in the database binlog, and create corresponding index files for each data operation record in sequence.

[0147] During this process, the processing device will continuously detect whether the index creation conditions are met, including the index creation start conditions and the index creation pause conditions. When the index creation start conditions are met, the index is continuously created according to the table data based on the binlog information; when the index creation pause conditions are met, the index creation is paused and the index creation thread is dormant.

[0148] (2.5) Continuously read binlog information (i.e. log data) to continuously create indexes.

[0149] (2.6) After the index is created, modify the table structure definition of the database and mark the index as created to facilitate subsequent determination of whether the index exists.

[0150] After (2.6), when a data query request for field A in the database is obtained, the processing device can query whether there is index data for field A in the database. If there is an index, the index is used to query the data. The processing process is the same as step (1.4).

[0151] During the entire elastic index creation process, business query and addition operations can be performed synchronously without locking the table, causing no blockage to the system, and not competing for resources with the main business. In addition, the entire elastic index creation process is stable and secure, and business operations are simple, making index creation less risky. The above method can be applied to various relational databases, such as MYSQL database, and can be deployed in a stand-alone environment.

[0152] Based on the above embodiments, the beneficial effects of the present application are as follows: the index creation method provided in the embodiments of the present application is a flexible, low-cost index creation mechanism. The system can create indexes in an elastic and scalable manner, and can automatically start an elastic index creation mechanism. The index creation process will not consume too many IO and CPU resources. Users will not be aware of it when using the database, and it will not affect online business. It can also improve the convenience and security of online operation indexes. At the same time, it can make up for the defects of the current relational database in the big data scenario, such as the imperfect transaction processing mechanism of online DDL index creation and the rough, reckless and uncontrollable logic processing.

[0153] The index creation method provided in this application can improve the following system performance or business characteristics:

[0154] 1. Improve the editability and security of online DDL operation and maintenance in big data scenarios: The automatic elastic index module can improve the defects of the current index creation method, improve the security of business operations, and standardize database security operation management.

[0155] 2. Optimize the problem of overall system performance degradation or blocking during index creation: The elastic index creation process will dynamically analyze the system load and operating status, and will not compete with the business for resources. On the premise of meeting the business operation, it will occupy fewer resources for index creation, thereby minimizing the overall index creation cost.

[0156] 3. Better support for public welfare and high-availability security services: The elastic index creation method can meet the business processing requirements of public welfare and other services that require high availability and high performance. Index performance can be optimized at any time to improve system operation efficiency and performance.

[0157] 4. Reduce system operation and maintenance costs: The elastic index creation method can reduce the requirements for operation and maintenance personnel and systems, improve operation and maintenance efficiency, and reduce operation and maintenance errors and accident rates.

[0158] See also Figure 5 , which is a schematic block diagram of an index creation device provided in an embodiment of the present application. The index creation device may specifically include:

[0159] The acquisition module 501 is used to obtain index creation conditions in response to an index creation request for a target field in a database, wherein the index creation conditions include an index creation start condition and an index creation pause condition;

[0160] Processing module 502, for performing index creation processing on the data operation records associated with the target field in the log data if it is determined according to the first state data associated with the database that the index creation start condition is met; the log data records include data operation records associated with each field in the database;

[0161] The processing module 502 is further configured to suspend the index creation process if it is determined according to the second state data associated with the database that the index creation suspension condition is met, so as to use the processing resources of the index creation process to perform business processing on the database, where the business processing is different from the index creation process;

[0162] The processing module 502 is further configured to restart the index creation process to obtain the index data of the target field if it is determined that the index creation start condition is met according to the third state data associated with the database.

[0163] Optionally, when the processing module 502 is used to determine that the index creation start condition is met according to the first state data associated with the database, it is specifically used to:

[0164] Acquire first status data associated with the database; the first status data includes a first system load and a first number of connections in a connection pool of the database;

[0165] If the first system load is less than the first load threshold, and the first number of connections is less than the first connection threshold, it is determined that the index creation start condition is met.

[0166] Optionally, when the processing module 502 is used to determine that the index creation suspension condition is met according to the second state data associated with the database, it is specifically used to:

[0167] Acquire second status data associated with the database; the second status data includes a second system load and a second number of connections in a connection pool of the database;

[0168] If the second system load is greater than or equal to the second load threshold, or the second number of connections is greater than or equal to the second connection threshold, it is determined that the index creation suspension condition is met.

[0169] Optionally, when the processing module 502 is used to perform index creation processing on the data operation record associated with the target field in the log data, it is specifically used to:

[0170] Perform data parsing on the data operation records associated with the target fields in the log data to obtain parsed data fields;

[0171] Generate an index creation statement according to the above parsed data field, and perform index creation processing according to the above index creation statement.

[0172] Optionally, the processing module 502 is further configured to:

[0173] In response to a first data query request for a target field in a database, querying from the database whether there is index data for the target field;

[0174] If the index data of the target field does not exist in the database, the index creation request for the target field in the database is generated, and the step of obtaining the index creation conditions in response to the index creation request for the target field in the database and subsequent steps are executed.

[0175] Optionally, the processing module 502 is further configured to:

[0176] In response to a second data query request for a target field in a database, querying the database whether there is index data for the target field;

[0177] If the index data of the target field exists in the database, determining the key value according to the index data of the target field;

[0178] Search the database for data matching the key value.

[0179] Optionally, the processing module 502 is further configured to:

[0180] Determine the number of cores of the processor of the processing device corresponding to the index creation process;

[0181] Determine the target number of threads based on the number of cores above;

[0182] The processing module 502, when used to perform index creation processing on the data operation record associated with the target field in the log data, is specifically used to:

[0183] Start the target number of threads mentioned above;

[0184] Utilizing the target number of threads, index creation processing is performed on each data operation record associated with the target field in turn to obtain index data of the target field; the log data is updated in real time according to the data operations on the database, and the order of index creation processing corresponding to each of the data operation records matches the order of occurrence of the data operations corresponding to each of the data operation records.

[0185] It should be noted that the functions of each functional module of the index creation device in the embodiment of the present application can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description of the above method embodiment, which will not be repeated here.

[0186] See also Figure 6 , which is a schematic block diagram of a computer device provided in an embodiment of the present application. Figure 6 The computer device in the embodiment shown may include: a processor 601, a storage device 602, and a communication interface 603. The processor 601, the storage device 602, and the communication interface 603 may exchange data.

[0187] The storage device 602 may include a volatile memory, such as a random-access memory (RAM); the storage device 602 may also include a non-volatile memory, such as a flash memory, a solid-state drive, or a memory device.

[0188] (solid-state drive, SSD), etc.; the above-mentioned storage device 602 may also include a combination of the above-mentioned types of memory.

[0189] The processor 601 may be a central processing unit (CPU). In one embodiment, the processor 601 may also be a graphics processing unit (GPU). The processor 601 may also be a combination of a CPU and a GPU. In one embodiment, the storage device 602 is used to store program instructions, and the processor 601 may call the program instructions to perform the following operations:

[0190] In response to an index creation request for a target field in a database, obtaining index creation conditions, the index creation conditions including an index creation start condition and an index creation pause condition;

[0191] If it is determined according to the first state data associated with the database that the index creation start condition is met, index creation processing is performed on the data operation records associated with the target field in the log data; the log data records include data operation records associated with each field in the database;

[0192] If it is determined according to the second state data associated with the database that the index creation suspension condition is met, suspending the index creation process so as to use the processing resources of the index creation process for business processing for the database, the business processing being different from the index creation process;

[0193] If it is determined according to the third state data associated with the database that the index creation start condition is met, the index creation process is restarted to obtain the index data of the target field.

[0194] Optionally, when the processor 601 is used to determine that the index creation start condition is met according to the first state data associated with the database, it is specifically used to:

[0195] Acquire first status data associated with the database; the first status data includes a first system load and a first number of connections in a connection pool of the database;

[0196] If the first system load is less than the first load threshold, and the first number of connections is less than the first connection threshold, it is determined that the index creation start condition is met.

[0197] Optionally, when the processor 601 is used to determine that the index creation suspension condition is met according to the second state data associated with the database, it is specifically used to:

[0198] Acquire second status data associated with the database; the second status data includes a second system load and a second number of connections in a connection pool of the database;

[0199] If the second system load is greater than or equal to the second load threshold, or the second number of connections is greater than or equal to the second connection threshold, it is determined that the index creation suspension condition is met.

[0200] Optionally, when the processor 601 is used to perform index creation processing on the data operation record associated with the target field in the log data, it is specifically used to:

[0201] Perform data parsing on the data operation records associated with the target fields in the log data to obtain parsed data fields;

[0202] Generate an index creation statement according to the above parsed data field, and perform index creation processing according to the above index creation statement.

[0203] Optionally, the processor 601 is further configured to:

[0204] In response to a first data query request for a target field in a database, querying from the database whether there is index data for the target field;

[0205] If the index data of the target field does not exist in the database, the index creation request for the target field in the database is generated, and the step of obtaining the index creation conditions in response to the index creation request for the target field in the database and subsequent steps are executed.

[0206] Optionally, the processor 601 is further configured to:

[0207] In response to a second data query request for a target field in a database, querying the database whether there is index data for the target field;

[0208] If the index data of the target field exists in the database, determining the key value according to the index data of the target field;

[0209] Search the database for data matching the key value.

[0210] Optionally, the processor 601 is further configured to:

[0211] Determine the number of cores of the processor of the processing device corresponding to the index creation process;

[0212] Determine the target number of threads based on the number of cores above;

[0213] The processing module 502, when used to perform index creation processing on the data operation record associated with the target field in the log data, is specifically used to:

[0214] Start the target number of threads mentioned above;

[0215] Utilizing the target number of threads, index creation processing is performed on each data operation record associated with the target field in turn to obtain index data of the target field; the log data is updated in real time according to the data operations on the database, and the order of index creation processing corresponding to each of the data operation records matches the order of occurrence of the data operations corresponding to each of the data operation records.

[0216] In a specific implementation, the processor 601, the storage device 602, and the communication interface 603 described in the embodiments of the present application can execute the aforementioned embodiments of the present application. Figure 2 or Figure 3 The implementation method described in the related embodiments of the provided index creation method can also be implemented in the embodiments of the present application. Figure 5 The implementation method described in the relevant embodiments of the provided index creation device will not be repeated here.

[0217] In the several embodiments provided in the present application, it should be understood that the disclosed methods, devices and systems can be implemented in other ways. For example, the device embodiments described above are merely schematic; for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0218] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the aforementioned index creation device, and the computer program includes program instructions. When the processor executes the above program instructions, it can execute the method in the aforementioned embodiment, so it will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the program instructions can be deployed on a computer device, or executed on multiple computer devices located at one location, or, executed on multiple computer devices distributed in multiple locations and interconnected by a communication network, and multiple computer devices distributed in multiple locations and interconnected by a communication network can constitute a blockchain system.

[0219] According to one aspect of the present application, a computer program product is provided, the computer program product comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device can execute the method in the aforementioned embodiment, and therefore, will not be described in detail here.

[0220] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the above-mentioned program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The above-mentioned storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0221] The above disclosure is only part of the embodiments of the present application, which certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and making equivalent changes according to the claims of this application are still within the scope of the invention.

Claims

1. A method for creating an index, characterized in that: The method comprises: In response to an index creation request for a target field in a database, obtaining index creation conditions, wherein the index creation conditions include an index creation start condition and an index creation pause condition; If it is determined according to the first state data associated with the database that the index creation start condition is met, index creation processing is performed on the data operation record associated with the target field in the log data; the log data record contains the data operation record associated with each field in the database; If it is determined according to the second state data associated with the database that the index creation suspension condition is met, suspending the index creation process so as to use the processing resources of the index creation process for business processing on the database, the business processing being different from the index creation process; If it is determined according to the third state data associated with the database that the index creation start condition is met, the index creation process is restarted to obtain the index data of the target field.

2. The method according to claim 1, characterized in that If it is determined according to the first state data associated with the database that the index creation start condition is met, it includes: Acquire first status data associated with the database; the first status data includes a first system load and a first number of connections in a connection pool of the database; If the first system load is less than a first load threshold, and the first number of connections is less than a first connection threshold, it is determined that the index creation start condition is met.

3. The method according to claim 1, characterized in that: If it is determined according to the second state data associated with the database that the index creation suspension condition is met, it includes: Acquire second state data associated with the database; the second state data includes a second system load and a second number of connections in a connection pool of the database; If the second system load is greater than or equal to a second load threshold, or the second number of connections is greater than or equal to a second connection threshold, it is determined that the index creation suspension condition is met.

4. The method according to any one of claims 1 to 3, characterized in that The index creation process for the data operation record associated with the target field in the log data includes: Performing data parsing processing on the data operation records associated with the target field in the log data to obtain parsed data fields; An index creation statement is generated according to the parsed data field, and index creation processing is performed according to the index creation statement.

5. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: In response to a first data query request for a target field in a database, querying the database whether there is index data for the target field; If the index data of the target field does not exist in the database, an index creation request for the target field in the database is generated, and the step of obtaining index creation conditions in response to the index creation request for the target field in the database and subsequent steps are executed.

6. The method according to any one of claims 1 to 3, characterized in that The method further comprises: In response to a second data query request for a target field in a database, querying the database whether there is index data for the target field; If the index data of the target field exists in the database, determining the key value according to the index data of the target field; The database is searched for data matching the key value.

7. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Determining the number of cores of a processor of a processing device corresponding to the index creation process; determining a target number of threads based on the number of cores; The index creation process for the data operation record associated with the target field in the log data includes: Start the target number of threads; Utilizing the target number of threads, index creation processing is performed on each data operation record associated with the target field in turn to obtain index data of the target field; the log data is updated in real time according to the data operations on the database, and the order of index creation processing corresponding to each data operation record matches the order of occurrence of the data operations corresponding to each data operation record.

8. An index creation device, characterized in that: The device comprises: An acquisition module, configured to acquire index creation conditions in response to an index creation request for a target field in a database, wherein the index creation conditions include an index creation start condition and an index creation pause condition; A processing module, configured to perform index creation processing on the data operation record associated with the target field in the log data if it is determined according to the first state data associated with the database that the index creation start condition is met; the log data record includes the data operation record associated with each field in the database; The processing module is further configured to suspend the index creation process if it is determined according to the second state data associated with the database that the index creation suspension condition is met, so as to use the processing resources of the index creation process for business processing on the database, the business processing being different from the index creation process; The processing module is further configured to restart the index creation process to obtain the index data of the target field if it is determined that the index creation start condition is met according to the third state data associated with the database.

9. A computer device, characterized in that: include: A processor, a storage device and a communication interface, wherein the processor, the communication interface and the storage device are interconnected, wherein the storage device stores an executable program code, and the processor is used to call the executable program code to implement the index creation method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, they are used to implement the index creation method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product comprises a computer program or a computer instruction, and when the computer program or the computer instruction is executed by a processor, it is used to implement the index creation method according to any one of claims 1 to 7.

Images