Index updating method and device, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the field of data processing, and relates to an index updating method and device, computer equipment and a storage medium, and the method comprises the following steps: obtaining data index information; calculating an index expansion ratio according to the data index information; judging whether the index expansion ratio is greater than or equal to a preset expansion threshold; if yes, concurrent index reconstruction is conducted on the data index information according to a simulated annealing algorithm, index state updating is conducted according to the reconstructed data index information, and first real-time index information is obtained; and if the index expansion rate is smaller than the index expansion rate, calculating an expansion level corresponding to the index expansion rate, and calling a corresponding updating task according to the expansion level to update the index state of the data index information to obtain second real-time index information. The data index updating method and device can be applied to the field of digital medical treatment, the data index can be quickly and effectively updated, and it is guaranteed that the system query performance cannot be reduced due to expansion of the data index.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, specifically to the field of digital medicine, and in particular to an index updating method, device, computer equipment and storage medium. Background Art

[0002] In modern business systems, databases play a vital role as the core information storage and retrieval engine. They not only carry massive amounts of key data, but also support managers' instant access to information and decision support through efficient query mechanisms. However, with the increasing frequency and complexity of business activities, the speed of data updates in databases has increased dramatically, causing a lot of trouble for database performance management.

[0003] Index is a key technology for optimizing query performance in databases. Its effectiveness directly determines the efficiency of data retrieval. However, in business systems, due to the dynamic changes of information (such as the entry of new data, the adjustment of data tables, the update of data content, etc.), data tables often face frequent insertion, deletion and update operations. If these operations are not properly managed, it is very easy to cause index fragmentation, which in turn causes index expansion problems. Index expansion not only increases the maintenance cost of the database, but more importantly, it will significantly reduce query performance and prolong response time. It also causes inefficient use of storage space and increases the operating cost of the business system. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an index updating method, apparatus, computer equipment and storage medium to solve the problem that medical index information cannot be updated effectively and quickly.

[0005] In order to solve the above technical problems, the present application embodiment provides an index update method, which adopts the following technical solution:

[0006] Get data index information;

[0007] Calculate the index expansion ratio according to the data index information;

[0008] Determining whether the index expansion ratio is greater than or equal to a preset expansion threshold;

[0009] If the index expansion ratio is greater than or equal to the preset expansion threshold, the data index information is concurrently reindexed according to a simulated annealing algorithm, and the index state is updated according to the reconstructed data index information to obtain first real-time index information;

[0010] If the index expansion ratio is less than the preset expansion threshold, the expansion level corresponding to the index expansion ratio is calculated, and the corresponding update task is called according to the expansion level to update the index status of the data index information to obtain second real-time index information.

[0011] Furthermore, before the step of obtaining the data index information, the following steps are also included:

[0012] Acquire data from multiple sources;

[0013] Performing data cleaning and format normalization processing on the multi-source data to obtain standard multi-source data;

[0014] Extracting key elements from the standard multi-source data to obtain key information of the data;

[0015] Building a data index library based on the key data information;

[0016] Acquire historical query information, and filter the data index library according to the historical query information to obtain the data index information.

[0017] Furthermore, the step of obtaining data index information specifically includes:

[0018] Get index information identifier;

[0019] Extracting original data index information from a preset database according to the index information identifier;

[0020] The original data index information is preprocessed to obtain the data index information.

[0021] Furthermore, the step of calculating the index expansion ratio according to the data index information specifically includes:

[0022] Obtain the index data size and index data structure of data index information;

[0023] Calculate the index data space according to the index data size and the index data structure;

[0024] The preset index occupied space is obtained, and the index expansion ratio is calculated according to the index data space and the preset index occupied space.

[0025] Furthermore, the step of concurrently reindexing the data index information according to the simulated annealing algorithm, and updating the index state according to the reconstructed data index information to obtain the first real-time index information specifically includes:

[0026] Dividing the data index information into data sets to obtain data index subsets;

[0027] Acquire a data index structure of the data index subset, and select an initial data index structure from the data index structure;

[0028] Get the preset simulated annealing parameters;

[0029] Allocating the data index subsets to a plurality of processing threads, performing simulated annealing calculations according to the preset simulated annealing parameters and the initial data index structure, and using the calculated optimal solution as the optimal data index structure;

[0030] Rebuilding the index according to the optimal data index structure to obtain valid data index information;

[0031] The data index information of the current record is updated according to the valid data index information to obtain the first real-time index information.

[0032] Furthermore, the step of performing index reconstruction according to the optimal data index structure to obtain valid data index information specifically includes:

[0033] Deleting the data index information;

[0034] Acquire index structure information of the optimal data index structure, wherein the index structure information includes index key, index type, and index partition;

[0035] Create an index according to the index key, the index type, and the index partition to obtain optimized data index information;

[0036] The optimized data index information is subjected to index verification, and the optimized data index information that passes the verification is used as the valid data index information.

[0037] Further, the step of calculating the expansion level corresponding to the index expansion ratio, and calling the corresponding update task according to the expansion level to update the index state of the data index information to obtain the second real-time index information specifically includes:

[0038] Performing a difference calculation based on the index expansion ratio and the preset expansion threshold to obtain an expansion difference;

[0039] Obtain a preset expansion level mapping table, and identify a corresponding expansion level in the expansion level mapping table according to the expansion difference;

[0040] Acquire an expansion level information identifier corresponding to the expansion level, and extract the update task from a database according to the expansion level information identifier;

[0041] The data index information is adjusted according to the update task, and the currently recorded data index information is updated according to the adjusted data index information to obtain the second real-time index information.

[0042] In order to solve the above technical problems, the embodiment of the present application further provides an index updating device, which adopts the following technical solution:

[0043] Information acquisition module, used to obtain data index information;

[0044] A ratio calculation module, used for calculating the index expansion ratio according to the data index information;

[0045] A data judgment module, used to judge whether the index expansion ratio is greater than or equal to a preset expansion threshold;

[0046] A first processing module is used for concurrently re-indexing the data index information according to a simulated annealing algorithm if the index expansion ratio is greater than or equal to the preset expansion threshold, and updating the index state according to the reconstructed data index information to obtain first real-time index information;

[0047] The second processing module is used to calculate the expansion level corresponding to the index expansion ratio if the index expansion ratio is less than the preset expansion threshold, and call the corresponding update task according to the expansion level to update the index status of the data index information to obtain the second real-time index information.

[0048] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0049] A computer device comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of any of the above index updating methods when executing the computer-readable instructions.

[0050] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0051] A computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the index updating method described in any one of the above items are implemented.

[0052] Compared with the prior art, the embodiment of the present application has the following beneficial effects: the embodiment obtains data index information; calculates the index expansion ratio according to the data index information; determines whether the index expansion ratio is greater than or equal to the preset expansion threshold; if the index expansion ratio is greater than or equal to the preset expansion threshold, the data index information is concurrently reindexed according to the simulated annealing algorithm, and the index status is updated according to the reconstructed data index information to obtain the first real-time index information; if the index expansion ratio is less than the preset expansion threshold, the expansion level corresponding to the index expansion ratio is calculated, and the corresponding update task is called according to the expansion level to update the index status of the data index information to obtain the second real-time index information. This enables the data index to be updated quickly and effectively to ensure that the data index will not cause the system query performance to decline due to expansion. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0055] Figure 2 A flowchart according to an embodiment of the index updating method of the present application;

[0056] Figure 3 yes Figure 2 A flowchart of a specific implementation of step S10;

[0057] Figure 4 yes Figure 2 A flowchart of a specific implementation of step S20;

[0058] Figure 5 yes Figure 2 A flowchart of a specific implementation of step S40;

[0059] Figure 6 yes Figure 5 A flowchart of a specific implementation of step S405;

[0060] Figure 7 yes Figure 2 A flowchart of a specific implementation of step S50;

[0061] Figure 8is a structural diagram of an embodiment of an index updating device according to the present application;

[0062] Fig. 9 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0064] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor does it refer to non-related or alternative embodiments that are mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0065] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0066] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0067] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0068] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to the laptop computer 1011, tablet computer 1012 or mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0069] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0070] It should be noted that the index updating method provided in the embodiment of the present application is generally executed by a server, and accordingly, the index updating device is generally arranged in the server.

[0071] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0072] Continue to refer Figure 2 , shows a flow chart of an embodiment of the method for image recognition according to the present application. The index updating method comprises the following steps:

[0073] Step S10, obtaining data index information;

[0074] In this embodiment, the data index information may be medical data index information, which includes a patient basic information index: including the patient's name, ID number, contact information, home address and other basic information, which is used to quickly locate the patient's relevant medical records; a medical history index: recording the time, location, diagnosis results, treatment plan, etc. of the patient's previous medical visits, which is used to provide doctors with comprehensive patient medical history information; a doctor's personal profile index: including the doctor's name, title, professional field, hospital and other information, which is used for patients to choose a suitable doctor for medical treatment; a doctor's medical history index: recording the information, diagnosis results, treatment plans, etc. of patients that the doctor has seen, which is used to provide a basis for the doctor's personal performance appraisal and medical quality assessment; a hospital basic information index: including the hospital name, address, etc. Address, contact information, hospital size, hospital grade, hospital department settings, etc., for patients to understand the overall situation of the hospital; Medical resource index: including the hospital's medical equipment, number of medical staff, number of beds, etc., to reflect the hospital's medical service capabilities; Electronic medical record index: detailed records of the patient's medical history, diagnosis results, treatment plan, medication records, etc.; Medical imaging index: including index information of medical imaging images such as X-rays, CT scans, MRI, etc., for doctors to quickly find and access patients' imaging data; Health habit index: records the patient's health habits, lifestyle and other information, which is used to provide a basis for formulating personalized health management and preventive measures; Family medical history index: records the patient's family medical history information, which is used to assess the patient's genetic risk and formulate targeted diagnosis and treatment plans.

[0075] Step S20, calculating the index expansion ratio according to the data index information;

[0076] In this embodiment, the index expansion ratio refers to the ratio of the index data space size corresponding to the current data index information to the preset index space size allocated by the system, and the ratio can be a percentage value or a decimal value. Among them, the index data space refers to the space occupied by the overall data of the data index information in the system memory, and the size of the index data space is affected by the index data size and the index data structure. The preset index space refers to the space that can be occupied by the default allocated index data in the system, and the size of the preset index space can be adjusted accordingly according to the actual situation.

[0077] Step S30, determining whether the index expansion ratio is greater than or equal to a preset expansion threshold;

[0078] In this embodiment, the numerical unit standard of the index expansion ratio and the preset expansion threshold is the same. By comparing the numerical values ​​of the index expansion ratio and the preset expansion threshold, the size relationship between the index expansion ratio and the preset expansion threshold is determined to facilitate subsequent further processing based on the size relationship.

[0079] Step S40: if the index expansion ratio is greater than or equal to the preset expansion threshold, the data index information is concurrently reindexed according to a simulated annealing algorithm, and the index state is updated according to the reconstructed data index information to obtain first real-time index information;

[0080] In this embodiment, the simulated annealing algorithm (SA) is a general probabilistic optimization algorithm used to find an approximate global optimal solution to a problem in a large search space. This embodiment uses the simulated annealing algorithm to optimize the data index structure of the data index information, thereby obtaining an optimal data retrieval structure, and then rebuilds the index based on the optimal data retrieval structure. The index status is updated with the valid data index information after the index reconstruction, thereby obtaining the first real-time index information.

[0081] Step S50: If the index expansion ratio is less than the preset expansion threshold, the expansion level corresponding to the index expansion ratio is calculated, and the corresponding update task is called according to the expansion level to update the index status of the data index information to obtain second real-time index information.

[0082] In this embodiment, different index expansion ratios correspond to different expansion levels, and different expansion levels correspond to different update tasks. In this embodiment, the expansion levels are initially set to level one, level two, and level three. The level one expansion level corresponds to the first update task, the level two expansion level corresponds to the second update task, and the level three expansion level corresponds to the third update task. The first update task is to perform basic index cleaning and compression operations, including removing invalid and expired index items, and performing simple compression on the index structure to optimize storage space and query efficiency. The second update task is to perform index optimization operations to eliminate index fragmentation and improve index query functions. The third update task is to adjust the index strategy (such as the granularity, quantity, type, etc. of the index) to optimize index performance.

[0083] In this embodiment, the above method can be applied to a medical service system, in which the system index information is optimized and updated to ensure the real-time validity of accessing medical data. Specifically, in this embodiment, the medical service system can be one or more of a medical insurance system and a disease insurance system, and the data index information is the medical data index information corresponding to the medical-related data in the medical service system, and the medical data index information is stored in the medical insurance system and the disease insurance system and obtained from the database of the above system, and the first real-time index information and the second real-time index information are stored in the above system to facilitate subsequent update record viewing.

[0084] This embodiment obtains data index information; calculates the index expansion ratio according to the data index information; determines whether the index expansion ratio is greater than or equal to a preset expansion threshold; if the index expansion ratio is greater than or equal to the preset expansion threshold, the data index information is concurrently reindexed according to the simulated annealing algorithm, and the index status is updated according to the reconstructed data index information to obtain the first real-time index information; if the index expansion ratio is less than the preset expansion threshold, the expansion level corresponding to the index expansion ratio is calculated, and the corresponding update task is called according to the expansion level to update the index status of the data index information to obtain the second real-time index information. This enables the data index to be updated quickly and effectively to ensure that the data index will not cause the system query performance to decline due to expansion.

[0085] In some optional implementations of this embodiment, before step S10, the following steps are further included:

[0086] Acquire data from multiple sources;

[0087] In this embodiment, multi-source data is system data from different data sources. In this embodiment, the source of multi-source data can be multi-source medical data. The data sources of multi-source medical data include hospital information system (HIS), electronic medical record system (EMR), laboratory information system (LIS), image archiving and communication system (PACS), telemedicine platform, health monitoring equipment, etc. Multi-source medical data includes hospital information system data: including comprehensive medical data such as basic information of patients, medical records, diagnostic information, doctor's orders, drug use, and fee settlement. Electronic medical record system data: including medical record documents (such as medical record homepage, medical history record, surgical record), examination and test results, nursing records, etc. Laboratory information system data: including reports and data analysis of various test items such as hematology, biochemistry, immunology, and microbiology. Image archiving and communication system data: medical imaging data, such as X-rays, CT, MRI and other image data, as well as related diagnostic reports. Telemedicine platform data: remote consultation records and remote monitoring data generated by platforms with functions such as remote consultation, remote monitoring, and remote teaching. Health monitoring equipment: personal health data collected by wearable devices, home health monitoring instruments, etc., such as heart rate, blood pressure, blood sugar, number of steps, sleep quality, etc. Medical insurance system: including patient medical insurance information, reimbursement records, payment status and other data.

[0088] Performing data cleaning and format normalization processing on the multi-source data to obtain standard multi-source data;

[0089] In this embodiment, data cleaning includes: identifying and processing missing values ​​(such as deleting and filling missing values), processing outliers (such as deleting, correcting or retaining and specially marking), correcting data errors (such as correcting data types, formats, logic), and removing duplicate values ​​(identifying and deleting identical data records); format normalization includes: unifying data types (converting data into a unified format), standardized coding (unifying multiple coding formats), data standardization (eliminating differences between numerical data and text data), data integration, and data verification.

[0090] Extracting key elements from the standard multi-source data to obtain key information of the data;

[0091] In this embodiment, the key data information may be the key medical data information, which includes basic patient information such as name, gender, age, ID number, contact information, etc. Diagnosis information: disease name, ICD code, diagnosis date, diagnosing doctor, etc. Treatment information: surgery name, surgery date, surgeon, drug name, dosage, medication time, etc. Examination results: laboratory test results, imaging examination results, examination date, examining doctor, etc. Cost information: medical expenses, payment method, reimbursement status, etc. Key elements can be extracted through natural language processing, regular expression matching, etc.

[0092] Building a data index library based on the key data information;

[0093] In this embodiment, the construction of the data index library includes the following steps: designing the index structure, determining the index fields (i.e., key information elements), index types (such as full-text index, hash index, B-tree index, etc.), index levels and relationships, etc. Selecting an indexing tool, such as the indexing function in a relational database management system (such as MySQL, Oracle), or a specialized search engine (such as Elasticsearch, Solr) based on factors such as data size, query requirements, and system environment. Establishing an index, using the selected indexing tool to index the key information of the data according to the designed index structure. Optimizing the index, optimizing the index based on query performance feedback, such as adjusting index parameters, increasing cache, optimizing query statements, etc.

[0094] Acquire historical query information, and filter the data index library according to the historical query information to obtain the data index information.

[0095] In this embodiment, the step of filtering information on the data index library includes analyzing historical queries: counting query frequency, query type, query keywords, etc., identifying high-frequency queries and hot data; adjusting index strategies: adjusting index strategies based on historical query analysis results, such as increasing index weights for high-frequency query keywords, optimizing query paths, etc.; information filtering: when executing a new query, using the adjusted index strategy to filter information on the index library to quickly locate the data most relevant to the query request to obtain data index information.

[0096] This embodiment obtains multi-source data; performs data cleaning and format normalization on the multi-source data to obtain standard multi-source data; extracts key elements from the standard multi-source data to obtain data key information; builds a data index library based on the data key information; obtains historical query information, and screens the data index library according to the historical query information, thereby effectively obtaining data index information collected and processed based on multiple different sources to provide reliable data support for subsequent operations.

[0097] Continue to refer Figure 3 In some optional implementations of this embodiment, step S10 includes the following steps:

[0098] Step S101, obtaining an index information identifier;

[0099] In this embodiment, the index information identifier is an information identifier generated based on the search term or query term input in the user interface, and the information identifier contains information content corresponding to the search term or query term, such as a specific patient ID, disease name, date range, etc.

[0100] Step S102, extracting original data index information from a preset database according to the index information identifier;

[0101] In this embodiment, the index information identifier is a unique information identifier pointing to the original data index information, and a matching query is performed in a preset database through the index information identifier to extract the original data index information.

[0102] Step S103, preprocessing the original data index information to obtain the data index information.

[0103] In this embodiment, preprocessing includes data cleaning: deleting or correcting erroneous data, processing missing values, removing duplicate records, etc. Data conversion: converting data into a unified format or unit, such as converting a date string into a date object, unifying different coding systems, etc. Data standardization: standardizing system data to ensure that data from different sources use the same terms when describing the same thing. By preprocessing the raw data index information, it is ensured that the acquired data index information meets the system processing standards.

[0104] This embodiment obtains index information identifiers; extracts original data index information from a preset database according to the index information identifier; and preprocesses the original data index information to obtain reliable and effective data index information to facilitate subsequent calculation operations of the index expansion ratio.

[0105] Continue to refer Figure 4 In some optional implementations of this embodiment, step S20 includes the following steps:

[0106] Step S201, obtaining the index data size and index data structure of the data index information;

[0107] In this embodiment, the index data size refers to the amount of storage space occupied by the index itself, which is usually measured in bytes, kilobytes, megabytes or larger units. The size depends on the amount of data contained in the index, the data type (such as integers, floating point numbers, strings, etc.) and the complexity of the index (such as single-column indexes, composite indexes, etc.). The index data structure determines the storage and access methods of the index data. Common index data structures include B-trees (such as B+ trees), hash tables, bitmap indexes, etc. Different data structures have different effects on the performance of the index (such as query speed, insertion / deletion performance) and storage space.

[0108] Step S202, calculating the index data space according to the index data size and the index data structure;

[0109] In this embodiment, the step of calculating the index data space includes: obtaining the data type of the index column, such as integer, floating point number, string, etc., and their storage requirements; estimating the space required for each index item (i.e., an entry in the index) according to the characteristics of the data type and the index structure. For example, in a B+ tree index, an index item may include a key value, a pointer (or an offset), and possible additional information (such as a transaction ID, a delete marker, etc.); determining the amount of index data: obtaining the amount of data that the index will contain, that is, the number of tables or rows in the table that the index will cover; calculating the total storage space required for the index according to the size of the index item, the overhead of the index structure, and the amount of index data, including estimating the hierarchical structure of the index structure, the number of nodes, the number of index items per node, etc., so as to obtain the index data space result occupied by the index.

[0110] Step S203, obtaining a preset index occupied space, and calculating the index expansion ratio according to the index data space and the preset index occupied space.

[0111] In this embodiment, the index expansion ratio is calculated according to the following formula: index expansion ratio=index data space / preset index occupied space, wherein the calculation result value of the index expansion ratio can be a percentage value or a decimal value.

[0112] This embodiment obtains the index data size and index data structure of the data index information; calculates the index data space according to the index data size and the index data structure; obtains the preset index occupied space, and calculates the index expansion ratio according to the index data space and the preset index occupied space. This effectively realizes the calculation of the data index expansion ratio according to the data space corresponding to the data index information, so as to facilitate the subsequent comparison and judgment operation.

[0113] Continue to refer Figure 5 In some optional implementations of this embodiment, step S40 includes the following steps:

[0114] Step S401, dividing the data index information into data sets to obtain data index subsets;

[0115] In this embodiment, the data index information may be medical data index information, and the data index subset may be a medical data index subset. For example, it may be initially divided according to the categories of medical data index information, such as information release index information (such as the establishment and scale of health institutions, distribution of health personnel resources, collection and allocation of health funds, etc.), and business construction index information (such as medical services, public health, drug supply, medical insurance, health management and other fields), etc., to obtain medical data index information sets, and then secondary division may be performed according to the attributes of the medical data index information sets (such as time, region, disease, discipline, institution, etc.) to obtain medical data index subsets.

[0116] Step S402, obtaining a data index structure of the data index subset, and selecting an initial data index structure from the data index structure;

[0117] In this embodiment, the data index structure is extracted from each data index subset, and one or more of them are selected as the initial data index structure. The selection method of the initial data index structure can be random, and the data index structure will be used as the starting point of the simulated annealing algorithm.

[0118] Step S403, obtaining preset simulated annealing parameters;

[0119] In this embodiment, the preset simulated annealing parameters refer to the parameters set by the simulated annealing algorithm, such as initial temperature, cooling rate, number of iterations, etc. These parameters can be adjusted and set according to actual conditions.

[0120] Step S404, allocating the data index subsets to multiple processing threads, and performing simulated annealing calculations according to the preset simulated annealing parameters and the initial data index structure, and using the calculated optimal solution as the optimal data index structure;

[0121] In this embodiment, the data index subsets are assigned to multiple processing threads, and the simulated annealing algorithm is executed in parallel, wherein each thread performs iterative calculations based on the current initial data index structure and preset simulated annealing parameters. In each iteration, the algorithm attempts to make minor adjustments to the index structure (such as changing the order of the index, adjusting the depth or breadth of the index, etc.), and calculates the adjusted cost (or energy) function value. If the new index structure has a lower cost (i.e., better index performance), the new structure is accepted as the current solution; otherwise, it is decided whether to accept the poor solution based on a certain probability (usually related to the current temperature and cost difference) to avoid falling into a local optimal solution. As the number of iterations increases and the temperature gradually decreases, the algorithm gradually converges to a global or near-global optimal solution, i.e., the optimal data index structure.

[0122] Step S405, reconstructing the index according to the optimal data index structure to obtain valid data index information;

[0123] In this embodiment, index reconstruction refers to reconstructing the current data index information according to the optimal data index structure. After the index is reconstructed according to the optimal data index structure, the obtained effective data index structure will have higher retrieval efficiency and lower query cost.

[0124] Step S406: Update the currently recorded data index information according to the valid data index information to obtain the first real-time index information.

[0125] In this embodiment, the data index information currently recorded is the information data recorded in the index information update record, which can be updated by overwriting the valid data index information on the data index information currently recorded. The data index information currently recorded is the same as the data index information processed above.

[0126] This embodiment divides the data index information into data sets to obtain data index subsets; obtains the data index structure of the data index subset, and selects an initial data index structure from the data index structure; obtains preset simulated annealing parameters; distributes the data index subset to multiple processing threads, and performs simulated annealing calculations according to the preset simulated annealing parameters and the initial data index structure, and uses the calculated optimal solution as the optimal data index structure; reconstructs the index according to the optimal data index structure to obtain valid data index information; updates the currently recorded data index information according to the valid data index information, thereby effectively and quickly updating the index status according to the data index information to obtain the first real-time index information.

[0127] Continue to refer Figure 6 In some optional implementations of this embodiment, step S405 includes the following steps:

[0128] Step S4051, deleting the data index information;

[0129] In this embodiment, which data index information needs to be deleted is determined based on the optimal data index structure, and the deletion command can be executed through the command or tool provided by the database system. When the deletion operation is completed, it can be confirmed that the data index information has been correctly deleted, and it can be checked whether the database performance and data access are affected.

[0130] Step S4052, obtaining index structure information of the optimal data index structure, wherein the index structure information includes index key, index type, and index partition;

[0131] In this embodiment, the index structure information includes the index key (column or combination of columns used for indexing), index type (such as B-tree, hash, bitmap, etc.), and index partition (if applicable, distributing index data to different physical areas to improve performance).

[0132] Step S4053, creating an index according to the index key, the index type, and the index partition to obtain optimized data index information;

[0133] In this embodiment, the step of index creation includes: writing an SQL statement to create a new index based on the extracted index structure information, which can be implemented by a CREATE INDEX statement, wherein the name of the index, the index key, the index type, and any partitioning options are specified; executing the above SQL statement in a database management system to create a new data index. During the index creation process, the performance and resource usage of the database can be monitored to ensure that the operation does not have a negative impact on the business.

[0134] Step S4054, performing index verification on the optimized data index information, and using the optimized data index information that passes the verification as the valid data index information.

[0135] In this embodiment, the optimized data index information is verified by indicators to determine the validity of the data index information obtained after the index is created. When the optimized data index information meets all the verification indicators, the optimized data index information is output as a result as valid data index information.

[0136] This embodiment deletes the data index information; obtains the index structure information of the optimal data index structure, wherein the index structure information includes the index key, the index type, and the index partition; creates an index according to the index key, the index type, and the index partition to obtain the optimized data index information; performs index verification on the optimized data index information, and uses the verified optimized data index information as the valid data index information. Thus, the index is effectively rebuilt according to the index structure information of the data index information to obtain the valid data index information, which facilitates the subsequent update operation of the data index information.

[0137] Continue to refer Figure 7 In some optional implementations of this embodiment, step 50 includes the following steps:

[0138] Step 501, performing difference calculation according to the index expansion ratio and the preset expansion threshold to obtain an expansion difference;

[0139] In this embodiment, the preset expansion threshold is a value set according to system performance requirements and storage efficiency, and is used to determine whether the index needs to be adjusted. The preset expansion threshold can be set and adjusted according to actual conditions.

[0140] Step 502, obtaining a preset expansion level mapping table, and identifying a corresponding expansion level in the expansion level mapping table according to the expansion difference;

[0141] In this embodiment, the expansion level mapping table defines the expansion levels corresponding to different expansion difference ranges, and each level corresponds to a different update task. According to the calculated expansion difference, a matching search is performed in the expansion level mapping table to identify the expansion level corresponding to the expansion difference. In this embodiment, the expansion levels are set to level 1, level 2, and level 3. The expansion difference corresponding to the level 1 expansion level is 0-0.3, the expansion difference corresponding to the level 2 expansion level is 0.3-0.6, and the expansion difference corresponding to the level 3 expansion level is 0.6-0.9. The expansion differences corresponding to the above expansion levels can be set and adjusted according to actual conditions.

[0142] Step 503, obtaining an expansion level information identifier corresponding to the expansion level, and extracting the update task from a database according to the expansion level information identifier;

[0143] In this embodiment, the expansion level information identifier is a unique information identifier corresponding to the update task, and a matching query is performed in the database according to the expansion level information identifier, so as to effectively extract the update task.

[0144] Step 504: adjust the data index information according to the update task, and update the currently recorded data index information according to the adjusted data index information to obtain the second real-time index information.

[0145] In this embodiment, the update task includes a first update task, a second update task, and a third update task. The first update task is to perform basic index cleaning and compression operations, including removing invalid and expired index items, and simply compressing the index structure to optimize storage space and query efficiency. The second update task is to perform index optimization operations to eliminate index fragmentation and improve index query functions. The third update task is to adjust the index strategy (such as the granularity, quantity, type, etc. of the index) to optimize index performance. After the data index information is adjusted according to the corresponding update task, the adjusted data index information is used as the second real-time index information.

[0146] This embodiment obtains an expansion difference by performing a difference calculation based on the index expansion ratio and the preset expansion threshold; obtains a preset expansion level mapping table, and identifies the corresponding expansion level in the expansion level mapping table according to the expansion difference; obtains an expansion level information identifier corresponding to the expansion level, and extracts the update task in a database according to the expansion level information identifier; adjusts the data index information according to the update task, and updates the data index information of the current record according to the adjusted data index information, thereby effectively updating the index status according to the expansion level corresponding to the index expansion ratio to obtain the second real-time index information.

[0147] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0148] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0149] Further references Figure 8 , as a response to the above Figure 1 The present application provides an embodiment of an index updating device, and the device embodiment is similar to Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0150] like Figure 8 As shown, the index updating device 600 described in this embodiment includes: an information acquisition module 601, a ratio calculation module 602, a data judgment module 603, a first processing module 604, and a second processing module 605. Among them:

[0151] Information acquisition module 601, used to acquire data index information;

[0152] A ratio calculation module 602, configured to calculate an index expansion ratio according to the data index information;

[0153] The data judgment module 603 is used to judge whether the index expansion ratio is greater than or equal to a preset expansion threshold;

[0154] The first processing module 604 is configured to perform concurrent index reconstruction on the data index information according to a simulated annealing algorithm if the index expansion ratio is greater than or equal to the preset expansion threshold, and update the index state according to the reconstructed data index information to obtain first real-time index information;

[0155] The second processing module 605 is used to calculate the expansion level corresponding to the index expansion ratio if the index expansion ratio is less than the preset expansion threshold, and call the corresponding update task according to the expansion level to update the index status of the data index information to obtain the second real-time index information.

[0156] By adopting the above-mentioned device, this embodiment can obtain data index information; calculate the index expansion ratio according to the data index information; determine whether the index expansion ratio is greater than or equal to the preset expansion threshold; if the index expansion ratio is greater than or equal to the preset expansion threshold, perform concurrent index reconstruction on the data index information according to the simulated annealing algorithm, and update the index status according to the reconstructed data index information to obtain the first real-time index information; if the index expansion ratio is less than the preset expansion threshold, calculate the expansion level corresponding to the index expansion ratio, call the corresponding update task according to the expansion level to update the index status of the data index information, and obtain the second real-time index information. In this way, the data index can be updated quickly and effectively to ensure that the data index will not cause the system query performance to decline due to expansion.

[0157] To solve the above technical problems, the present application also provides a computer device. Fig. 9 , Fig. 9 This is a basic structural block diagram of the computer device in this embodiment.

[0158] The computer device 7 includes a memory 71, a processor 72, and a network interface 73 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 7 with components 71-73, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0159] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0160] The memory 71 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 71 can be an internal storage unit of the computer device 7, such as a hard disk or memory of the computer device 7. In other embodiments, the memory 71 can also be an external storage device of the computer device 7, such as a plug-in hard disk equipped on the computer device 7, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 71 can also include both the internal storage unit of the computer device 7 and its external storage device. In this embodiment, the memory 71 is generally used to store the operating system and various application software installed on the computer device 7, such as computer-readable instructions of the index update method, etc. In addition, the memory 71 can also be used to temporarily store various types of data that have been output or are to be output.

[0161] The processor 72 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 72 is generally used to control the overall operation of the computer device 7. In this embodiment, the processor 72 is used to run computer-readable instructions stored in the memory 71 or process data, such as computer-readable instructions for running the index update method.

[0162] The network interface 73 may include a wireless network interface or a wired network interface. The network interface 73 is generally used to establish a communication connection between the computer device 7 and other electronic devices.

[0163] By adopting the above-mentioned computer device, this embodiment can obtain data index information; calculate the index expansion ratio according to the data index information; determine whether the index expansion ratio is greater than or equal to the preset expansion threshold; if the index expansion ratio is greater than or equal to the preset expansion threshold, then perform concurrent index reconstruction on the data index information according to the simulated annealing algorithm, and update the index status according to the reconstructed data index information to obtain the first real-time index information; if the index expansion ratio is less than the preset expansion threshold, then calculate the expansion level corresponding to the index expansion ratio, call the corresponding update task according to the expansion level to update the index status of the data index information, and obtain the second real-time index information. In this way, the data index can be updated quickly and effectively to ensure that the data index will not cause the system query performance to decline due to expansion.

[0164] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the index updating method as described above.

[0165] This embodiment, by using the above-mentioned computer-readable storage medium, can obtain data index information; calculate the index expansion ratio according to the data index information; determine whether the index expansion ratio is greater than or equal to the preset expansion threshold; if the index expansion ratio is greater than or equal to the preset expansion threshold, then perform concurrent index reconstruction on the data index information according to the simulated annealing algorithm, and update the index status according to the reconstructed data index information to obtain the first real-time index information; if the index expansion ratio is less than the preset expansion threshold, then calculate the expansion level corresponding to the index expansion ratio, call the corresponding update task according to the expansion level to update the index status of the data index information, and obtain the second real-time index information. This enables fast and effective updating of the data index to ensure that the data index will not cause system query performance to decline due to expansion.

[0166] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0167] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. An index updating method, characterized in that: The steps include: Get data index information; Calculate the index expansion ratio according to the data index information; Determining whether the index expansion ratio is greater than or equal to a preset expansion threshold; If the index expansion ratio is greater than or equal to the preset expansion threshold, the data index information is concurrently reindexed according to a simulated annealing algorithm, and the index state is updated according to the reconstructed data index information to obtain first real-time index information; If the index expansion ratio is less than the preset expansion threshold, the expansion level corresponding to the index expansion ratio is calculated, and the corresponding update task is called according to the expansion level to update the index status of the data index information to obtain second real-time index information.

2. The index updating method according to claim 1, characterized in that: Before the step of obtaining the data index information, the following steps are also included: Obtain data from multiple sources; Performing data cleaning and format normalization processing on the multi-source data to obtain standard multi-source data; Extracting key elements from the standard multi-source data to obtain key information of the data; Building a data index library based on the key data information; Acquire historical query information, and filter the data index library according to the historical query information to obtain the data index information.

3. The index updating method according to claim 1, characterized in that: The step of obtaining data index information specifically includes: Get index information identifier; Extracting original data index information from a preset database according to the index information identifier; The original data index information is preprocessed to obtain the data index information.

4. The index updating method according to claim 1, characterized in that: The step of calculating the index expansion ratio according to the data index information specifically includes: Obtain the index data size and index data structure of data index information; Calculate the index data space according to the index data size and the index data structure; The preset index occupied space is obtained, and the index expansion ratio is calculated according to the index data space and the preset index occupied space.

5. The index updating method according to claim 1, characterized in that: The step of concurrently reindexing the data index information according to the simulated annealing algorithm, and updating the index state according to the reconstructed data index information to obtain the first real-time index information specifically includes: Dividing the data index information into data sets to obtain data index subsets; Acquire a data index structure of the data index subset, and select an initial data index structure from the data index structure; Get the preset simulated annealing parameters; Allocating the data index subsets to a plurality of processing threads, performing simulated annealing calculations according to the preset simulated annealing parameters and the initial data index structure, and using the calculated optimal solution as the optimal data index structure; Rebuilding the index according to the optimal data index structure to obtain valid data index information; The data index information of the current record is updated according to the valid data index information to obtain the first real-time index information.

6. The index updating method according to claim 5, characterized in that: The step of performing index reconstruction according to the optimal data index structure to obtain valid data index information specifically includes: Deleting the data index information; Acquire index structure information of the optimal data index structure, wherein the index structure information includes index key, index type, and index partition; Create an index according to the index key, the index type, and the index partition to obtain optimized data index information; The optimized data index information is subjected to index verification, and the optimized data index information that passes the verification is used as the valid data index information.

7. The index updating method according to claim 1, characterized in that: The step of calculating the expansion level corresponding to the index expansion ratio, and calling the corresponding update task according to the expansion level to update the index state of the data index information to obtain the second real-time index information specifically includes: Performing a difference calculation based on the index expansion ratio and the preset expansion threshold to obtain an expansion difference; Obtain a preset expansion level mapping table, and identify a corresponding expansion level in the expansion level mapping table according to the expansion difference; Acquire an expansion level information identifier corresponding to the expansion level, and extract the update task from a database according to the expansion level information identifier; The data index information is adjusted according to the update task, and the currently recorded data index information is updated according to the adjusted data index information to obtain the second real-time index information.

8. An index updating device, characterized in that: include: Information acquisition module, used to obtain data index information; A ratio calculation module, used for calculating the index expansion ratio according to the data index information; A data judgment module, used to judge whether the index expansion ratio is greater than or equal to a preset expansion threshold; A first processing module is used for concurrently re-indexing the data index information according to a simulated annealing algorithm if the index expansion ratio is greater than or equal to the preset expansion threshold, and updating the index state according to the reconstructed data index information to obtain first real-time index information; The second processing module is used to calculate the expansion level corresponding to the index expansion ratio if the index expansion ratio is less than the preset expansion threshold, and call the corresponding update task according to the expansion level to update the index status of the data index information to obtain the second real-time index information.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the index updating method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the index updating method according to any one of claims 1 to 7 are implemented.

Images