Data indexing method and system, electronic equipment and readable storage medium

Abstract

The invention discloses a data indexing method and system, electronic equipment and a readable storage medium, relates to the technical field of databases, is applied to a data management system connected with a business system, and the data management system comprises a database and a search engine. Creating a target index of which the version type is the latest version in the first index type of the search engine; storing the latest version in a database, and determining a data synchronization state of the target index in the database; when it is detected that the target index completes data synchronization and it is determined that the target index is not abnormal, the data synchronization state of the target index is adjusted to be a completed state; and if a reading instruction sent by the service system is received and the index requirement corresponding to the reading instruction is the first index type, sending the latest version of the first index type stored in the database to the service system. The technical problem of low reliability of data indexing is solved.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a data indexing method, system, electronic device and readable storage medium. Background Art

[0002] In the field of information technology, especially in the field of database management and search engine integration, with the advent of the big data era, efficient data retrieval and analysis functions have become crucial. In order to achieve efficient data retrieval and analysis, ES (Elasticsearch, search engine) is usually used as an independent data layer to synchronize changes (such as additions, modifications, etc.) in the database MySQL (MyStructured Query Language, structured query language) to ES in real time or regularly, so as to regularly update ES and ensure that the data in ES is consistent with the data in the database MySQL.

[0003] The update of ES may lead to the diversification of indexes used to search for the same data in ES, such as expired indexes or indexes that failed during synchronization. When the business system calls the index in ES to perform data indexing operations, the business system often cannot distinguish the latest or accurate index in ES. This means that the business system may mistakenly select an index that is expired or failed to synchronize, and then obtain incorrect data or encounter indexing failure when performing data indexing operations. This uncertainty reduces the reliability of data indexing.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a data indexing method, system, electronic device and readable storage medium, aiming to solve the technical problem of low reliability of data indexing.

[0006] To achieve the above object, the present application provides a data indexing method, which is applied to a data management system connected to a business system, the data management system including a database and a search engine, and the method includes:

[0007] In case of a data synchronization operation between the database and the search engine, creating a target index with a version type of the latest version in the first index type of the search engine;

[0008] storing the latest version in the database, and determining a data synchronization status of a target index in the database;

[0009] When it is detected that the target index has completed data synchronization and it is determined that there is no abnormality in the target index, adjusting the data synchronization state of the target index to a completed state;

[0010] If a read instruction is received from a business system, and when the index requirement corresponding to the read instruction is a first index type, the latest version of the first index type stored in the database is sent to the business system, wherein the business system is used to call the search engine and the latest version to perform data indexing operations.

[0011] In one embodiment, when it is detected that the target index has completed data synchronization and it is determined that the target index has no abnormality, the step of adjusting the data synchronization state of the target index to a completed state includes:

[0012] When it is detected that the target index has completed data synchronization, the synchronized data volume of the target index is obtained, and the historical index of the historical version of the latest version at the previous time step is obtained, and the historical data volume of the historical index is determined;

[0013] When the absolute difference between the synchronized data volume and the historical data volume is less than a preset fluctuation threshold, it is determined that there is no abnormality in the target index, and the data synchronization state of the target index is adjusted to a completed state.

[0014] In one embodiment, after the step of obtaining the historical index of the historical version of the latest version at the previous time step and determining the amount of historical data of the historical index, the following step is further included:

[0015] When the absolute difference between the synchronized data volume and the historical data volume is greater than or equal to a preset fluctuation threshold, it is determined that an abnormality exists in the target index, and the data synchronization state of the target index is adjusted to a failed state, and a synchronization failure prompt is output.

[0016] In one embodiment, after the step of adjusting the data synchronization state of the target index to a failed state and outputting a synchronization failure prompt, the method further includes:

[0017] The failed version is deleted from the database, and the failed index corresponding to the failed version is deleted from the search engine, wherein the failed version is an index version whose synchronization status is a failed status.

[0018] In one embodiment, the number of versions of each index type is obtained;

[0019] Determine any target index type in each of the index types, and when the number of target versions of the target index type is greater than a preset version threshold, use the difference between the number of target versions and the preset version threshold as the target deletion number;

[0020] Sorting the index versions belonging to the target index type in order of their synchronization time, and determining the index versions whose sorting order is less than or equal to the target deletion number as expired versions;

[0021] Each of the expired versions is deleted from the database, and the expired index corresponding to each of the expired versions is deleted from the search engine.

[0022] In one embodiment, the database includes a target offline version and / or a target switching version, and the method further includes at least one of the following:

[0023] If an index version offline indication sent by the business system is received, a target offline version is determined from the index version offline indication, and a data synchronization state corresponding to the target offline version is adjusted to a failed state;

[0024] If an index version switching indication sent by the business system is received, a target switching version is determined from the index version switching indication, and the synchronization time corresponding to the target switching version is modified to the current time.

[0025] To achieve the above object, the present application provides a data indexing method, which is applied to a business system connected to a data management system, wherein the data management system includes a database and a search engine, and the method includes:

[0026] Sending a read instruction to the data management system, and receiving a latest version of the first index type stored in the database and sent by the data management system based on the read instruction;

[0027] Upon receiving a data index indication triggered externally on an index interface visually displayed in the business system, and the data index indication includes the latest version, calling a search engine in the data management system to determine a target index of the latest version;

[0028] Performing a data indexing operation through the target index;

[0029] The database is also used to create a target index with the latest version type in the first index type of the search engine when performing data synchronization operations between the database and the search engine, and the target index belongs to the first index type; store the latest version in the database, and determine the data synchronization status of the target index in the database; and adjust the data synchronization status of the target index to a completed status when it is detected that the target index has completed data synchronization and it is determined that there is no abnormality in the target index.

[0030] In addition, to achieve the above-mentioned purpose, the present application also provides an electronic device, which includes: a memory, a processor, and a program of the data indexing method stored in the memory and executable on the processor. When the program of the data indexing method is executed by the processor, the steps of the data indexing method as described above can be implemented.

[0031] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which is stored a program for implementing the data indexing method, and when the program of the data indexing method is executed by a processor, the steps of the data indexing method as described above are implemented.

[0032] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned data indexing method when executed by a processor.

[0033] One or more technical solutions proposed in this application have at least the following technical effects: when performing data synchronization operations between the database and the search engine, this application creates a target index with the latest version type in the search engine, and the target index belongs to the first index type; the latest version is stored in the database, and the data synchronization status of the target index is determined in the database; when it is detected that the target index has completed data synchronization and it is determined that there is no abnormality in the target index, the data synchronization status of the target index is adjusted to the completion status, so that the latest version and data synchronization status of the first index type are saved in the database while the data is synchronized, so that the subsequent business system can obtain the latest version from the database. Specifically, when a read instruction sent by the business system is received, and the index requirement corresponding to the read instruction is the first index type, the latest version of the first index type stored in the database will be sent to the business system, so that the business system can call the search engine and the latest version to perform data indexing operations.

[0034] Since the data synchronization status corresponding to the latest version is completed, the business system also obtains the latest version with the data synchronization status completed. Therefore, when the business system calls the search engine and the latest version to perform data indexing operations, it can search for the latest and accurate data, thereby improving the reliability of data search. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the description, are used to explain the principles of the present application.

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0037] Figure 1 This is a flow chart of an embodiment of a data indexing method of the present application;

[0038] Figure 2 This is a structural diagram of a data management system and a business system in an embodiment of a data indexing method of this application;

[0039] Figure 3 A flowchart of another embodiment of the data indexing method of the present application;

[0040] Figure 4 A schematic diagram showing the display of the index interface in the data indexing method of this application;

[0041] Figure 5 This is a schematic diagram of the system structure of the data indexing system according to an embodiment of the present application;

[0042] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the data indexing method in the embodiment of the present application.

[0043] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0045] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0046] In the current field of database management and search engine integration, Elasticsearch (ES) is often used to provide efficient data retrieval and analysis functions, while MySQL, as a relational database, is used to store structured data.

[0047] At present, the integration of database management and search engines is usually as follows: ES is used as an independent data layer, and data is interacted with MySQL through data synchronization tools (such as Logstash, JDBC River, etc.). At the hardware level, each server runs a MySQL instance and / or ES node, and communicates through the network. At the software level, data synchronization is usually driven by scheduled tasks or events, and changes in MySQL are synchronized to ES to update ES.

[0048] Frequent updates to ES may lead to the diversification of indexes used to search for the same data in ES, such as expired indexes or indexes that failed during synchronization. When the business system calls the index in ES to perform data indexing operations, the business system often cannot distinguish the latest or accurate index in ES. This means that the business system may mistakenly select an index that is expired or failed to synchronize, and then obtain incorrect data or encounter indexing failure when performing data indexing operations. This uncertainty reduces the reliability of data indexing.

[0049] The main solution of the embodiment of the present application is: when performing data synchronization, the data management system will create a target index with the latest version type in the search engine, and will store the latest version in the database, so that different versions of indexes can also be distinguished in the database. And when the target index is performing data synchronization, the data management system will record the synchronization status of the target index in the database, and then the data synchronization status of the target index can be determined in the database. Thus, the business system can read the latest version from the database, so that when the business system is performing data indexing, it can call the search engine to determine the latest version of the target index, so as to perform data indexing operations through the target index.

[0050] Based on this, the embodiment of the present application provides a data indexing method, which is applied to a data management system connected to a business system, wherein the data management system includes a database and a search engine, Figure 1 , Figure 1 This is a flow chart of an embodiment of the data indexing method of the present application.

[0051] In this embodiment, the data indexing method includes steps S10 to S40:

[0052] Step S10, when a data synchronization operation is performed between the database and the search engine, a target index whose version type is the latest version is created in the first index type of the search engine;

[0053] It should be noted that the present implementation is applied to a data management system, which includes a search engine and a database, and the data management system can communicate with the business system. The database is used to store data, and the database can be Mysql. The search engine is used to process fast search and analysis of data, and the search engine can be ES. The search engine can include multiple indexes to search for data through the index. The business system can call the search engine, and the business system is user-oriented. The user can trigger a data index indication on the business system, and the business system can call the search engine to index data.

[0054] The data in the database can be synchronized to the search engine. When the data in the database changes, the changed data can be synchronized to the search engine. When the data in the database changes, is added, etc., and needs to be synchronized to the search engine, the search engine will create a new index to synchronize the data to be synchronized to the new index to achieve data synchronization.

[0055] The target index refers to the index currently created in the search engine, and the latest version refers to the index version of the target index. The latest version can be used to indicate the synchronization time of the target index. Since different types of data may be stored in the database, they need to be searched through different indexes. For the same type of data, there may be multiple changes in the database, and each change requires the changed data of that type to be synchronized to the search engine. Therefore, there may be multiple indexes for the same type of data in the search engine. Therefore, there may be multiple indexes in the first index type, and the different indexes in the first index type can be distinguished by the index version of each index. The version type refers to the type of index version of each index in the first index type. The version type can be classified based on the synchronization time corresponding to the index. For example, in this embodiment, the latest version refers to the latest index version of the target index in the first index type. The synchronization time of the latest version is later than the synchronization time of other indexes in the first index type. Other indexes refer to the indexes corresponding to all versions of the first index type except the latest version.

[0056] Exemplarily, when the search engine receives a data synchronization request, a data synchronization operation is performed between the database and the search engine, a first index type is determined from the data synchronization request, and a target index with a version type of the latest version is created in the first index type of the search engine. The latest version can be determined based on the first index type and the time when the target index is created, and the time when the target index is created can be used as the synchronization time. The first index type can be any index type in the database.

[0057] Step S20, storing the latest version in the database, and determining the data synchronization status of the target index in the database;

[0058] The data synchronization status can reflect the data synchronization status of the target index. Exemplarily, when the search engine creates a target index with the latest version type, the database stores the latest version in the database and records the data synchronization status of the target index in the database. When the target index is in the process of data synchronization, the data synchronization status is synchronizing. For example, a separate area can be allocated in the database to store the latest version and record the data synchronization status of the target index. The index versions corresponding to multiple indexes of the same index type can be stored in the database, and the index versions corresponding to multiple index types can be stored in one piece. The database can store index versions corresponding to multiple index types.

[0059] The database may perform data synchronization on the target index when the latest version is stored in the database and a data synchronization state for recording the data synchronization status of the target index is determined in the database.

[0060] In a feasible embodiment, during the data synchronization of the target index, there may be data synchronization anomalies. In the case of synchronization anomalies, the data synchronization state of the target index is adjusted to a failed state, and a synchronization failure prompt is output to prompt the relevant maintenance personnel that the data synchronization of the target index has failed. For example, the database is locked during the synchronization process, or the corresponding data to be synchronized is changed during the synchronization process. This embodiment only gives examples of abnormal situations that may occur during the data synchronization process, but it is not limited to the above-mentioned abnormal situations. The data synchronization state of the target index is changed to a failed state. When the data synchronization state is changed to a failed state, it means that the data synchronization of the target index has failed.

[0061] Step S30: When it is detected that the target index has completed data synchronization and it is determined that the target index has no abnormality, the data synchronization state of the target index is adjusted to a completed state.

[0062] It should be noted that when the target index completes data synchronization, the database can adjust the data synchronization status of the target index to the completed status, which means that the data synchronization of the target index is successful. The business system can read the latest version of the database with the completed synchronization status, so that the target index corresponding to the target version can be called in the search engine to perform data indexing operations. The target index has no abnormalities, which means that there are no abnormalities in the target index during and after the data synchronization is completed.

[0063] Step S40, if a read instruction sent by the business system is received, and when the index requirement corresponding to the read instruction is the first index type, the latest version of the first index type stored in the database is sent to the business system, wherein the business system is used to call the search engine and the latest version to perform data indexing operations.

[0064] It should be noted that the business system can read the database in the data management system, and the read instruction is used to instruct the data management system to send the latest version of the first index type stored in the database to the business system. The data management system will send the latest version of the first index type to the business system. The business system can call the search engine and the latest version to perform data indexing operations, so as to search for the most accurate and latest data.

[0065] Exemplarily, when it is detected that the target index has completed data synchronization and it is determined that there is no abnormality in the target index, the data synchronization status of the target index is adjusted to the completed status. If a read instruction sent by the business system is received, and when the index requirement corresponding to the read instruction is the first index type, the latest version of the first index type stored in the database is sent to the business system. Among them, the latest version of the first index type may change. When a data synchronization operation is performed in the first index type in the search engine, a new latest version of the index will be created in the first index type. The business system can call the search engine and the latest version to perform data indexing operations.

[0066] In the embodiment of the present application, when data synchronization is performed between the database and the search engine, a target index with the latest version type is created in the search engine, and the target index belongs to the first index type; the latest version is stored in the database, and the data synchronization status of the target index is determined in the database; when it is detected that the target index has completed data synchronization and it is determined that there is no abnormality in the target index, the data synchronization status of the target index is adjusted to the completed state, so that the latest version and data synchronization status of the first index type are saved in the database while the data is synchronized, so that the subsequent business system can obtain the latest version from the database. Specifically, when a read instruction sent by the business system is received, and the index requirement corresponding to the read instruction is the first index type, the latest version of the first index type stored in the database will be sent to the business system, so that the business system can call the search engine and the latest version to perform data indexing operations.

[0067] Since the data synchronization status corresponding to the latest version is completed, the business system also obtains the latest version with the data synchronization status completed. Therefore, when the business system calls the search engine and the latest version to perform data indexing operations, it can search for the latest and accurate data, thereby improving the reliability of data search.

[0068] To better understand data management systems and business systems, please refer to Figure 2 , describes the connection between the data management system and the business system. Figure 2 It includes a data management system 10 and a business system 20. The data management system 10 includes a database Mysql and a search engine ElasticSesrch. When the data management system 10 starts data synchronization, it will create the latest version of the target index in the search engine, store the latest version corresponding to the target index in the database, and monitor the amount of synchronized data of the target index to monitor whether the target index is synchronized abnormally. The business system can call the search engine of the data management system to search for data. The local cache module LoadingCache of the business system can obtain the latest version from the database. The business system can also obtain the latest version from the local cache module to call the search engine to search for data through the target index corresponding to the latest version. The local cache module of the business system will obtain the latest version with a data synchronization status of completed. When the data management system is performing data synchronization, the local cache module cannot obtain the latest version of the data being synchronized. Therefore, in Figure 2 In the process, the order of each process can be to start data synchronization, synchronize the target index in the search engine, and store the latest version corresponding to the target index. When the data synchronization status of the target index is completed, the local module of the business system can obtain the latest version to save it in the business system, so that when the business system searches for data, it can use the latest version to search for data in the search engine. During data synchronization, the process of monitoring the amount of synchronized data can be run all the time.

[0069] In other embodiments, when synchronizing data of the target index, if the local cache module of the business system needs to read a version from the database, the version with the synchronization time closest to the current time in the database can be read, that is, the version of the previous time step that can be considered as the latest version. The business system can also search for data based on the version of the previous time step, so as to achieve data search and data synchronization at the same time.

[0070] In a feasible embodiment, step S20 further includes steps S21 to S22:

[0071] Step S21, when the target index completes data synchronization, obtain the synchronized data volume of the target index, obtain the historical index of the historical version of the latest version at the previous time step, and determine the historical data volume of the historical index;

[0072] It should be noted that when the target index completes data synchronization, the amount of synchronized data corresponding to the target index can be detected to avoid synchronization anomalies caused by repeated data synchronization. The amount of synchronized data refers to the amount of data in the database synchronized to the target index, and the amount of historical data refers to the amount of data corresponding to the historical index. The target index has a corresponding index type, and there can be multiple index versions in the index type. The historical index and the target index belong to the same index type. The synchronization time of the latest version is later than the historical version, and the synchronization time of the historical index is earlier than the target index. The historical index can also be considered as the index of the previous time step of the target index in the index type.

[0073] Step S22: When the absolute difference between the synchronized data volume and the historical data volume is less than a preset fluctuation threshold, it is determined that the target index has no abnormality, and the data synchronization state of the target index is adjusted to a completed state.

[0074] It should be noted that the preset fluctuation threshold can be determined based on the actual situation, and the difference is the absolute value of the difference between the synchronized data volume and the historical data volume. When the difference is less than the preset fluctuation threshold, it means that the size of the synchronized data volume is similar to the size of the historical data volume, and there is no abnormal increase or decrease, so the synchronization status of the target index can be adjusted to the completed status.

[0075] Exemplarily, when the target index completes data synchronization, the amount of synchronized data corresponding to the latest version is obtained in the database to obtain the amount of synchronized data of the target index, and the amount of historical data corresponding to the historical version of the last time step of the latest version can also be obtained in the database to obtain the amount of historical data of the historical index. The absolute value of the difference between the amount of synchronized data and the amount of historical data is determined to obtain the absolute difference, and when the absolute difference is less than a preset fluctuation threshold, the synchronization state of the target index is adjusted to the completion state.

[0076] This embodiment determines whether the data volume changes abnormally after the synchronization is completed, thereby more accurately determining the index with successful synchronization, avoiding determining the data synchronization status of the index with abnormal synchronization as the completed status, and thus avoiding data indexing failure.

[0077] In a feasible embodiment, after step S21, step S211 is further included:

[0078] Step S211, when the absolute difference between the synchronized data volume and the historical data volume is greater than or equal to the preset fluctuation threshold, it is determined that there is an abnormality in the target index, and the data synchronization state of the target index is adjusted to a failed state, and a synchronization failure prompt is output.

[0079] It should be noted that when the absolute difference is greater than or equal to the preset fluctuation threshold, it means that the amount of synchronized data has increased or decreased abnormally compared with the historical data amount, so there may be synchronization anomalies in the target index. Therefore, it is necessary to adjust the synchronization status of the target index to a failed state and output a synchronization failure prompt.

[0080] Exemplarily, when the absolute difference between the amount of synchronized data and the amount of historical data is greater than or equal to a preset fluctuation threshold, it indicates that there is an abnormality in the target index, and the synchronization state of the target index is adjusted to a failed state. A synchronization failure prompt is output to the alarm group to prompt relevant maintenance personnel that the target index synchronization has failed, so as to prompt relevant maintenance personnel to perform repairs. The alarm group is an online communication channel for relevant maintenance personnel, and relevant maintenance personnel can obtain synchronization failure prompts from the alarm group.

[0081] This embodiment adjusts the synchronization state to a failed state when the data volume is abnormal, thereby avoiding synchronization completion, but adjusts the state of the index with synchronization abnormality to a completed state, thereby improving the accuracy of determining whether the target index is synchronized successfully.

[0082] In a feasible embodiment, the data indexing method further includes step S2111: deleting the failed version from the database, and deleting the failed index corresponding to the failed version in the search engine, wherein the failed version is an index version whose synchronization status is a failed status.

[0083] A failed version refers to an index version whose synchronization status is a failed status, and a failed index is the index corresponding to the failed version. The data management system can periodically delete failed versions from the database, and can also periodically delete failed indexes corresponding to failed versions from the search engine. This can avoid calling failed indexes for data indexing operations, which may lead to data indexing anomalies.

[0084] Further, see Figure 3 In a feasible embodiment, the method further includes steps A10 to A40:

[0085] Step A10, obtaining the number of versions of each index type;

[0086] Step A20, determining any target index type in each index type, and when the target version number of the target index type is greater than a preset version threshold, taking the difference between the target version number and the preset version threshold as the target deletion number;

[0087] It should be noted that the number of versions of each index type can be obtained from the database of the data management system or from the search engine of the data management system. The number of versions refers to the number of different versions of the same index type. The target index type can be any index type.

[0088] The preset version threshold refers to the number of versions that can be stored in the target index type configured in the business system. The preset version thresholds of different target index types can be the same or different. The target version number is the number of versions of the target index type, and the target deletion number refers to the number of expired versions that need to be deleted, or the number of expired indexes that need to be deleted.

[0089] Exemplarily, the number of versions of each index type may be obtained periodically to delete expired versions and expired indexes periodically.

[0090] Step A30, sorting the index versions belonging to the target index type in order of their synchronization time, and determining the index versions whose sorting order is less than or equal to the target deletion number as expired versions;

[0091] Step A40, deleting each expired version from the database, and deleting the expired index corresponding to each expired version in the search engine.

[0092] It should be noted that the synchronization time refers to the synchronization time of the index version. The synchronization time of different index versions in the same index type is different, and there is a time sequence. In this embodiment, the index versions in the same index type can be sorted in sequence according to the sequence of their respective synchronization times, that is, the order of the index version with an earlier synchronization time is less than the order of the index version with a later synchronization time.

[0093] In order to avoid storing too many index versions in the data management system, which would cause storage pressure, when the number of target versions is greater than the preset version threshold, the index versions with earlier synchronization time can be deleted first, and the index versions with later synchronization time can be retained. It can be understood that the old index versions are deleted and the new index versions are saved. Exemplarily, the index versions with a sorting order less than or equal to the target deletion number can be sorted to delete the old index versions and retain the new index versions. The expired version can be considered as the old index version, and the expired index is the index corresponding to the expired version.

[0094] Exemplarily, the number of versions of each index type in the data management system can be periodically obtained. For each target index type, when the number of target versions of the target index type is greater than a preset version threshold, the difference between the target version number and the preset version threshold is used as the target deletion number, and expired versions can be determined in the target index type. For example, the index versions belonging to the target index type can be sorted in order of their respective synchronization times, and each index version whose sorting order is less than or equal to the target deletion number is determined as an expired version. The data management system deletes each expired version from the database, and the data management system also has the expired index corresponding to each expired version in the search engine. In addition, after determining the expired version, the data management system can also send a deletion instruction to the business system so that the business system can delete the expired version based on the deletion instruction.

[0095] This embodiment can reduce unnecessary storage, release the storage space of the database, and relieve the storage pressure of the search engine by deleting the expired versions and expired indexes.

[0096] In a feasible embodiment, the database includes the target offline version and / or the target switching version, and the data indexing method further includes steps B10 to B20:

[0097] Step C10: if an index version offline indication sent by the business system is received, a target offline version is determined from the index version offline indication, and a data synchronization state corresponding to the target offline version is adjusted to a failed state;

[0098] It should be noted that the index version offline indication can be triggered by the user on the data index interface visually displayed on the business system. For example, after the user triggers the index version offline indication on the business system, the business system can send the index version offline indication to the data management system. After receiving the index version offline indication, the data management system can determine the target offline version from the index version offline indication.

[0099] The target offline version refers to the index version whose data synchronization status needs to be adjusted to a failed state. Since no data synchronization anomaly may be found in the index during or after data synchronization, but there may be a data search failure when calling the index for data indexing operations, the user can indicate the offline index version to the data management system to adjust the data synchronization status of the target offline version specified by the index version offline indication to a failed state. The data management system can then delete the target offline version from the database, and delete the index corresponding to the target offline version from the search engine, thereby avoiding subsequent calls to the index corresponding to the target offline version for data indexing operations to avoid data indexing failures.

[0100] Exemplarily, an index version offline indication sent by a business system is received, a target offline version is determined from the index version indication, and a data synchronization state corresponding to the target offline version is adjusted to a failed state.

[0101] Step B20: If an index version switching indication sent by the business system is received, a target switching version is determined from the index version switching indication, and the synchronization time corresponding to the target switching version is modified to the current time.

[0102] It should be noted that the index version switching indication may be triggered by a user on a data index interface visually displayed on a business system.

[0103] Exemplarily, after a user triggers an index version switching indication on a business system, the business system may send an index version switching indication to a data management system. After receiving the index version switching indication, the data management system may determine a target switching version from the index version switching indication.

[0104] Since the number of indexes called simultaneously in the same index type is 1 when performing data indexing, in this embodiment, the latest index version in the index type will be called, that is, the index version with the synchronization time closest to the current time. In this embodiment, the data indexing operation is performed by the business system calling the index in the search engine in the data management system. Before performing the data indexing operation, the business system will first read the latest index version in the index type corresponding to the data indexing operation from the data management system. Therefore, when the user wants to call the historical index version in the index type to perform the data indexing operation, the historical index version can be switched to the latest version to perform the data indexing operation with the changed historical index version.

[0105] The target switching version may refer to the historical index version corresponding to the latest index version in the same index type. When the data management system modifies the synchronization time corresponding to the target switching version to the current time in the database, it indicates that the target switching version is the latest index version in the corresponding index type, so the business system can read the target switching version to perform operations such as data indexing with the target switching version.

[0106] Exemplarily, an index version switching indication is received, and a target switching version is determined from the index version switching indication; and the synchronization time corresponding to the target switching version is modified to the current time. This implementation can flexibly select an index version for data indexing operations by receiving an index version switching indication, thereby improving the flexibility of data indexing while ensuring the accuracy of data indexing.

[0107] Furthermore, in another embodiment, the data indexing method can also be applied to a business system connected to a data management system, wherein the data management system includes a database and a search engine, and the method includes:

[0108] Step E10, sending a read instruction to the data management system, and receiving a latest version of the first index type stored in the database sent by the data management system based on the read instruction;

[0109] It should be noted that the business system can communicate with the data management system, and the business system can call the search engine in the data management system to perform data indexing operations. The business system can periodically read the latest version from the database, and the business system can synchronously read the latest version of multiple index types. The business system reads the latest version of the index type with the synchronization time closest to the current time in the database, and the corresponding data synchronization status is the completed status.

[0110] The latest version is the latest index version in the corresponding index type, and is also the index version with the synchronization time closest to the current time in the corresponding index type. For any index type, the business system can periodically obtain the latest version from the database.

[0111] Since multiple data synchronizations may be performed for the same index type in the data management system, the latest versions of the same index type at different times may be different in the data management system, so the business system can periodically read the latest version from the database to read the latest index version. There is no need to manually switch the index version on the business system, and the business system can automatically read the target version to search for data through the target version.

[0112] Step E20, when receiving a data index indication triggered externally on the index interface visually displayed by the business system, and the data index indication includes the latest version, calling a search engine in the data management system to determine a target index of the latest version;

[0113] Step E30, performing a data indexing operation through the target index;

[0114] The database is also used to create a target index with the latest version type in the first index type of the search engine when performing data synchronization operations between the database and the search engine, and the target index belongs to the first index type; store the latest version in the database, and determine the data synchronization status of the target index in the database; and adjust the data synchronization status of the target index to a completed status when it is detected that the target index has completed data synchronization and it is determined that there is no abnormality in the target index.

[0115] It should be noted that when the business system reads data in the database, it will first determine which index type in the database to read. When it is determined to be the first index type, it will read the latest version from the first index type and save it in the local cache of the business system.

[0116] It should be noted that a data indexing interface can be displayed on the business system, and the data indexing interface can be used to display the index version corresponding to the index type. The data indexing indication is used to instruct the business system to perform a data indexing operation, and the latest version can be determined from the data indexing indication. The latest version is the target index corresponding to the latest version indicated by the business system for performing a data indexing operation. When receiving a data indexing indication for instructing to perform a data indexing operation using a target index corresponding to the latest version, the business system can call a search engine, determine the target index corresponding to the latest version through the search engine, and then call the target index through the search engine to perform the data indexing operation.

[0117] Exemplarily, when the business system receives a data index indication triggered by a user on a visually displayed index interface, and the data index indication includes the latest version, the business system calls a search engine to determine a target index corresponding to the latest version, and then calls the corresponding target index through the search engine to perform data indexing operations.

[0118] In other embodiments, when a user triggers a data index indication on a business system, the user will select an index type on the index interface of the business system to determine the version for data indexing corresponding to the index type to generate a data index indication. When the user selects an index type, the business system will synchronously determine the version for data indexing under the index type, specifically, the version with the synchronization time closest to the current time under the index type. It is understandable that the business system can determine the version that needs to be indexed and the index type corresponding to the version from the data index indication. When the business system performs a data search, the corresponding index type may be a hotel index type, a product index type, etc. This embodiment does not specifically limit this.

[0119] For a better understanding of this implementation, please refer to Figure 4 , an example is given to illustrate the content displayed on the index interface in this embodiment. Figure 4 This is a schematic diagram of the index interface. Figure 4 The index interface shown in is an example. Other contents can also be displayed in the index interface. Among them, index management means that the index interface is currently in index management mode, and the currently selected index type is hotel. Figure 4Two index versions are displayed in the table. Each index version can include index ID, index type, index name, editing time, and index status. Each index version can support switching and offline operations. However, the index version in the enabled state cannot perform switching and offline operations, while the index version in the disabled state can perform switching and offline operations. The index name is the name of the index version, and the index type refers to the type of the index version. Figure 4 The index type of the two index versions is hotel. Figure 4 The editing time shown in refers to the synchronization time of the index version. For example, the index version with the index name Hotel_2400 has an index type of Hotel, an editing time of 00:22:13 on July 3, 2024, and an index status of Enabled. The corresponding operations include switching and offline operations, but since the index status is Enabled, the index version will not be switched or indexed for the time being. The index version with the index name Hotel_2410 has an index type of Hotel, an editing time of 00:21:13 on July 2, 2024, and an index status of Not Enabled. The corresponding operations include switching and offline operations, and the index status is Not Enabled, so the index version can be switched or offline. The user can click Switch to trigger the index switching operation, and can click Offline to trigger the index offline indication.

[0120] This embodiment provides an index interface that is easy for users to operate, allowing users to intuitively view and manage indexes, including index versions, and switching and offline operations corresponding to index versions, thereby improving user experience and operation and maintenance efficiency.

[0121] The present application also provides a data indexing system. Figure 5 The data indexing system includes a business system and a data management system. The data management system includes a database and a search engine. The data indexing system includes:

[0122] The data management system 10 is used to create a target index with the latest version type in the first index type of the search engine when a data synchronization operation is performed between the database and the search engine; store the latest version in the database, and determine the data synchronization status of the target index in the database; when it is detected that the target index has completed data synchronization and it is determined that there is no abnormality in the target index, adjust the data synchronization status of the target index to a completed state; if a read instruction sent by a business system is received, and when the index requirement corresponding to the read instruction is a first index type, send the latest version of the first index type stored in the database to the business system;

[0123] The business system 20 is used to send a read instruction to the data management system, receive the latest version of the first index type stored in the database sent by the data management system based on the read instruction; upon receiving a data index indication triggered externally on the index interface visually displayed by the business system, and the data index indication contains the latest version, call the search engine in the data management system to determine the target index of the latest version; and perform data indexing operations through the target index.

[0124] An embodiment of the present application provides an electronic device, which may be a playback device, and includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data indexing method in the above-mentioned embodiment.

[0125] Reference below Figure 6 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure. The electronic device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0126] like Figure 6As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM) 1004. In RAM 1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM 1002, and RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device may allow the electronic device to communicate wirelessly or wired with other devices to exchange data. Although the electronic device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have instead.

[0127] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1009, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0128] The electronic device provided by the present application adopts the data indexing method in the above embodiment 1 to solve the technical problem of low reliability of data indexing. Compared with the prior art, the beneficial effect of product flow data distribution provided by the embodiment of the present application is the same as the beneficial effect of the data indexing method provided by the above embodiment, and the other technical features in the electronic device are the same as the features disclosed by the above embodiment method, which will not be repeated here. It should be understood that the various parts of the present disclosure can be implemented by hardware, software, firmware or a combination thereof. In the description of the above implementation, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner. The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims. The present embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the data indexing method in the above embodiment 1. The computer-readable storage medium provided in the embodiment of the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, equipment or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable EPROM (ElectricalProgrammable Read Only Memory, read-only memory) or flash memory, an optical fiber, a portable compact disk CD-ROM (compact disc read-only memory, read-only memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution device, device or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency, radio frequency), etc., or any suitable combination of the above. The above-mentioned computer-readable storage medium may be contained in an electronic device; it may also exist alone without being assembled into an electronic device.The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by an electronic device, the electronic device: when performing data synchronization operations between the database and the search engine, creates a target index with a version type of the latest version in the first index type of the search engine; stores the latest version in the database, and determines the data synchronization status of the target index in the database; when it is detected that the target index has completed data synchronization and it is determined that there is no abnormality in the target index, adjusts the data synchronization status of the target index to a completed status; if a read instruction sent by a business system is received, and when the index requirement corresponding to the read instruction is a first index type, the latest version of the first index type stored in the database is sent to the business system, wherein the business system is used to call the search engine and the latest version to perform data indexing operations.

[0129] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a LAN (local area network) or WAN (Wide Area Network), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based device that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0131] The modules involved in the embodiments described in the present disclosure may be implemented by software or by hardware. The name of the module does not constitute a limitation on the unit itself under certain circumstances. The computer-readable storage medium provided in the present application stores computer-readable program instructions for executing the above-mentioned data indexing method, which is intended to solve the technical problem of low reliability of data indexing. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the embodiments of the present application are the same as the beneficial effects of the data indexing method provided in the above-mentioned embodiments, which will not be repeated here.

[0132] The present application also provides a computer program product, including a computer program, which implements the steps of the data indexing method as described above when executed by a processor. The computer program product provided by the present application is intended to solve the technical problem of low reliability of data indexing. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as the beneficial effects of the data indexing method provided by the above embodiment, which will not be repeated here. The above is only a preferred embodiment of the present application, and the patent scope of the present application is not limited thereto. All equivalent structures or equivalent process transformations made using the contents of the specification and drawings of this application, or directly or indirectly used in other related technical fields, are similarly included in the patent processing scope of the present application.

Claims

1. A data indexing method, It is characterized in that A data management system is applied to a data management system connected to a business system, the data management system includes a database and a search engine, and the method includes: In case of a data synchronization operation between the database and the search engine, creating a target index with a version type of the latest version in the first index type of the search engine; storing the latest version in the database, and determining a data synchronization status of a target index in the database; When it is detected that the target index has completed data synchronization and it is determined that there is no abnormality in the target index, adjusting the data synchronization state of the target index to a completed state; If a read instruction is received from a business system, and when the index requirement corresponding to the read instruction is a first index type, the latest version of the first index type stored in the database is sent to the business system, wherein the business system is used to call the search engine and the latest version to perform data indexing operations.

2. The method according to claim 1, It is characterized in that The step of adjusting the data synchronization state of the target index to a completed state when it is detected that the target index has completed data synchronization and it is determined that the target index has no abnormality includes: When it is detected that the target index has completed data synchronization, the synchronized data volume of the target index is obtained, and the historical index of the historical version of the latest version at the previous time step is obtained, and the historical data volume of the historical index is determined; When the absolute difference between the synchronized data volume and the historical data volume is less than a preset fluctuation threshold, it is determined that there is no abnormality in the target index, and the data synchronization state of the target index is adjusted to a completed state.

3. The method according to claim 2, It is characterized in that After the step of obtaining the historical index of the historical version of the latest version at the previous time step and determining the historical data volume of the historical index, the method further includes: When the absolute difference between the synchronized data volume and the historical data volume is greater than or equal to a preset fluctuation threshold, it is determined that an abnormality exists in the target index, and the data synchronization state of the target index is adjusted to a failed state, and a synchronization failure prompt is output.

4. The method according to claim 3, It is characterized in that After the step of adjusting the data synchronization state of the target index to a failed state and outputting a synchronization failure prompt, the method further includes: The failed version is deleted from the database, and the failed index corresponding to the failed version is deleted from the search engine, wherein the failed version is an index version whose synchronization status is a failed status.

5. The method according to claim 1, It is characterized in that The method includes: Get the number of versions of each index type; Determine any target index type in each of the index types, and when the number of target versions of the target index type is greater than a preset version threshold, use the difference between the number of target versions and the preset version threshold as the target deletion number; Sorting the index versions belonging to the target index type in order of their synchronization time, and determining the index versions whose sorting order is less than or equal to the target deletion number as expired versions; Each of the expired versions is deleted from the database, and the expired index corresponding to each of the expired versions is deleted from the search engine.

6. The method according to claim 1, It is characterized in that The database includes a target offline version and / or a target switching version, and the method further includes at least one of the following: If an index version offline indication sent by the business system is received, a target offline version is determined from the index version offline indication, and a data synchronization state corresponding to the target offline version is adjusted to a failed state; If an index version switching indication sent by the business system is received, a target switching version is determined from the index version switching indication, and the synchronization time corresponding to the target switching version is modified to the current time.

7. A data indexing method, It is characterized in that Applied to a business system connected to a data management system, the data management system including a database and a search engine, the method comprising: Sending a read instruction to the data management system, and receiving a latest version of the first index type stored in the database and sent by the data management system based on the read instruction; Upon receiving a data index indication triggered externally on an index interface visually displayed in the business system, and the data index indication includes the latest version, calling a search engine in the data management system to determine a target index of the latest version; Performing a data indexing operation through the target index; The database is also used to create a target index with the latest version type in the first index type of the search engine when performing data synchronization operations between the database and the search engine, and the target index belongs to the first index type; store the latest version in the database, and determine the data synchronization status of the target index in the database; and adjust the data synchronization status of the target index to a completed status when it is detected that the target index has completed data synchronization and it is determined that there is no abnormality in the target index.

8. A data indexing system, It is characterized in that The data indexing system includes a data management system and a business system, and the data management system also includes a search engine and a database; A data management system, configured to create a target index with a latest version in a first index type of the search engine when a data synchronization operation is performed between the database and the search engine; store the latest version in the database, and determine a data synchronization status of the target index in the database; When it is detected that the target index has completed data synchronization and it is determined that there is no abnormality in the target index, the data synchronization state of the target index is adjusted to the completion state; if a read instruction sent by the business system is received, and when the index requirement corresponding to the read instruction is a first index type, the latest version of the first index type stored in the database is sent to the business system; A business system is used to send a read instruction to the data management system, receive the latest version of the first index type stored in the database sent by the data management system based on the read instruction; upon receiving a data index indication triggered externally on the index interface visually displayed by the business system, and the data index indication contains the latest version, call the search engine in the data management system to determine the target index of the latest version; and perform data indexing operations through the target index.

9. An electronic device, It is characterized in that The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the data indexing method described in any one of claims 1 to 7.

10. A readable storage medium, It is characterized in that The readable storage medium is a computer-readable storage medium on which a program for implementing the data indexing method is stored. The program for implementing the data indexing method is executed by a processor to implement the steps of the data indexing method as described in any one of claims 1 to 7.

Images