Data indexing method, system, electronic device and readable storage medium

Abstract

The application discloses a data indexing method and system, electronic equipment and readable storage medium, relates to the technical field of databases, is applied to a data management system connected with a business system, the data management system comprises a database and a search engine, and comprises the following steps: in the case that a data synchronization operation is performed between the database and the search engine, a target index with a version type of a latest version is created in a first index type of the search engine; the latest version is stored in the database, and a data synchronization state of the target index is determined in the database; in the case that it is detected that the target index completes data synchronization and it is determined that the target index is normal, the data synchronization state of the target index is adjusted to a completion state; if a read instruction sent by the business system is received, and the index demand corresponding to the read instruction is the first index type, the latest version of the first index type stored in the database is sent to the business system. The application solves the technical problem of low data indexing reliability.

Description

Technical Field

[0001] This application relates to the field of database technology, and in particular to data indexing methods, systems, electronic devices, and readable storage media. Background Technology

[0002] In the field of information technology, especially in database management and search engine integration, efficient data retrieval and analysis capabilities have become crucial with the advent of the big data era. To achieve efficient data retrieval and analysis, Elasticsearch (ES) is typically used as an independent data layer. Changes (such as inserts and modifications) in the MySQL database are synchronized to ES in real time or periodically to ensure that the data in ES remains consistent with the data in the MySQL database.

[0003] Updates to Elasticsearch (ES) can lead to a diversification of indexes used for searching the same data, such as expired or failed-in-synchronization indexes. When a business system calls an index in ES for data indexing operations, it often cannot distinguish between the latest and most accurate indexes. This means that the business system may incorrectly select an expired or failed-in-synchronization index, resulting in incorrect data or indexing failures during the data indexing operation. This uncertainty reduces the reliability of the data index.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this application is to provide a data indexing method, system, electronic device, and readable storage medium, aiming to solve the technical problem of low reliability of data indexing.

[0006] To achieve the above objectives, this application provides a data indexing method applied to a data management system connected to a business system. The data management system includes a database and a search engine. The method includes:

[0007] In the case of data synchronization between the database and the search engine, a target index with the latest version type is created in the first index type of the search engine;

[0008] The latest version is stored in the database, and the data synchronization status of the target index is determined in the database;

[0009] If the target index is detected to have completed data synchronization and it is determined that the target index is not abnormal, the data synchronization status of the target index is adjusted to the completed state.

[0010] If a read instruction is received from the business system, and the index requirement corresponding to the read instruction is a first index type, the latest version of the first index type stored in the database is sent to the business system. The business system is used to call the search engine and the latest version to perform data indexing operations.

[0011] In one embodiment, the step of adjusting the data synchronization status of the target index to a completed state after detecting that the target index has completed data synchronization and determining that the target index has no abnormalities includes:

[0012] If the target index is detected to have completed data synchronization, the amount of synchronized data of the target index is obtained, as well as the historical index of the previous time step of the latest version is obtained, and the amount of historical data of the historical index is determined.

[0013] If the absolute difference between the synchronized data volume and the historical data volume is less than a preset fluctuation threshold, it is determined that the target index is not abnormal, and the data synchronization status of the target index is adjusted to the completed state.

[0014] In one embodiment, after the steps of obtaining the historical index of the historical version at the previous time step of the latest version and determining the historical data volume of the historical index, the method further includes:

[0015] If the absolute difference between the synchronized data volume and the historical data volume is greater than or equal to a preset fluctuation threshold, it is determined that the target index is abnormal, the data synchronization status of the target index is adjusted to a failure status, and a synchronization failure prompt is output.

[0016] In one embodiment, after adjusting the data synchronization status of the target index to a failed state and outputting a synchronization failure message, the method further includes:

[0017] The failed version is deleted from the database, and the failed index corresponding to the failed version is deleted from the search engine, wherein the failed version is an index version whose synchronization status is failed.

[0018] In one embodiment, the number of versions for each index type is obtained;

[0019] In each of the index types, a target index type is determined. If the number of target versions of the target index type is greater than a preset version threshold, the difference between the number of target versions and the preset version threshold is taken as the target deletion quantity.

[0020] Each index version belonging to the target index type is sorted sequentially according to its synchronization time, and each index version whose sorting order is less than or equal to the target deletion quantity is determined as an expired version.

[0021] Each expired version is deleted from the database, and the expired index corresponding to each expired version is deleted from the search engine.

[0022] In one embodiment, the database includes a target offline version and / or a target switching version, and the method further includes at least one of the following:

[0023] If an index version offline instruction is received from the business system, the target offline version is determined from the index version offline instruction, and the data synchronization status corresponding to the target offline version is adjusted to the failure status.

[0024] If an index version switching instruction is received from the business system, the target switching version is determined from the index version switching instruction, and the synchronization time corresponding to the target switching version is modified to the current time.

[0025] To achieve the above objectives, this application provides a data indexing method applied to a business system connected to a data management system, wherein the data management system includes a database and a search engine, and the method includes:

[0026] Send a read command to the data management system and receive the latest version of the first index type stored in the database, sent by the data management system based on the read command;

[0027] Upon receiving a data index instruction triggered by an external source on the index interface displayed in the business system, and if the data index instruction contains the latest version, the search engine in the data management system is invoked to determine the target index of the latest version.

[0028] Perform data indexing operations using the target index;

[0029] The database is further configured to, in the event of data synchronization between the database and the search engine, create a target index of the latest version type in the first index type of the search engine, wherein the target index belongs to the first index type; store the latest version in the database and determine the data synchronization status of the target index in the database; and, upon detecting that the target index has completed data synchronization and determining that the target index is without abnormalities, adjust the data synchronization status of the target index to the completed state.

[0030] In addition, to achieve the above objectives, this application also provides an electronic device, the electronic device comprising: a memory, a processor, and a program of the data indexing method stored in the memory and executable on the processor, wherein when the program of the data indexing method is executed by the processor, it implements the steps of the data indexing method as described above.

[0031] In addition, to achieve the above objectives, this application also provides a computer-readable storage medium storing a program implementing a data indexing method, wherein when the program is executed by a processor, it implements the steps of the data indexing method as described above.

[0032] In addition, to achieve the above objectives, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the data indexing method described above.

[0033] One or more technical solutions proposed in this application have at least the following technical effects: When performing data synchronization operations between the database and the search engine, this application creates a target index of the latest version type in the search engine, where the target index belongs to a first index type; stores the latest version in the database and determines the data synchronization status of the target index in the database; upon detecting that the target index has completed data synchronization and determining that the target index is without abnormalities, adjusts the data synchronization status of the target index to a completed state. This achieves the simultaneous storage of the latest version of the first index type and the data synchronization status in the database, facilitating subsequent retrieval of the latest version from the database by the business system. Specifically, upon receiving a read instruction from the business system, where the index requirement corresponding to the read instruction is of the first index type, the latest version of the first index type stored in the database is sent to the business system, enabling the business system to call the search engine and the latest version to perform data indexing operations.

[0034] Since the data synchronization status of the latest version is complete, the business system also obtains the latest version with the data synchronization status complete. Therefore, when the business system calls the search engine and the latest version to perform data indexing operations, it can search for the latest and most accurate data, thereby improving the reliability of data search. Attached Figure Description

[0035] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0036] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart illustrating an embodiment of the data indexing method of this application;

[0038] Figure 2 This is a schematic diagram of the structure of the data management system and the business system in one embodiment of the data indexing method of this application;

[0039] Figure 3 This is a flowchart illustrating another embodiment of the data indexing method of this application;

[0040] Figure 4 This is a schematic diagram showing the index interface in the data indexing method of this application;

[0041] Figure 5 This is a schematic diagram of the system architecture of the data indexing system according to an embodiment of this application;

[0042] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the data indexing method in this application embodiment.

[0043] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0044] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0045] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0046] In the current field of database management and search engine integration, Elasticsearch (ES) is often used to provide efficient data retrieval and analysis capabilities, while MySQL, as a relational database, is used to store structured data.

[0047] Currently, the integration of database management and search engines typically involves treating Elasticsearch (ES) as a separate data layer, interacting with MySQL through data synchronization tools (such as Logstash, JDBC River, etc.). At the hardware level, each server runs a MySQL instance and / or an ES node, communicating over a network. At the software level, data synchronization is usually driven by scheduled tasks or events, synchronizing changes in MySQL to ES to update ES.

[0048] Frequent updates to Elasticsearch (ES) can lead to a diversification of indexes used for searching the same data, such as expired or failed-to-synchronize indexes. When business systems call on ES indexes for data indexing operations, they often cannot distinguish between the latest and most accurate indexes. This means that business systems may incorrectly select expired or failed-to-synchronize indexes, resulting in incorrect data or indexing failures during the indexing process. This uncertainty reduces the reliability of data indexing.

[0049] The main solution of this application embodiment is as follows: During data synchronization, the data management system creates a target index with the latest version type in the search engine and stores the latest version in the database, allowing the database to distinguish between different versions of the index. Furthermore, during data synchronization of the target index, the data management system records the synchronization status of the target index in the database, thereby determining the data synchronization status of the target index within the database. Thus, the business system can read the latest version from the database, and when performing data indexing, it can call the search engine to determine the latest version of the target index for data indexing operations.

[0050] Based on this, this application provides a data indexing method applied to a data management system connected to a business system. The data management system includes a database and a search engine. (Refer to...) Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the data indexing method of this application.

[0051] In this embodiment, the data indexing method includes steps S10 to S40:

[0052] Step S10: In the case of data synchronization between the database and the search engine, create a target index with the latest version type in the first index type of the search engine.

[0053] It should be noted that this implementation applies to a data management system, which includes a search engine and a database. The data management system can communicate with business systems. The database is used to store data; the database can be MySQL. The search engine is used for fast data searching and analysis; the search engine can be Elasticsearch (ES). The search engine can include multiple indexes to perform data searches through these indexes. Business systems can call the search engine. Since business systems are user-facing, users can trigger data indexing instructions on the business system, and the business system can then call the search engine to index the data.

[0054] Data in a database can be synchronized to a search engine. When data in a database changes, the changed data can be synchronized to the search engine. When data in a database changes or is added, and needs to be synchronized to the search engine, the search engine will create a new index to synchronize the data to that new index, thus achieving data synchronization.

[0055] The target index refers to the currently created index in the search engine, while the latest version refers to the index version of the target index. The latest version can be used to indicate the synchronization time of the target index. Since the database may store different types of data, requiring different indexes for searching, and for the same type of data, there may be multiple changes in the database, each requiring synchronization of the changed data to the search engine, multiple indexes may exist for the same type of data in the search engine. Therefore, the first index type may contain multiple indexes, which can be distinguished by their index versions. The version type refers to the type of index version for each index in the first index type. Version types can be categorized based on the synchronization time of the index. For example, in this embodiment, the latest version refers to the newest index version of the target index in the first index type. The synchronization time of the latest version is later than the synchronization time of other indexes in the first index type. Other indexes refer to all indexes in the first index type corresponding to versions other than the latest version.

[0056] For example, when the search engine receives a data synchronization request, a data synchronization operation is performed between the database and the search engine. A first index type is determined from the data synchronization request, and a target index with the latest version type is created within the first index type of the search engine. The latest version can be determined based on the first index type and the time the target index was created; the time the target index was created can be used as the synchronization time. The first index type can be any index type in the database.

[0057] Step S20: Store the latest version in the database and determine the data synchronization status of the target index in the database;

[0058] Data synchronization status reflects the data synchronization progress of the target index. For example, if a search engine creates a target index with the latest version, the database will store the latest version and record the data synchronization status of the target index. When the target index is undergoing data synchronization, the data synchronization status will be "Synchronizing". For instance, a separate area can be allocated in the database to store the latest version and record the data synchronization status of the target index. The database can store the index versions corresponding to multiple indexes of the same index type, and these index versions can be stored together. The database can also store index versions corresponding to multiple index types.

[0059] The database can synchronize data for the target index if it stores the latest version in the database and determines the data synchronization status in the database for recording the data synchronization status of the target index.

[0060] In a feasible embodiment, during the data synchronization process of the target index, data synchronization anomalies may occur. In the event of an anomaly, the data synchronization status of the target index is changed to a failure state, and a synchronization failure message is output to notify relevant maintenance personnel that the data synchronization of the target index has failed. Examples include situations such as the database being locked during synchronization, or the data to be synchronized being modified during the synchronization process. This embodiment only illustrates possible anomalies during data synchronization and does not limit the change of the target index's data synchronization status to a failure state only under these specific anomaly conditions. A change of the data synchronization status to a failure state indicates that the data synchronization of the target index has failed.

[0061] Step S30: If the target index is found to have completed data synchronization and there are no abnormalities in the target index, the data synchronization status of the target index is adjusted to the completed state.

[0062] It's important to note that once the target index has completed data synchronization, the database can adjust its synchronization status to "Completed," indicating successful data synchronization. Business systems can read the latest version of the target index from the database with a "Completed" synchronization status, allowing them to perform data indexing operations on the target index corresponding to that version within the search engine. "No anomalies in the target index" means that the target index exhibits no anomalies during or after data synchronization.

[0063] Step S40: If a read instruction is received from the business system, and the index requirement corresponding to the read instruction is the first index type, the latest version of the first index type stored in the database is sent to the business system. The business system is used to call the search engine and the latest version to perform data indexing operations.

[0064] It should be noted that the business system can read the database in the data management system. The read command instructs the data management system to send the latest version of the first index type stored in the database to the business system. The data management system will then send the latest version of the first index type to the business system. This allows the business system to invoke the search engine and the latest version to perform data indexing operations, thereby retrieving the most accurate and up-to-date data.

[0065] For example, upon detecting that the target index has completed data synchronization and confirming that the target index is free of anomalies, the data synchronization status of the target index is adjusted to "complete." If a read command is received from the business system, and the index requirement corresponding to the read command is the first index type, the latest version of the first index type stored in the database is sent to the business system. The latest version of the first index type may change; when data synchronization is performed on the first index type in the search engine, a new latest version index will be created in the first index type. The business system can then invoke the search engine and the latest version to perform data indexing operations.

[0066] In this embodiment of the application, when performing data synchronization operations between the database and the search engine, a target index of the latest version type is created in the search engine, and the target index belongs to a first index type. The latest version is stored in the database, and the data synchronization status of the target index is determined in the database. When it is detected that the target index has completed data synchronization and it is determined that the target index has no abnormalities, the data synchronization status of the target index is adjusted to a completed state. This achieves the simultaneous data synchronization and storage of the latest version of the first index type and the data synchronization status in the database, facilitating subsequent retrieval of the latest version from the database by the business system. Specifically, when a read instruction is received from the business system, and the index requirement corresponding to the read instruction is the first index type, the latest version of the first index type stored in the database is sent to the business system, enabling the business system to call the search engine and the latest version to perform data indexing operations.

[0067] Since the data synchronization status of the latest version is complete, the business system also obtains the latest version with the data synchronization status complete. Therefore, when the business system calls the search engine and the latest version to perform data indexing operations, it can search for the latest and most accurate data, thereby improving the reliability of data search.

[0068] To better understand data management systems and business systems, please refer to... Figure 2 This section describes the connection between the data management system and the business systems. Figure 2 The system includes a data management system 10 and a business system 20. The data management system 10 includes a MySQL database and an Elasticsearch search engine. When the data management system 10 initiates data synchronization, it creates the latest version of the target index in the search engine and stores the corresponding latest version in the database. It also monitors the amount of synchronized data for the target index to detect any synchronization anomalies. The business system can call the data management system's search engine to search for data. The business system's local caching module, LoadingCache, can retrieve the latest version from the database, and the business system can also retrieve the latest version from its local caching module to call the search engine and search for data using the target index corresponding to the latest version. The business system's local caching module retrieves the latest version when the data synchronization status is complete. When the data management system is performing data synchronization, the local caching module cannot retrieve the latest version that is currently being synchronized. Therefore, in... Figure 2 In this process, the sequence of steps can be as follows: initiate data synchronization; synchronize the target index in the search engine and store the latest version of the target index; when the data synchronization status of the target index is complete, the local module of the business system can obtain the latest version and save it in the business system so that when the business system performs data searches, it can use the latest version to search for data in the search engine. During data synchronization, the process of monitoring the amount of synchronized data can continue to run.

[0069] In other embodiments, while synchronizing data on the target index, if the local cache module of the business system needs to read a version from the database, it can read the version in the database whose synchronization time is closest to the current time, which can be considered the previous time step of the latest version. The business system can also perform data searches based on the previous time step version, thus achieving simultaneous data searching and data synchronization.

[0070] In a feasible embodiment, step S20 further includes steps S21 to S22:

[0071] Step S21: If the target index has completed data synchronization, obtain the amount of synchronized data of the target index, and obtain the historical index of the previous time step of the latest version, and determine the amount of historical data of the historical index.

[0072] It's important to note that after the target index has completed data synchronization, the amount of synchronized data corresponding to the target index can be monitored to avoid synchronization anomalies caused by duplicate data synchronization. The amount of synchronized data refers to the size of the data synchronized to the database in the target index, while the historical data amount refers to the size of the data corresponding to the historical index. The target index has a corresponding index type, and multiple index versions can exist within that index type. Historical indexes and the target index belong to the same index type. The latest version's synchronization time is later than the historical version, and the historical index's synchronization time is earlier than the target index. A historical index can also be considered as an index at the previous time step of the target index within the index type.

[0073] Step S22: If the absolute difference between the synchronized data volume and the historical data volume is less than the preset fluctuation threshold, it is determined that the target index is not abnormal, and the data synchronization status of the target index is adjusted to the completed state.

[0074] It should be noted that the preset fluctuation threshold can be determined based on the actual situation. The difference is the absolute value of the difference between the synchronized data volume and the historical data volume. If the difference is less than the preset fluctuation threshold, it means that the size of the synchronized data volume is about the same as the size of the historical data volume, and there is no abnormal increase or decrease. Therefore, the synchronization status of the target index can be adjusted to the completed state.

[0075] For example, after the target index has completed data synchronization, the amount of synchronized data corresponding to the latest version in the database is obtained to obtain the amount of synchronized data for the target index. Alternatively, the amount of historical data corresponding to the previous time step of the latest version can be obtained to obtain the amount of historical data for the historical index. The absolute value of the difference between the synchronized data amount and the historical data amount is determined to obtain the absolute difference. If the absolute difference is less than a preset fluctuation threshold, the synchronization status of the target index is adjusted to the completed state.

[0076] This embodiment determines whether the data volume changes abnormally after synchronization is completed, thus more accurately identifying successfully synchronized indexes and avoiding the situation where the data synchronization status of abnormally synchronized indexes is also determined to be complete, thereby preventing data indexing failure.

[0077] In one possible embodiment, after step S21, step S211 is further included:

[0078] Step S211: If the absolute difference between the amount of synchronized data and the amount of historical data is greater than or equal to the preset fluctuation threshold, it is determined that there is an anomaly in the target index, and the data synchronization status of the target index is adjusted to a failure status, and a synchronization failure prompt is output.

[0079] It should be noted that if the absolute difference is greater than or equal to the preset fluctuation threshold, it indicates that the amount of synchronized data has increased or decreased abnormally compared with the historical data amount. Therefore, the target index may have a synchronization error, so the synchronization status of the target index needs to be adjusted to the failure status and a synchronization failure prompt should be output.

[0080] For example, if the absolute difference between the synchronized data volume and the historical data volume is greater than or equal to a preset fluctuation threshold, it indicates that there is an anomaly in the target index. The synchronization status of the target index is adjusted to a failure status, and a synchronization failure message is output to the alarm group to notify the relevant maintenance personnel that the target index synchronization has failed and to prompt the relevant maintenance personnel to perform maintenance. The alarm group is an online communication channel for the relevant maintenance personnel, and the relevant maintenance personnel can obtain the synchronization failure message from the alarm group.

[0081] This embodiment adjusts the synchronization status to a failed state when the data volume is abnormal, thereby avoiding synchronization completion. However, it adjusts the status of indexes with synchronization abnormalities to a completed state, which can improve the accuracy of determining whether the target index has been successfully synchronized.

[0082] In a feasible embodiment, the data indexing method further includes step S2111: deleting the failed version from the database and deleting the failed index corresponding to the failed version from the search engine, wherein the failed version is an index version whose synchronization status is failed.

[0083] A failed version refers to an index version whose synchronization status has failed, and a failed index is the index corresponding to that failed version. Data management systems can periodically delete failed versions from the database and also periodically delete the failed indexes corresponding to those failed versions from the search engine. This is to avoid potential data indexing operations that might use failed indexes, leading to data index anomalies.

[0084] Furthermore, refer to Figure 3 In a feasible embodiment, the method further includes steps A10 to A40:

[0085] Step A10: Obtain the number of versions for each index type;

[0086] Step A20: Determine any target index type among all index types. If the number of target versions of the target index type is greater than the preset version threshold, use the difference between the number of target versions and the preset version threshold as the target deletion quantity.

[0087] It should be noted that the number of versions for each index type can be obtained from the data management system's database or from its search engine. The number of versions refers to the number of different versions within the same index type. The target index type can be any index type.

[0088] The preset version threshold refers to the number of versions of the target index type that can be stored in the business system. The preset version thresholds for different target index types can be the same or different. The target version count refers to the total number of versions of the target index type, and the target deletion count refers to the number of expired versions that need to be deleted, or it can refer to the number of expired indexes that need to be deleted.

[0089] For example, the number of versions of each index type can be obtained periodically to delete expired versions and expired indexes on a regular basis.

[0090] Step A30: Sort each index version belonging to the target index type in order of their synchronization time, and determine each index version whose sorting order is less than or equal to the target number of deletions as an expired version.

[0091] Step A40: Delete each expired version from the database and delete the corresponding expired index for each expired version from the search engine.

[0092] It should be noted that the synchronization time refers to the synchronization time of the index version. Different index versions within the same index type have different synchronization times and exist in a temporal order. In this embodiment, the index versions within the same index type can be sorted sequentially according to their respective synchronization times. That is, the index version with an earlier synchronization time has a lower order than the index version with a later synchronization time.

[0093] To avoid storing too many index versions in the data management system and causing storage pressure, when the target number of versions exceeds a preset version threshold, index versions with earlier synchronization times can be deleted first, while those with later synchronization times are retained. In other words, older index versions are deleted, and newer index versions are saved. For example, index versions with a sorting order less than or equal to the target deletion quantity can be prioritized, deleting older versions and retaining newer ones. Expired versions can be considered older index versions, and expired indexes are the indexes corresponding to expired versions.

[0094] For example, the number of versions of each index type in the data management system can be periodically retrieved. For each target index type, if the number of target versions exceeds a preset version threshold, the difference between the target version number and the preset version threshold is used as the target deletion quantity. Expired versions can be identified within the target index type. For instance, the index versions belonging to the target index type can be sorted sequentially according to their synchronization time. Index versions whose sorting order is less than or equal to the target deletion quantity are identified as expired versions. The data management system deletes each expired version from the database and also updates the corresponding expired indexes of each expired version in the search engine. Furthermore, after identifying expired versions, the data management system can also send deletion commands to the business system, allowing the business system to delete the expired versions based on these commands.

[0095] This embodiment reduces unnecessary storage, frees up database storage space, and relieves storage pressure on the search engine by deleting expired versions and expired indexes.

[0096] In a feasible embodiment, the database includes the target offline version and / or the target switching version, and the data indexing method further includes steps B10 to B20:

[0097] Step C10: If an index version offline instruction is received from the business system, the target offline version is determined from the index version offline instruction, and the data synchronization status corresponding to the target offline version is adjusted to the failure status.

[0098] It should be noted that the index version offline indication can be triggered by the user on the data index interface that is visually displayed on the business system. For example, after the user triggers the index version offline indication on the business system, the business system can send the index version offline indication to the data management system. After receiving the index version offline indication, the data management system can determine the target offline version from the index version offline indication.

[0099] A target offline version refers to an index version whose data synchronization status needs to be adjusted to a failed state. Since no data synchronization anomalies may be detected during or after data synchronization, data search failures may occur when calling the index for indexing operations. Therefore, users can instruct the data management system to remove the target offline version from the database and its corresponding index from the search engine, thus preventing subsequent indexing operations on the target offline version and avoiding indexing failures.

[0100] For example, the system receives an index version offline instruction from the business system, determines the target offline version from the index version instruction, and adjusts the data synchronization status corresponding to the target offline version to a failed state.

[0101] Step B20: If an index version switching instruction is received from the business system, determine the target switching version from the index version switching instruction, and modify the synchronization time corresponding to the target switching version to the current time.

[0102] It should be noted that the index version switching indicator can be triggered by the user on the data index interface that is visually displayed on the business system.

[0103] For example, after a user triggers an index version switching instruction on the business system, the business system can send an index version switching instruction to the data management system. After receiving the index version switching instruction, the data management system can determine the target switching version from the index version switching instruction.

[0104] Since only one index of the same index type can be accessed simultaneously during data indexing, this embodiment will use the latest index version within the index type, i.e., the index version with the synchronization time closest to the current time. In this embodiment, data indexing is performed by the business system calling the index in the search engine within the data management system. Before performing the data indexing operation, the business system first reads the latest index version of the corresponding index type from the data management system. Therefore, when a user wants to access a historical index version within the index type for data indexing, they can switch that historical index version to the latest version to perform the data indexing operation.

[0105] A target version change can refer to a historical index version corresponding to the latest index version within the same index type. When the data management system modifies the synchronization time corresponding to the target version change in the database to the current time, it indicates that the target version change is the latest index version within the corresponding index type. Therefore, the business system can read the target version change to perform data indexing and other operations using it.

[0106] For example, an index version switching instruction is received, and the target switching version is determined from the instruction. The synchronization time corresponding to the target switching version is then modified to the current time. This implementation, by receiving the index version switching instruction, allows for flexible selection of the index version for data indexing operations, improving the flexibility of data indexing while ensuring its accuracy.

[0107] Furthermore, in another embodiment, the data indexing method can also be applied to a business system connected to a data management system, which includes a database and a search engine, and the method includes:

[0108] Step E10: Send a read command to the data management system and receive the latest version of the first index type stored in the database, sent by the data management system based on the read command;

[0109] It should be noted that the business system and the data management system can communicate with each other. The business system can call the search engine in the data management system to perform data indexing operations. The business system can periodically read the latest version from the database, and can synchronously read the latest version from multiple index types. The latest version read by the business system from the database is the one with the synchronization time closest to the current time and whose corresponding data synchronization status is complete.

[0110] The latest version is the most recent index version for the corresponding index type, and it is also the index version with the closest synchronization time to the current time. For any index type, the business system can periodically retrieve the latest version from the database.

[0111] Because data synchronization may occur multiple times for the same index type within a data management system, the latest version of the same index type may differ at different times. Therefore, business systems can periodically read the latest version from the database to obtain the most up-to-date index version. This eliminates the need for manual switching of index versions on the business system; the system can automatically read the target version for data searching.

[0112] Step E20: Upon receiving a data index indication triggered by an external source on the index interface displayed in the business system, and if the data index indication contains the latest version, the search engine in the data management system is invoked to determine the target index of the latest version.

[0113] Step E30: Perform a data indexing operation using the target index;

[0114] The database is further configured to, in the event of data synchronization between the database and the search engine, create a target index of the latest version type in the first index type of the search engine, wherein the target index belongs to the first index type; store the latest version in the database and determine the data synchronization status of the target index in the database; and, upon detecting that the target index has completed data synchronization and determining that the target index is without abnormalities, adjust the data synchronization status of the target index to the completed state.

[0115] It should be noted that when the business system reads data from the database, it first determines which index type to read from the database. When it is determined to be the first index type, it will read the latest version from the first index type and save it in the local cache of the business system.

[0116] It should be noted that the business system can display a data index interface. This interface shows the index version corresponding to the index type. Data index instructions instruct the business system to perform data indexing operations. The latest version can be determined from these instructions; the latest version is the target index corresponding to that version, as indicated by the business system. Upon receiving a data index instruction instructing the business system to perform data indexing operations using the target index corresponding to the latest version, the business system can invoke a search engine to determine the target index corresponding to the latest version, and then use the search engine to execute the data indexing operation using that target index.

[0117] For example, when a business system receives a data index instruction triggered by a user on a visually displayed index interface, and the data index instruction includes the latest version, the business system calls the search engine to determine the target index corresponding to the latest version, and then calls the corresponding target index through the search engine to perform data indexing operations.

[0118] In other embodiments, when a user triggers a data indexing instruction on the business system, the user selects an index type on the business system's indexing interface to determine the version corresponding to that index type for data indexing, thus generating the data indexing instruction. When the user selects an index type, the business system synchronously determines the version used for data indexing under that index type; specifically, it determines the version with the closest synchronization time to the current time under that index type. It is understood that the business system can determine the version to be indexed and the corresponding index type from the data indexing instruction. When the business system performs a data search, the corresponding index type can be a hotel index type, a product index type, etc. This embodiment does not specifically limit this.

[0119] For a better understanding of this implementation, please refer to Figure 4 The following is an example illustrating the content displayed on the index interface in this embodiment. Figure 4 This is a schematic diagram of the index interface. Figure 4 The index interface shown is for illustrative purposes only. Other content can also be displayed on the index interface. Here, "index management" indicates that the index interface is currently in index management mode, and the currently selected index type is "hotel." Figure 4The system displays two index versions. Each index version can include an index ID, index type, index name, edit time, and index status. Switching to and deactivating each index version is supported. However, index versions that are currently active cannot be switched to or deactivated, while those that are currently inactive can be switched to or deactivated. The index name is the name of the index version, and the index type refers to the type of index version. Figure 4 The two index versions belong to the hotel index type. Figure 4 The edit time displayed refers to the synchronization time of the index version. For example, an index version named Hotel_2400 has a hotel index type, was edited on July 3, 2024 at 00:22:13, and its index status is "Enabled." The corresponding operations include switching and delisting. However, since the index status is "Enabled," this index version will not undergo switching or indexing operations at this time. An index version named Hotel_2410 has a hotel index type, was edited on July 2, 2024 at 00:21:13, and its index status is "Disabled." The corresponding operations include switching and delisting. Since the index status is "Disabled," this index version can undergo switching and delisting operations. Users can click "Switch" to trigger an index switching operation and click "Delist" to trigger an index delisting indication.

[0120] This embodiment provides an easy-to-use index interface, allowing users to intuitively view and manage indexes, including index versions and the corresponding switching and decommissioning operations, thereby improving user experience and operational efficiency.

[0121] This application also provides a data indexing system, see embodiments thereof. Figure 5 The data indexing system includes business systems and data management systems. The data management system includes databases and search engines. The data indexing system includes:

[0122] The data management system 10 is configured to, in the case of data synchronization operations between the database and the search engine, create a target index of the latest version type in the first index type of the search engine; store the latest version in the database and determine the data synchronization status of the target index in the database; adjust the data synchronization status of the target index to the completed state when it is detected that the target index has completed data synchronization and it is determined that the target index has no abnormalities; and if a read instruction is received from the business system and the index requirement corresponding to the read instruction is the first index type, send the latest version of the first index type stored in the database to the business system.

[0123] Business system 20 is used to send a read instruction to the data management system and receive the latest version of the first index type stored in the database, sent by the data management system based on the read instruction; when it receives a data index indication triggered by an external source on the index interface displayed in the business system, and the data index indication contains the latest version, it calls the search engine in the data management system to determine the target index of the latest version; and performs a data indexing operation through the target index.

[0124] This application provides an electronic device, which can be a playback device. The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the data indexing method described above.

[0125] The following is for reference. Figure 6 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0126] like Figure 6As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device allows the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although the diagrams show electronic devices with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented alternatively.

[0127] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1009, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of embodiments of this disclosure.

[0128] The electronic device provided in this application employs the data indexing method in Embodiment 1 above to solve the technical problem of low reliability of data indexing. Compared with the prior art, the beneficial effects of product traffic data allocation provided in this application are the same as the beneficial effects of the data indexing method provided in the above embodiments, and other technical features in this electronic device are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here. It should be understood that the various parts of this disclosure can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any suitable manner in one or more embodiments or examples. The above description is only a specific embodiment of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims. This embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, which are used to execute the data indexing method in Embodiment 1 above. The computer-readable storage medium provided in this application embodiment may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, apparatuses, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable EPROM (Electrical Programmable Read Only Memory) or flash memory, optical fiber, portable compact disk CD-ROM (compact disc read-only memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution device, apparatus, or apparatus. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof. The aforementioned computer-readable storage medium may be included in an electronic device; or it may exist independently and not assembled into an electronic device.The aforementioned computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to: create a target index of the latest version type in a first index type of the search engine during data synchronization operations between the database and the search engine; store the latest version in the database and determine the data synchronization status of the target index in the database; adjust the data synchronization status of the target index to a completed state when it is detected that the target index has completed data synchronization and it is determined that the target index has no abnormalities; and if a read instruction is received from a business system, and the index requirement corresponding to the read instruction is the first index type, send the latest version of the first index type stored in the database to the business system, wherein the business system is used to invoke the search engine and the latest version to perform data indexing operations.

[0129] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a LAN (local area network) or WAN (wide area network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based device that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0131] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules do not necessarily limit the specific unit. The computer-readable storage medium provided in this application stores computer-readable program instructions for executing the above-described data indexing method, aiming to solve the technical problem of low reliability in data indexing. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the embodiments of this application are the same as the beneficial effects of the data indexing method provided in the above embodiments, and will not be repeated here.

[0132] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the data indexing method described above. The computer program product provided by this application aims to solve the technical problem of low reliability in data indexing. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of this application are the same as the beneficial effects of the data indexing method provided in the above embodiments, and will not be repeated here. The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. A data indexing method, characterized in that, The method described is applied to a data management system connected to a business system, which includes a database and a search engine. In the case of data synchronization between the database and the search engine, a target index with the latest version type is created in the first index type of the search engine; The latest version is stored in the database, and the data synchronization status of the target index is determined in the database; If the target index is detected to have completed data synchronization and it is determined that the target index is not abnormal, the data synchronization status of the target index is adjusted to the completed state. If a read instruction is received from the business system, and the index requirement corresponding to the read instruction is a first index type, the latest version of the first index type stored in the database is sent to the business system, wherein the business system is used to call the search engine and the latest version to perform data indexing operations; The step of adjusting the data synchronization status of the target index to a completed state when it is detected that the target index has completed data synchronization and it is determined that the target index has no abnormalities includes: If the target index is detected to have completed data synchronization, the amount of synchronized data of the target index is obtained, as well as the historical index of the previous time step of the latest version is obtained, and the amount of historical data of the historical index is determined. If the absolute difference between the synchronized data volume and the historical data volume is less than a preset fluctuation threshold, it is determined that the target index is not abnormal, and the data synchronization status of the target index is adjusted to the completed state.

2. The method as described in claim 1, characterized in that, After the steps of obtaining the historical index of the historical version at the previous time step of the latest version and determining the historical data volume of the historical index, the method further includes: If the absolute difference between the synchronized data volume and the historical data volume is greater than or equal to a preset fluctuation threshold, it is determined that the target index is abnormal, the data synchronization status of the target index is adjusted to a failure status, and a synchronization failure prompt is output.

3. The method as described in claim 2, characterized in that, After adjusting the data synchronization status of the target index to a failed state and outputting a synchronization failure message, the method further includes: The failed version is deleted from the database, and the failed index corresponding to the failed version is deleted from the search engine, wherein the failed version is an index version whose synchronization status is failed.

4. The method as described in claim 1, characterized in that, The method includes: Get the number of versions for each index type; In each of the index types, a target index type is determined. If the number of target versions of the target index type is greater than a preset version threshold, the difference between the number of target versions and the preset version threshold is taken as the target deletion quantity. Each index version belonging to the target index type is sorted sequentially according to its synchronization time, and each index version whose sorting order is less than or equal to the target deletion quantity is determined as an expired version. Each expired version is deleted from the database, and the expired index corresponding to each expired version is deleted from the search engine.

5. The method as described in claim 1, characterized in that, The database includes the target offline version and / or the target switching version, and the method further includes at least one of the following: If an index version offline instruction is received from the business system, the target offline version is determined from the index version offline instruction, and the data synchronization status corresponding to the target offline version is adjusted to the failure status. If an index version switching instruction is received from the business system, the target switching version is determined from the index version switching instruction, and the synchronization time corresponding to the target switching version is modified to the current time.

6. A data indexing method, characterized in that, The method, applied to a business system connected to a data management system, which includes a database and a search engine, comprises: Send a read command to the data management system and receive the latest version of the first index type stored in the database, sent by the data management system based on the read command; Upon receiving a data index instruction triggered by an external source on the index interface displayed in the business system, and if the data index instruction contains the latest version, the search engine in the data management system is invoked to determine the target index of the latest version. Perform data indexing operations using the target index; The database is further configured to, in the event of data synchronization between the database and the search engine, create a target index of the latest version type in the first index type of the search engine, wherein the target index belongs to the first index type; store the latest version in the database and determine the data synchronization status of the target index in the database; and, upon detecting that the target index has completed data synchronization and determining that the target index is without abnormalities, adjust the data synchronization status of the target index to the completed state. The data management system is further configured to, upon detecting that the target index has completed data synchronization, obtain the amount of synchronized data of the target index, and obtain the historical index of the previous time step of the latest version, and determine the amount of historical data of the historical index; if the absolute difference between the amount of synchronized data and the amount of historical data is less than a preset fluctuation threshold, determine that the target index is without abnormality, and adjust the data synchronization status of the target index to the completed state.

7. A data indexing system, characterized in that, The data indexing system includes a data management system and a business system. The data management system also includes a search engine and a database. A data management system is used to create a target index with the latest version type in a first index type of the search engine when performing data synchronization operations between the database and the search engine; store the latest version in the database; and determine the data synchronization status of the target index in the database. If the target index is detected to have completed data synchronization and it is determined that the target index is without abnormalities, the data synchronization status of the target index is adjusted to the completed state; if a read instruction is received from the business system and the index requirement corresponding to the read instruction is the first index type, the latest version of the first index type stored in the database is sent to the business system; The data management system is further configured to, upon detecting that the target index has completed data synchronization, obtain the amount of synchronized data of the target index, and obtain the historical index of the previous time step of the latest version, and determine the amount of historical data of the historical index; if the absolute difference between the amount of synchronized data and the amount of historical data is less than a preset fluctuation threshold, determine that the target index is without abnormality, and adjust the data synchronization status of the target index to the completed state. The business system is configured to send a read instruction to the data management system, receive the latest version of a first index type stored in the database from the data management system based on the read instruction; and, upon receiving a data index indication triggered by an external source on the index interface displayed in the business system's visualization, and if the data index indication contains the latest version, invoke the search engine in the data management system to determine the target index of the latest version; and perform a data indexing operation through the target index.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the steps of the data indexing method according to any one of claims 1 to 6.

9. A readable storage medium, characterized in that, The readable storage medium is a computer-readable storage medium on which a program implementing the data indexing method is stored, the program implementing the data indexing method being executed by a processor to implement the steps of the data indexing method as claimed in any one of claims 1 to 6.

Images