ES extension word bank high availability implementation method and device, equipment and storage medium

Abstract

The invention relates to the technical field of data processing, and provides an ES extended word bank high availability implementation method and device, equipment and a storage medium, and the method comprises the following steps: receiving a self-defined ES extended word bank file; writing the ES extension word bank file into a corresponding cloud storage according to a cluster architecture, and recording a corresponding file access path during writing; and issuing the file access path to a cluster configuration parameter of a production cluster in an ES cluster, so that the production cluster remotely loads an ES extension lexicon file according to the file access path. According to the embodiment of the invention, the problem of configuration inconsistency between the customized ES extension word banks of the ES cluster can be reduced or avoided, and the availability of the ES extension word banks is improved.

Description

Methods, apparatus, devices and storage media for achieving high availability of ES extended dictionary Technical Field

[0001] This specification relates to the field of data processing technology, and in particular to a method, apparatus, device and storage medium for implementing high availability of an ES extended thesaurus. Background Technology

[0002] Elasticsearch (ES) is an open-source, distributed, highly scalable, near real-time search and data analysis engine that can easily handle the search and analysis of large amounts of data. To improve search relevance and accuracy, ES's IK segmenter (IK segmentation plugin) supports custom ES extended dictionaries, allowing users to customize and extend the ES segmentation vocabulary according to specific needs.

[0003] An Elasticsearch (ES) cluster consists of multiple nodes, each configured with a custom extended term file (ES extended term library) locally. This can easily lead to inconsistencies in the configuration of ES extended terms across multiple nodes. Furthermore, if one or more ES nodes fail, their ES extended terms become unusable, which can negatively impact the relevance and accuracy of searches across the entire ES cluster. Summary of the Invention

[0004] The purpose of the embodiments in this specification is to provide a method, apparatus, device and storage medium for achieving high availability of ES extended dictionaries, reduce or avoid configuration inconsistencies between custom ES extended dictionaries in an ES cluster, and improve the availability of ES extended dictionaries.

[0005] To achieve the above objectives, this specification provides a method for implementing high availability of an ES extended thesaurus, including:

[0006] Receive custom ES extended dictionary files;

[0007] According to the cluster architecture, the ES extended dictionary file is written to the corresponding cloud storage, and the file access path at the time of writing is recorded.

[0008] The file access path is distributed to the cluster configuration parameters of the production cluster in the ES cluster, so that the production cluster can remotely load the ES extended dictionary file according to the file access path.

[0009] The high availability implementation method of the ES extended dictionary in the embodiments of this specification, wherein writing the ES extended dictionary file to the corresponding cloud storage according to the cluster architecture includes:

[0010] If the ES cluster is a dual-active cluster in the same city, then the ES extended dictionary file is written to the first object storage and the second object storage; the first object storage is the object storage corresponding to the production cluster, and the second object storage is the object storage corresponding to the disaster recovery cluster in the same city.

[0011] If the ES cluster is a multi-site active-active cluster, the ES extended dictionary file is written to the first object storage and the second object storage, and the ES extended dictionary file in the first object storage is asynchronously transmitted to the third object storage; the third object storage is the object storage corresponding to the disaster recovery cluster.

[0012] The ES extended thesaurus high availability implementation method in this specification, according to the embodiments, distributes the file access path to the cluster configuration parameters of the production cluster in the ES cluster, including:

[0013] The middleware management service distributes the file access path to the production cluster in the ES cluster, so that the production cluster can store it in the corresponding cluster configuration parameters.

[0014] The high availability implementation method of the ES extended dictionary in the embodiments of this specification, wherein the ES cluster remotely loads the ES extended dictionary file according to the file access path, includes:

[0015] If the ES cluster is a local active-active cluster or a multi-active cluster in different locations, the ES cluster reads the ES extended dictionary file from the first object storage and the second object storage respectively according to the corresponding file access path, and takes the union of the read ES extended dictionary files.

[0016] The high availability implementation method for the ES extended dictionary in the embodiments of this specification further includes, before writing the ES extended dictionary file to the corresponding cloud storage according to the cluster architecture:

[0017] Write the ES extended thesaurus file into the database corresponding to the production cluster.

[0018] The ES extended dictionary high availability implementation method in the embodiments of this specification also includes:

[0019] When an error occurs while writing the ES extended dictionary file to object storage, the corresponding ES extended dictionary file is retrieved from the database and retried.

[0020] When an error occurs while reading the ES extended dictionary file from object storage, the corresponding ES extended dictionary file is read from the database.

[0021] On the other hand, embodiments of this specification also provide a high-availability implementation device for an ES extended dictionary, including:

[0022] The receiving module is used to receive custom ES extended dictionary files;

[0023] The storage module is used to write the ES extended dictionary file to the corresponding cloud storage according to the cluster architecture, and record the file access path corresponding to the writing process.

[0024] The distribution module is used to distribute the file access path to the cluster configuration parameters of the production cluster in the ES cluster, so that the production cluster can remotely load the ES extended dictionary file according to the file access path.

[0025] On the other hand, embodiments of this specification also provide a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the computer program, when run by the processor, executes instructions for the above-described method.

[0026] On the other hand, embodiments of this specification also provide a computer storage medium storing a computer program thereon, which, when run by the processor of a computer device, executes instructions for the above-described method.

[0027] On the other hand, embodiments of this specification also provide a computer program product, which includes a computer program that, when run by the processor of a computer device, executes instructions for the above-described method.

[0028] As can be seen from the technical solutions provided in the embodiments of this specification above, the embodiments of this specification can write the ES extended dictionary file to the corresponding cloud storage according to the cluster architecture, record the corresponding file access path when writing, and then send the file access path to the cluster configuration parameters of the production cluster in the ES cluster, so that the production cluster can remotely load the ES extended dictionary file according to the file access path; in this way, the remote centralized storage and centralized maintenance of synonyms are realized, thereby breaking down the barrier of each node maintaining its own customized extended dictionary file locally, avoiding the problem of inconsistent extended dictionary file configurations between ES cluster nodes, and improving the availability of the ES extended dictionary. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0030] Figure 1 shows a schematic diagram of the architecture of a dual-active ES cluster in the same city in some embodiments of this specification;

[0031] Figure 2 shows a flowchart of a method for implementing high availability of the ES extended dictionary in some embodiments of this specification;

[0032] Figure 3 shows a flowchart of a high-availability implementation method for the ES extended lexicon in some other embodiments of this specification;

[0033] Figure 4 shows a structural block diagram of a high-availability implementation device for the ES extended lexicon in some embodiments of this specification;

[0034] Figure 5 shows a structural block diagram of a computer device in some embodiments of this specification.

[0035] [Explanation of Labels in the Attached Image]

[0036] 41. Receiving module;

[0037] 42. Save module;

[0038] 43. Distribution module;

[0039] 502. Computer equipment;

[0040] 504, Processor;

[0041] 506. Memory;

[0042] 508. Drive mechanism;

[0043] 510. Input / output interface;

[0044] 512. Input devices;

[0045] 514. Output devices;

[0046] 516. Presentation equipment;

[0047] 518. Graphical User Interface;

[0048] 520. Network interface;

[0049] 522. Communication link;

[0050] 524. Communication bus. Detailed Implementation

[0051] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0052] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in the embodiments of this specification are all information and data authorized and agreed upon by the user and fully authorized by all parties. That is, the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with the relevant provisions of national laws and regulations.

[0053] Figure 1 illustrates the architecture of a dual-active Elasticsearch (ES) cluster within the same city in some embodiments of this application. This architecture can include a control plane and a compute plane. The control plane is primarily responsible for cluster management, cluster resource monitoring, and scheduling and monitoring of containerized ES resources within the compute plane. The control plane employs a hot standby architecture (including a primary control plane and a backup control plane), deployed in the production data center and the same city data center, respectively. Under normal circumstances, only the control plane in the production data center (i.e., the primary control plane) provides cluster management and ES resource management functions within the compute plane. Resource management may include front-end services, back-end services, middleware management services, databases, and object storage services (OBS). The backup control plane serves only as a hot standby and does not provide services under normal circumstances. When the primary control plane fails, the backup control plane takes over the responsibilities of the primary control plane, performing cluster management and ES resource management on the compute plane. In this case, the primary and backup control planes switch roles, thereby achieving a cross-data center disaster recovery and high availability architecture for ES, ensuring the business continuity of the application system.

[0054] In Figure 1, M represents the Master node in the compute plane cluster. The Master node is responsible for managing and maintaining the state and metadata information of the Elasticsearch (ES) cluster, including but not limited to deciding which shards are assigned to which nodes, tracking node joining or leaving, and handling operations such as creating or deleting indexes to ensure the normal operation of the ES cluster. D represents the data node in the compute plane cluster. The data node is responsible for storing, retrieving, and processing data, providing search and aggregation functions. C represents the coordinating node in the compute plane cluster. The coordinating node is similar to a load balancer, responsible for routing client requests to appropriate data nodes and aggregating the results and returning them to the client. V represents the voting node in the compute plane cluster. The voting node, also known as the arbitration node, does not carry service traffic and is only used for the election of the Master node in the ES cluster. That is, it only has the right to vote for the Master node, but does not have the right to be elected as the Master node, nor does it assume the responsibilities of the Master node. Since the ES cluster must have a running Master node and the election of the Master node requires more than half of the Master nodes to vote, when the ES cluster is deployed across data centers, an arbitration data center needs to be reserved for deploying the voting node to ensure cluster stability and data consistency.

[0055] In some embodiments of this application, each node is an Elasticsearch instance or process. Nodes can be deployed on physical machines, virtual machines, or containers, and they begin running whenever Elasticsearch starts. These Master nodes, data nodes, and vote nodes in the compute plane work together to form an Elasticsearch cluster.

[0056] Figure 1 only illustrates a dual-active ES cluster architecture within the same city. In other embodiments, a multi-active ES cluster architecture with two locations and three centers (local primary + local backup + remote backup) can also be adopted as needed. This specification does not limit this to a single architecture.

[0057] This specification provides an embodiment of a high-availability implementation method for an ES extended dictionary, which can be applied to the aforementioned control plane (main control plane). Referring to Figure 2, some embodiments of this specification may include the following steps:

[0058] Step 201: Receive the custom ES extended dictionary file.

[0059] Step 202: Write the ES extended dictionary file to the corresponding cloud storage according to the cluster architecture, and record the file access path corresponding to the writing.

[0060] Step 203: Send the file access path to the cluster configuration parameters of the production cluster in the ES cluster, so that the production cluster remotely loads the ES extended thesaurus file according to the file access path.

[0061] In the embodiments of this specification, the ES extended thesaurus file can be written into the corresponding cloud storage according to the cluster architecture, and the corresponding file access path during writing is recorded. Then, the file access path is sent to the cluster configuration parameters of the production cluster in the ES cluster, so that the production cluster can remotely load the ES extended thesaurus file according to the file access path. In this way, remote centralized storage and centralized maintenance of synonyms are achieved, breaking the barrier of each node locally maintaining its own defined configuration of the extended thesaurus file, avoiding the problem of inconsistent configuration of the extended thesaurus files between ES cluster nodes, and improving the availability of the ES extended thesaurus. Moreover, since the method for realizing the high availability of the ES extended thesaurus in the embodiments of this specification runs under the同城双活架构 (dual-active architecture in the same city), the high availability of the ES extended thesaurus is further improved.

[0062] In some embodiments of this specification, the ES extended thesaurus can include, but is not limited to, a custom synonym extended thesaurus, a word segmentation extended thesaurus, a stop word extended thesaurus, etc., to enhance the ES word segmentation ability and improve the relevance and accuracy of search. These ES extended thesauruses all exist in the form of files (i.e., ES extended thesaurus files).

[0063] Among them, the synonym extended thesaurus is mainly used to handle situations such as one word having multiple meanings and multiple words having the same meaning, thereby enhancing the accuracy and richness of search. For example, searching for "ES" can query "Elasticsearch". Custom word segmentation supports users to customize ES word segmentation vocabulary according to specific needs, such as professional terms, brand names, and new network words. Custom stop words support users to customize stop word vocabulary according to specific needs, which are often used for some common but meaningless words, such as "的", "了", "呢", etc., so as to ignore these words during word segmentation and improve the relevance of search.

[0064] In some embodiments of this specification, the control plane supports users to upload ES extended thesaurus files such as custom synonyms, custom word segmentation, and custom stop words. After receiving the ES extended thesaurus file, the control plane can write the ES extended thesaurus file into the corresponding database of the production cluster (such as the production DB in Figure 3, that is, the production cluster database) for data backup. In this way, if an exception occurs during subsequent execution, the stored ES extended thesaurus file can be read from the database and retried. For example, when writing the ES extended thesaurus file to the object storage is abnormal, the corresponding ES extended thesaurus file can be obtained from the database and retried.

[0065] Referring to Figure 3, in some embodiments of this specification, after receiving a user's request to send an ES extended dictionary file uploaded via the front end, the user can be authenticated. After the user is authenticated, the ES extended dictionary file can be written to the corresponding cloud storage according to the cluster architecture. Specifically, this can include the following situations:

[0066] (I) Active-active clusters within the same city

[0067] In some embodiments of this specification, if the ES cluster is a dual-active cluster in the same city, the ES extended dictionary file is written to a first object storage (e.g., the production OBS in Figure 3) and a second object storage (e.g., the same-city OBS in Figure 3); wherein, the first object storage is the object storage (OBS) corresponding to the production cluster, and the second object storage is the object storage (OBS) corresponding to the disaster recovery cluster in the same city; that is, in the scenario of a dual-active cluster in the same city, the ES extended dictionary file is written to cloud storage twice.

[0068] (II) Multi-site active cluster

[0069] In some embodiments of this specification, if the ES cluster is a multi-site active-active cluster, the ES extended dictionary file is written to a first object storage (e.g., the production OBS in Figure 3) and a second object storage (e.g., the local OBS in Figure 3), and the ES extended dictionary file in the first object storage is asynchronously transmitted to a third object storage (e.g., the remote OBS in Figure 3). The first object storage is the object storage (OBS) corresponding to the production cluster, the second object storage is the object storage (OBS) corresponding to the local disaster recovery cluster, and the third object storage is the object storage (OBS) corresponding to the remote disaster recovery cluster. That is, in a multi-site active-active cluster scenario, the ES extended dictionary file is written to multiple cloud storages.

[0070] (III) Single Cluster

[0071] In some embodiments of this specification, if the ES cluster is a single cluster, the ES extended dictionary file can be written to the object storage corresponding to the production cluster (e.g., the production OBS in Figure 3).

[0072] In some embodiments of this specification, regardless of the cluster scenario, after the ES extended dictionary file is written to the corresponding cloud storage according to the cluster architecture, the corresponding file access path at the time of writing is recorded. Since it is cloud storage, the recorded file access path is a remote access path. In dual-write or multi-cloud storage scenarios, two or more file access paths need to be recorded.

[0073] In some embodiments of this specification, after the ES extended dictionary file is uploaded to OBS, the corresponding file access path can be distributed to the cluster configuration parameters of the production cluster in the ES cluster for storage, so that the production cluster can remotely load the ES extended dictionary file according to the file access path. For example, in some embodiments of this specification, the file access path can be distributed to the production cluster in the ES cluster through a middleware management service, so that the production cluster can store it in the corresponding cluster configuration parameters.

[0074] In some embodiments of this specification, the ES cluster remotely loads the ES extended dictionary file according to the file access path, which may include: if the ES cluster is a local active-active cluster or a multi-active cluster in different locations, the ES cluster reads the ES extended dictionary file from the first object storage and the second object storage respectively according to the corresponding file access path, and takes the union of the read ES extended dictionary files; thus, by reading the ES extended dictionary files twice and taking the union of the read ES extended dictionary files, the search range can be effectively expanded, which is conducive to further improving the relevance and accuracy of the search.

[0075] Referring to Figure 3, in an exemplary embodiment of this specification, taking a thesaurus extension in a local active-active cluster scenario as an example, the ES cluster can filter out indexes involving the use of synonyms from the full index (e.g., automatically filtering and loading the configured synonyms in Figure 3); obtain the remote access paths (production OBS remote address, local OBS remote address) corresponding to the indexes involving synonyms; and load the thesaurus file using dual reads based on the remote access paths (production OBS remote address, local OBS remote address). In the embodiments of this specification, the index is a logical container in ES used to store and manage related data, containing a group of documents with similar structures.

[0076] In an exemplary embodiment of this specification, since the control plane has previously written the ES extended dictionary file to the database corresponding to the production cluster, when there is an error reading the ES extended dictionary file from OBS, the ES cluster can still read the corresponding ES extended dictionary file from the database, thereby further improving the availability of the ES extended dictionary.

[0077] In an exemplary embodiment of this specification, taking a dual-active cluster scenario within the same city as an example, the control plane can update the ES extended thesaurus according to the following steps:

[0078] Step 1: Receive the request to update the extended dictionary file;

[0079] Step 2: Modify the file status and update the file content in the database. If an error occurs in subsequent steps, the extended dictionary content stored in the database can be used for retry.

[0080] Step 3: A scheduled task scans the file status in the database. If the status is "pending update", the file content is integrated and updated sequentially to the production and local object storage services, and the download addresses of the production and local remote files are also updated.

[0081] Step 4: Send the production and local download addresses to the running Elasticsearch cluster and update the corresponding extended dictionary remote address cluster configuration in the Elasticsearch cluster.

[0082] In an exemplary embodiment of this specification, taking a dual-active cluster scenario within the same city as an example, the control plane can delete the ES extended thesaurus according to the following steps:

[0083] Step 1: Receive the request to delete the extended dictionary file.

[0084] Step 2: Modify the file status and delete the file content from the database.

[0085] Step 3: The scheduled task scans the file status in the database. If the status is "to be deleted", the file is deleted from the production or local object storage service. If the status is "an execution error occurs at a certain stage", the process is retried from that stage.

[0086] Step 4: Clear the configuration content of the production and local extended dictionary remote address of the running ES cluster.

[0087] Although the process described above includes multiple operations that occur in a specific order, it should be clearly understood that these processes may include more or fewer operations that can be executed sequentially or in parallel (e.g., using parallel processors or a multithreaded environment).

[0088] Corresponding to the above-described method for implementing high availability of the ES extended dictionary, this specification also provides an ES extended dictionary high availability implementation device, which can be configured on the aforementioned control plane. Referring to Figure 4, in some embodiments of this specification, the ES extended dictionary high availability implementation device may include:

[0089] Receiver module 41 is used to receive custom ES extended dictionary files;

[0090] The storage module 42 is used to write the ES extended dictionary file to the corresponding cloud storage according to the cluster architecture, and record the file access path corresponding to the writing process.

[0091] The distribution module 43 is used to distribute the file access path to the cluster configuration parameters of the production cluster in the ES cluster, so that the production cluster can remotely load the ES extended dictionary file according to the file access path.

[0092] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0093] Embodiments of this specification also provide a computer device. As shown in FIG. 5, in some embodiments of this specification, the computer device 502 may include one or more processors 504, such as one or more central processing units (CPUs) or graphics processing units (GPUs), each processing unit implementing one or more hardware threads. The computer device 502 may also include any memory 506 for storing any kind of information such as code, settings, data, etc. In one specific embodiment, a computer program on the memory 506 and executable on the processor 504, when run by the processor 504, can execute instructions of the ES extended dictionary high availability implementation method described in any of the above embodiments. Non-limitingly, for example, the memory 506 may include any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 502. In one scenario, when processor 504 executes associated instructions stored in any memory or combination of memories, computer device 502 can perform any operation of the associated instructions. Computer device 502 also includes one or more drive mechanisms 508 for interacting with any memory, such as hard disk drive mechanisms, optical disk drive mechanisms, etc.

[0094] Computer device 502 may also include an input / output interface 510 (I / O) for receiving various inputs (via input device 512) and providing various outputs (via output device 514). A specific output mechanism may include a presentation device 516 and an associated graphical user interface 518 (GUI). In other embodiments, the input / output interface 510 (I / O), input device 512, and output device 514 may be omitted, and the device may function solely as a computer device within a network. Computer device 502 may also include one or more network interfaces 520 for exchanging data with other devices via one or more communication links 522. One or more communication buses 524 couple the components described above together.

[0095] Communication link 522 can be implemented in any way, such as via a local area network, a wide area network (e.g., the Internet), a point-to-point connection, or any combination thereof. Communication link 522 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0096] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), computer-readable storage media, and computer program products according to some embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processor to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processor, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0097] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processor to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0098] These computer program instructions may also be loaded onto a computer or other programmable data processor to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable device, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0099] In a typical configuration, a computer device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0100] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0101] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by computer equipment. As defined in this specification, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0102] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of computer program products implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] The embodiments described in this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processors connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0104] It should also be understood that, in the embodiments of this specification, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0105] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0106] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0107] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for implementing high availability of an ES extended thesaurus, characterized in that, include: Receive custom ES extended dictionary files; According to the cluster architecture, the ES extended dictionary file is written to the corresponding cloud storage, and the file access path at the time of writing is recorded; the file access path is sent to the cluster configuration parameters of the production cluster in the ES cluster, so that the production cluster can remotely load the ES extended dictionary file according to the file access path.

2. The ES extended thesaurus high availability implementation method as described in claim 1, characterized in that, The step of writing the ES extended dictionary file to the corresponding cloud storage according to the cluster architecture includes: if the ES cluster is a local active-active cluster, then the ES extended dictionary file is written to a first object storage and a second object storage; the first object storage is the object storage corresponding to the production cluster, and the second object storage is the object storage corresponding to the local disaster recovery cluster; if the ES cluster is a geographically distributed multi-active cluster, then the ES extended dictionary file is written to the first object storage and the second object storage, and the ES extended dictionary file in the first object storage is asynchronously transmitted to a third object storage; the third object storage is the object storage corresponding to the geographically distributed disaster recovery cluster.

3. The ES extended thesaurus high availability implementation method as described in claim 1, characterized in that, The process of sending the file access path to the production cluster in the ES cluster includes: sending the file access path to the production cluster in the ES cluster through the middleware management service, so that the production cluster can store it in the corresponding cluster configuration parameters.

4. The ES extended thesaurus high availability implementation method as described in claim 2, characterized in that, The ES cluster remotely loads the ES extended dictionary file according to the file access path, including: if the ES cluster is a local dual-active cluster or a multi-active cluster in different locations, the ES cluster reads the ES extended dictionary file from the first object storage and the second object storage respectively according to the corresponding file access path, and takes the union of the read ES extended dictionary files.

5. The ES extended thesaurus high availability implementation method as described in claim 1, characterized in that, Before writing the ES extended thesaurus file to the corresponding cloud storage according to the cluster architecture, the method further includes: writing the ES extended thesaurus file to the database corresponding to the production cluster.

6. The ES extended thesaurus high availability implementation method as described in claim 5, characterized in that, Also includes: When an error occurs while writing the ES extended dictionary file to object storage, the corresponding ES extended dictionary file is retrieved from the database and retried. When an error occurs while reading the ES extended dictionary file from object storage, the corresponding ES extended dictionary file is read from the database.

7. A high-availability implementation device for an ES extended thesaurus, characterized in that, include: The receiving module is used to receive custom ES extended dictionary files; The storage module is used to write the ES extended dictionary file to the corresponding cloud storage according to the cluster architecture, and record the file access path corresponding to the writing process. The distribution module is used to distribute the file access path to the cluster configuration parameters of the production cluster in the ES cluster, so that the production cluster can remotely load the ES extended dictionary file according to the file access path.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the computer program is run by the processor, it executes the instructions of the method according to any one of claims 1-6.

9. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor of the computer device, it executes the instructions of the method according to any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when run by the processor of a computer device, executes instructions according to any one of claims 1-6.

Images