Elasticsearch version migration method, device and storage medium

By creating the target version of XML files in Elasticsearch and calling the Reindex API, the problem of Mapping not supporting delete and modifying fields is solved, and field type upgrades and data migration is realized, which meets business needs and improves data storage and retrieval efficiency.

CN113138792BActive Publication Date: 2024-11-12NINGBO NINGSHU SAFETY TECHNOLOGY CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202110442896.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-04-23
Publication Date
2024-11-12
Estimated Expiration
2041-04-23

AI Technical Summary

Technical Problem

Elasticsearch's Mapping does not support deleting and modifying fields, and cannot meet business needs to be fast iteration and field type upgrades.

Method used

By creating the target version of the XML file, calling the Mapping API to obtain the field attribute information of the current production environment, matching and upgrading the field type, and using the Reindex API to migrate the data to the target index to achieve the upgrade of the field attribute information.

Benefits of technology

It has realized the upgrade of field types to meet user needs, overcome the limitations of Mapping in Elasticsearch that does not support deleting and modifying fields, and improves data storage and retrieval efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113138792B_ABST
    Figure CN113138792B_ABST
Patent Text Reader

Abstract

The present application discloses an Elasticsearch version migration method, including: when the project update iteration starts, create an XML file of the target version; call the Mapping API to obtain the field attribute information of the current production environment, and match the field attribute information in the XML file of the target version with the field attribute information of the current production environment; if the match is not successful, create a target index, define the field type of the target index as the field attribute information in the XML file of the target version; call the Reindex API to migrate the data of the old index corresponding to the current production environment to the target index. This method can realize the upgrade of field attribute information and can better meet user needs. The present application also provides an Elasticsearch version migration device, an electronic device and a computer-readable storage medium, which have the above-mentioned beneficial effects.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of Elasticsearch, and in particular to an Elasticsearch version migration method, device, electronic device, and computer-readable storage medium. Background Art

[0002] Elasticsearch is an open source search engine built on a full-text search engine library Apache Lucene TM In the field of big data technology, Elasticsearch's distributed retrieval function facilitates multi-dimensional data presentation, analysis, and prediction, which helps to maximize the value of data. Although retrieval is the most commonly used function for Elasticsearch, because a large amount of data needs to be stored in Elasticsearch, the efficiency of retrieval and the rational use of storage space often depend on the design of its field type Mapping. A reasonable Mapping design can greatly improve the rational use of data in Elasticsearch storage space and retrieval efficiency.

[0003] In the related technology, Mapping does not support deleting and modifying fields. When a business function needs to introduce or delete key fields, it is not possible to modify or delete the definition of existing fields, which cannot meet business needs. Summary of the invention

[0004] The purpose of this application is to provide an Elasticsearch version migration method that can achieve field type upgrades and better meet user needs. The specific solution is as follows:

[0005] In a first aspect, the present application discloses an Elasticsearch version migration method, comprising:

[0006] When a project update iteration starts, create an XML file for the target version;

[0007] Calling the Mapping API to obtain the field attribute information of the current production environment, and matching the field attribute information in the target version's XML file with the field attribute information of the current production environment;

[0008] If the match fails, a target index is created, and the field type of the target index is defined as the field attribute information in the XML file of the target version;

[0009] Call the Reindex API to migrate the data of the old index corresponding to the current production environment to the target index.

[0010] Optionally, before calling the Mapping API, also include:

[0011] Obtain a field definition record table in a MySQL database; the field definition record table stores various XML files under the project;

[0012] Matching the target version's XML file with each XML file under the item in the field definition record table;

[0013] If the match is not successful, the step of calling the Mapping API is executed.

[0014] Optionally, create an XML file for the target version, including:

[0015] According to the field upgrade information, field attribute information corresponding to the field upgrade information is added to the target XML file under the project to generate the XML file of the target version.

[0016] Optionally, matching the field attribute information in the XML file of the target version with the field attribute information of the current production environment includes:

[0017] Parse the schema_num in the field definition record table to obtain each XML file under the project;

[0018] Determine whether each XML file under the project includes the XML file of the target version.

[0019] Optionally, before matching the field attribute information in the target version's XML file with the field attribute information in the current production environment, the method further includes:

[0020] According to the schema_num in the field definition record table, determine whether the target version of the XML file is a content modification of the XML file under the project; the schema_num is a string column stored after reading the XML file of the project and performing base64 encoding;

[0021] If not, the step of matching the field attribute information of the target version of the XML file with the field attribute information of the current production environment is performed.

[0022] Optionally, after migrating the data of the old index corresponding to the current production environment to the target index, the method further includes:

[0023] Add a record of the target version of the XML file in the field definition record table.

[0024] Optionally, after migrating the data of the old index corresponding to the current production environment to the target index, the method further includes:

[0025] An alias of the target index is set so that a user can search for the XML file of the target version according to the alias.

[0026] In a second aspect, the present application discloses an Elasticsearch version migration device, comprising:

[0027] The first creation module is used to create an XML file of the target version when the project update iteration is started;

[0028] A matching module, used to call a Mapping API to obtain field attribute information of the current production environment, and match the field attribute information in the XML file of the target version with the field attribute information of the current production environment;

[0029] The second creation module is used to create a target index if the match fails, and define the field type of the target index as the field attribute information in the XML file of the target version;

[0030] The migration module is used to call the Reindex API to migrate the data of the old index corresponding to the current production environment to the target index.

[0031] In a third aspect, the present application discloses an electronic device, comprising:

[0032] Memory for storing computer programs;

[0033] A processor is used to implement the steps of the above-mentioned Elasticsearch version migration method when executing the computer program.

[0034] In a fourth aspect, the present application discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the Elasticsearch version migration method as described above are implemented.

[0035] The present application provides an Elasticsearch version migration method, including: when a project update iteration is started, creating an XML file of a target version; calling a Mapping API to obtain field attribute information of a current production environment, and matching the field attribute information in the XML file of the target version with the field attribute information of the current production environment; if the match is not successful, creating a target index, defining the field type of the target index as the field attribute information in the XML file of the target version; calling a Reindex API to migrate data corresponding to an old index of the current production environment to the target index.

[0036] It can be seen that the present application realizes version migration by creating an XML file of the target version, creating a target index, defining the field type of the target index as the field attribute information of the XML file of the target version, and then calling the Reindex API to migrate the data of the old index corresponding to the current production environment to the target index; that is, the present application can realize the upgrade of the field type by applying the field attribute information in the XML file of the target version to the newly created target index, and then use the Reindex API to migrate the data of the old index to the target index, avoiding the defect that the Mapping in Elasticsearch in the related technology does not support the deletion and modification of field attributes and cannot meet the project requirements. The present application realizes the upgrade of field attribute information and can better meet user needs. The present application also provides an Elasticsearch version migration device, an electronic device and a computer-readable storage medium, which have the above-mentioned beneficial effects and are not repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0038] Figure 1 A flowchart of an Elasticsearch version migration method provided in an embodiment of the present application;

[0039] Figure 2 A flowchart of a specific implementation example of Elasticsearch version migration provided in an embodiment of the present application;

[0040] Figure 3 A schematic diagram of the structure of an Elasticsearch version migration device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] When using Elasticsearch to introduce it into the project, a reasonable Mapping design can greatly improve the reasonable use of data in Elasticsearch storage space and retrieval efficiency. However, Elasticsearch has certain limitations in modifying Mapping. A document in an index does not support direct deletion and modification of fields. This will result in that when the business function needs to introduce new key fields, developers cannot modify and delete the existing field definitions. Therefore, although Elasticsearch is a popular enterprise-level search engine that can solve many of people's requirements for search. Among them, Elasticsearch field types, also known as Mappings, have types for each document in the index. Each type has its own mapping, or schema definition. The mapping defines the domains in the type, the data types of each domain, and how Elasticsearch handles these domains. Mappings are also used to configure metadata related to types. Elasticsearch can be understood as an RDBMS (relational database, such as MySQL), then index is equivalent to a database instance, so Mapping can be understood as the structure of the table and related settings. If there are business scenarios that require rapid iteration and have modified many field definitions during business development, it will cause certain difficulties. Normally, when data is written to Elasticsearch, the default Mapping field type is automatically identified and matched by the system based on the imported data. The main field types are string, integer, date, long, byte, etc. Often in business, developers recommend custom Mapping, i.e. static Mapping. For example, the string type is divided into two types by default: text and keyword. However, if word segmentation and retrieval are not required in the business, and only exact matching is required, just set it to keyword. Therefore, it can be known that Mapping is mainly used to define which string fields should be defined as full-text search fields; which fields contain numbers, dates or geographic locations; define the format of date values ​​(timestamp or date type, etc.); and control the custom rules for mapping dynamically added fields. Therefore, developers need to choose the appropriate type according to the business, which is conducive to saving space and improving retrieval accuracy.

[0043] Based on the above technical problems, this embodiment provides an Elasticsearch version migration method, which realizes the upgrade of field attribute information and can better meet user needs. For details, please refer to Figure 1 , Figure 1 A flowchart of an Elasticsearch version migration method provided in an embodiment of the present application specifically includes:

[0044] S101. When a project update iteration is started, an XML file of the target version is created.

[0045] In this embodiment, when the project update iteration starts, that is, when the developer wants to modify or add the Elasticsearch field type, namely Mapping, an XML file (Extensible Markup Language) of the target version is created. The XML file stores the field attribute information of the version to be upgraded. It is understandable that the XML file can clearly display the field definition of each index. In this embodiment, there is no specific limitation on the target version, which can be set according to the actual situation. For example, if the current version is V1.1, the target version can be V1.2. In this embodiment, there is no limitation on the file name format of the XML file. For example, it can be V_version number__version description.xml, which can be used as a unique identifier for version comparison. It is understandable that the file format of the XML file can be the root node. <indexs>, the secondary node <index name="xxxx">, sets the index name of the secondary node name, and the tertiary node <fieldkey="field name" name="field description" type="field type" agg="whether to perform an aggregation query">. If necessary, other field attribute information can also be set.

[0046] This embodiment does not limit the specific process of creating the XML file of the target version. In a specific embodiment, creating the XML file of the target version may include:

[0047] According to the field upgrade information, add the field attribute information corresponding to the field upgrade information to the target XML file under the project to generate the XML file of the target version.

[0048] This embodiment does not limit the specific content of the field upgrade information. It can be the field attribute information to be upgraded, or the instruction corresponding to the field attribute information to be upgraded, which can be set according to the actual situation, as long as the field attribute information corresponding to the field upgrade information can be determined. It can be understood that the target XML file in this embodiment can be any XML file that the developer wants to upgrade the field type in the current production environment. That is, in this embodiment, by adding the field attribute information corresponding to the field upgrade information to the target XML file under the project, the XML file of the target version can be generated.

[0049] S102. Call the Mapping API to obtain the field attribute information of the current production environment, and match the field attribute information in the XML file of the target version with the field attribute information of the current production environment.

[0050] After the XML file of the target version is created, this embodiment uses the Mapping API to obtain the field attribute information of the current production environment. This embodiment does not limit the specific content of the field attribute information of the current production environment, which can be determined according to the actual situation. Matching the obtained field attribute information of the current production environment with the field attribute information in the created XML file of the target version can be understood as a process of judging whether the field attribute information in the current production environment contains the field attribute information in the XML file of the target version. In this embodiment, the specific matching process is not limited.

[0051] In a specific embodiment, in order to manage the Mapping field attribute information, before calling the Mapping API in this embodiment, it may further include:

[0052] Obtain the field definition record table in the MySQL database; the field definition record table stores each XML file under the project;

[0053] Match the target version's XML file with each XML file under the project in the field definition record table;

[0054] If the match is not successful, execute the step of calling the Mapping API.

[0055] In this embodiment, the MySQL database stores a field definition record table, which is used to store each XML file under the project, that is, in this embodiment, the MySQL database is used to manage the Elasticsearch field attribute information. Specifically, each XML file under the project is stored in the field definition record table in the form of a record, that is, one version of the XML file is a record. This embodiment does not limit the specific form of the field definition record table storage. For example, it can be id in the field definition record table: the primary key of the current table; version: the version number of the corresponding XML file; description: the version description of the corresponding XML file; schema_num: the string column stored after reading the content in the XML file and performing base64 encoding, which is used by developers to compare whether the current XML file has been modified. This embodiment does not limit the specific creation process of the field definition record table. For example, when the project is initialized and started, the directory where the XML file is located is first read, all versions of the XML files in the current directory are read, and then the MySQL database is connected to create a field definition record table. One version of the XML file is a record. That is to say, all XML files can be directly read in the project initialization startup stage, newly created and stored in the field definition record table.

[0056] After obtaining the field definition record table, this embodiment matches the target version of the XML file with each XML file under the project in the field definition record table. When the match is not successful, the step of calling the Mapping API is executed. It can be understood that in this embodiment, the target version of the XML file is matched with each XML file in the field definition record table, that is, it is checked whether each XML file in the field definition record table contains the target version of the XML file. When the target version of the XML file is not contained, that is, the match is not successful, the step of calling the Mapping API is executed. In this embodiment, the field definition record table of the MySQL database is used to manage the Mapping field attribute information in Elasticsearch, which is convenient for managing and viewing the field attribute information to achieve the purpose of field type upgrade.

[0057] This embodiment does not limit the specific process of matching the field attribute information of the target version of the XML file with the field attribute information of the current production environment. In a specific embodiment, matching the field attribute information in the target version of the XML file with the field attribute information of the current production environment may include:

[0058] Parse the schema_num in the field definition record table to obtain the XML files under the project;

[0059] Determine whether each XML file under the project contains the target version of the XML file.

[0060] That is, in this embodiment, the field definition record table is used to manage the Mapping field attribute information in Elasticsearch, and the schema_num in the field definition record table is parsed, where schema_num is a string column stored after reading the content in the XML file and performing base64 encoding. Then, each XML file under the project is obtained, and then it is determined whether each XML file under the project contains the target version of the XML file. When the target version of the XML file is contained, the match is successful; when the target version of the XML file is not contained, the match is unsuccessful.

[0061] It is understandable that during the project update and iteration startup phase, if the developer makes changes to the original version of the XML file, it is necessary to compare the schema_num in the field definition record table. If they are the same, the next step is performed. Otherwise, the project startup is interrupted and an error of inconsistent versions is displayed. In a specific embodiment, before matching the field attribute information in the target version of the XML file with the field attribute information in the current production environment, it may also include:

[0062] According to the schema_num in the field definition record table, determine whether the target version of the XML file is a content modification of the XML file under the project; schema_num is a string column stored after reading the XML file of the project and performing base64 encoding;

[0063] If not, the step of matching the field attribute information in the target version's XML file with the field attribute information in the current production environment is performed.

[0064] That is, in this embodiment, before matching, it is checked whether the target version of the XML file is a content modification of the XML file under the project. If it is not a modification of the original content, the step of matching the field attribute information of the target version of the XML file with the field attribute information of the current production environment is performed. This embodiment does not perform subsequent operations that are modifications to the original content, such as terminal project startup.

[0065] S103: If the match fails, create a target index, and define the field type of the target index as the field attribute information in the XML file of the target version.

[0066] In this embodiment, when the field attribute information of the target version of the XML file fails to match the field attribute information of the current production environment, a target index is created, and the field type of the target index is defined as the field attribute information in the target version of the XML file. Through this operation interface, the field type information to be upgraded is the field attribute information in the target version of the XML file, that is, the upgrade of the field type mapping is realized. It can be understood that the target index is used to store the data generated under the project.

[0067] S104: Call the Reindex API to migrate the data of the old index corresponding to the current production environment to the target index.

[0068] It is understandable that the Reindex API of Elasticsearch is used to copy documents in the index from the source to the target. In this embodiment, by calling the Reindex API, the data corresponding to the old index of the current production environment can be migrated to the target index. It is understandable that the old index in this embodiment is used to store the data generated under the project of the current production environment. At this time, the target index applies the XML file of the target version, and the data corresponding to the old index of the current production environment is migrated to the target index, that is, the Elasticsearch version migration is realized.

[0069] This embodiment does not limit the specific operations after migrating the data of the old index corresponding to the current production environment to the target index. For example, deleting the data of the old index can reduce the memory usage. In a specific embodiment, in order to update the field definition record table in time and realize the management of Mapping, this embodiment can also include the following after migrating the data of the old index corresponding to the current production environment to the target index:

[0070] Add a record of the target version of the XML file to the field definition record table.

[0071] After migrating the data of the old index corresponding to the current production environment to the target index, this embodiment also updates the field definition record table, that is, adds the target version of the XML file as a new record to the field definition record table. It can timely update the field definition record table and realize the management of Mapping.

[0072] In another specific embodiment, in order to improve flexibility and query efficiency, after migrating the data corresponding to the old index of the current production environment to the target index, the following steps may also be included:

[0073] Set the target index alias so that users can find the target version's XML file based on the alias.

[0074] That is, by setting the alias of the target index, this embodiment allows users to quickly find the target version of the XML file according to the alias. It can be understood that the alias of the index is like a shortcut or soft link, which can point to one or more indexes, and can also be used by any API that requires an index name. Aliases can bring great flexibility. For example, in a running cluster, you can seamlessly switch from one index to another, group multiple indexes (such as last_three_months), and create a view for a subset of the index. That is, by creating an alias for the target index, this embodiment can improve flexibility and query efficiency.

[0075] Based on the above technical solution, this embodiment can upgrade the field type by applying the field attribute information in the target version's XML file to the newly created target index, and then use the Reindex API to migrate the data of the old index to the target index, so as to upgrade the field attribute information and better meet user needs.

[0076] The following is a specific implementation example of Elasticsearch version migration. Figure 2 This is a flowchart of a specific implementation of the Elasticsearch version migration given in this embodiment.

[0077] A. Create a new XML file, that is, create an XML file of the target version, named V_1.2__addAssetType.xml;

[0078] B. Create a field definition record table in the MySQL database based on the current version of the XML file;

[0079] C. Version comparison. If the current production environment version is "aggregation":{"type":"keyword"}, and the type in the upgrade file version is inconsistent with long, then reindex upgrade is performed;

[0080] D. Reindex stage: First, create a new index, namely the target index access-index_v1, migrate the data of the old index access-index to access-index_v1, set an alias for the new index, and delete the old index.

[0081] E. Get the result corresponding to the new index.

[0082]

[0083] Based on the above embodiments, this embodiment manages the Elasticsearch field type mapping by creating a field definition record table in the MySQL database, and can perform upgrades in different directions according to the version; and when you want to modify and upgrade the field definition, you do not need to manually operate the Elasticsearch Reindex, you can directly create an XML file for the upgrade function, which overcomes the defect that the field type in each index of Elasticsearch cannot be flexibly modified or deleted when performing business iterations.

[0084] An Elasticsearch version migration device provided in an embodiment of the present application is introduced below. The Elasticsearch version migration device described below and the Elasticsearch version migration method described above can be referred to each other. The relevant modules are all set in, and reference is made to Figure 3 , Figure 3 A schematic diagram of the structure of an Elasticsearch version migration device provided in an embodiment of the present application includes:

[0085] In some specific embodiments, it specifically includes:

[0086] The first creation module 301 is used to create an XML file of a target version when a project update iteration is started;

[0087] The matching module 302 is used to call the Mapping API to obtain the field attribute information of the current production environment, and match the field attribute information in the target version of the XML file with the field attribute information of the current production environment;

[0088] The second creation module 303 is used to create a target index if the match fails, and define the field type of the target index as the field attribute information in the XML file of the target version;

[0089] The migration module 304 is used to call the Reindex API to migrate the data corresponding to the old index in the current production environment to the target index.

[0090] In some specific embodiments, it also includes:

[0091] The acquisition module is used to obtain the field definition record table in the MySQL database; the field definition record table stores various XML files under the project;

[0092] The first matching module is used to match the XML file of the target version with each XML file under the item in the field definition record table.

[0093] In some specific embodiments, the first creation module 301 includes:

[0094] The generation unit is used to add field attribute information corresponding to the field upgrade information in the target XML file under the project according to the field upgrade information, and generate an XML file of the target version.

[0095] In some specific embodiments, the matching module 302 includes:

[0096] The parsing unit is used to parse the schema_num in the field definition record table to obtain the various XML files under the project;

[0097] The judging unit is used to judge whether each XML file under the project contains an XML file of a target version.

[0098] In some specific embodiments, it also includes:

[0099] The judgment module is used to judge whether the target version of the XML file is a content modification of the XML file under the project according to the schema_num in the field definition record table; schema_num is a string column stored after reading the XML file of the project and performing base64 encoding.

[0100] In some specific embodiments, it also includes:

[0101] Add a module to add records of the target version of the XML file in the field definition record table.

[0102] In some specific embodiments, it also includes:

[0103] The setting module is used to set the alias of the target index so that the user can find the XML file of the target version according to the alias.

[0104] Since the embodiments of the Elasticsearch version migration device part correspond to the embodiments of the Elasticsearch version migration method part, please refer to the description of the embodiments of the Elasticsearch version migration method part for the embodiments of the Elasticsearch version migration device part, which will not be repeated here.

[0105] An electronic device provided in an embodiment of the present application is introduced below. The electronic device described below and the Elasticsearch version migration method described above can refer to each other.

[0106] The present application discloses an electronic device, comprising:

[0107] Memory for storing computer programs;

[0108] A processor is used to implement the steps of the above-mentioned Elasticsearch version migration method when executing a computer program.

[0109] Since the embodiments of the electronic device part correspond to the embodiments of the Elasticsearch version migration method part, please refer to the description of the embodiments of the Elasticsearch version migration method part for the embodiments of the electronic device part, which will not be repeated here.

[0110] A computer-readable storage medium provided in an embodiment of the present application is introduced below. The computer-readable storage medium described below and the Elasticsearch version migration method described above can be referenced to each other.

[0111] The present application discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the Elasticsearch version migration method as described above are implemented.

[0112] Since the embodiments of the computer-readable storage medium part correspond to the embodiments of the Elasticsearch version migration method part, please refer to the description of the embodiments of the Elasticsearch version migration method part for the embodiments of the computer-readable storage medium part, which will not be repeated here.

[0113] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0114] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0115] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0116] The above is a detailed introduction to an Elasticsearch version migration method, device, electronic device and computer-readable storage medium provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.< / indexs>

Claims

1. An Elasticsearch version migration method, characterized in that: include: When a project update iteration starts, create an XML file for the target version; Calling the Mapping API to obtain the field attribute information of the current production environment, and matching the field attribute information in the target version's XML file with the field attribute information of the current production environment; If the match fails, a target index is created, and the field type of the target index is defined as the field attribute information in the XML file of the target version; Call the Reindex API to migrate the data of the old index corresponding to the current production environment to the target index; Among them, before calling the Mapping API, it also includes: obtaining a field definition record table in a MySQL database; the field definition record table stores various XML files under the project; matching the XML file of the target version with the various XML files under the project in the field definition record table; if the match is not successful, executing the step of calling the Mapping API.

2. The Elasticsearch version migration method according to claim 1, characterized in that: The XML file for creating the target version includes: According to the field upgrade information, field attribute information corresponding to the field upgrade information is added to the target XML file under the project to generate the XML file of the target version.

3. The Elasticsearch version migration method according to claim 1, characterized in that: The matching of the field attribute information in the XML file of the target version with the field attribute information of the current production environment includes: Parse the schema_num in the field definition record table to obtain each XML file under the project; Determine whether each XML file under the project includes the XML file of the target version.

4. The Elasticsearch version migration method according to claim 1, characterized in that: Before matching the field attribute information in the XML file of the target version with the field attribute information of the current production environment, the method further includes: According to the schema_num in the field definition record table, determine whether the target version of the XML file is a content modification of the XML file under the project; the schema_num is a string column stored after reading the XML file of the project and performing base64 encoding; If not, the step of matching the field attribute information in the target version's XML file with the field attribute information in the current production environment is performed.

5. The Elasticsearch version migration method according to claim 1, characterized in that: After migrating the data of the old index corresponding to the current production environment to the target index, the method further includes: Add a record of the target version of the XML file in the field definition record table.

6. The Elasticsearch version migration method according to claim 1, characterized in that: After migrating the data of the old index corresponding to the current production environment to the target index, the method further includes: An alias of the target index is set so that a user can search for the XML file of the target version according to the alias.

7. An Elasticsearch version migration device, characterized in that: include: The first creation module is used to create an XML file of the target version when the project update iteration is started; A matching module, used to call a Mapping API to obtain field attribute information of the current production environment, and match the field attribute information in the XML file of the target version with the field attribute information of the current production environment; The second creation module is used to create a target index if the match fails, and define the field type of the target index as the field attribute information in the XML file of the target version; A migration module, used to call the Reindex API to migrate the data of the old index corresponding to the current production environment to the target index; The device is also used to obtain a field definition record table in a MySQL database; the field definition record table stores various XML files under the project; the XML file of the target version is matched with the various XML files under the project in the field definition record table; if the match is not successful, the step of calling the Mapping API is executed.

8. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the Elasticsearch version migration method as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the Elasticsearch version migration method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Data migration system based on software-as-a-service (SaaS)

    CN103176988A

  • Distributed search system building method and device based on Elasticsearch

    CN111400362A

  • Data migration method and device, storage medium and electronic equipment

    CN112286905A

  • Index changing method and device

    CN112507187A