Method, device and equipment for repairing fragmentation of damaged fragmentation state file and medium

Abstract

The invention discloses a fragment repair method and device for a damaged fragment state file, equipment and a medium, and relates to the technical field of data storage, and the method comprises the following steps: searching a target fragment of the damaged fragment state file based on a preset fragment search command, and determining associated information corresponding to the target fragment; a target Elasticsearch node where the target fragment is located is determined according to the associated information, and the target Elasticsearch node is shut down by using a node host corresponding to the target Elasticsearch node; and after the target Elasticsearch node is shut down, the preset fragment repair toolkit is uploaded to the node host, so that the node host can repair a target fragment corresponding to the target Elasticsearch node by using the preset fragment repair toolkit. According to the method and the device, the problem that the damaged fragment of the fragment state file cannot be repaired is solved, so that the risk of data loss is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of data repair, and in particular to a method, device, equipment and medium for repairing a damaged slice status file. Background Art

[0002] Due to sudden power outages or system anomalies, some data files are damaged, resulting in the inability to write and query the shards where the damaged data files are located. Elasticsearch (a search and analysis engine) has special tools for detection and repair, but its repair is only for some scenarios. In extreme scenarios where the shard status file is damaged, it cannot be repaired and the entire shard needs to be cleared and reset.

[0003] Currently, Elasticsearch officially provides a tool command to detect whether the data of a shard under a specified index is damaged. If it is damaged, you can choose whether to discard the damaged data and repair it to restore the writing and query capabilities of the shard. This command can repair the damage of some data files in common shards, but because it needs to be scanned and detected based on the shard status file, if this file is also damaged, the shard data cannot be repaired and the entire shard data needs to be discarded, reset and generate a new shard status file. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a shard repair method, device, equipment and medium for a damaged shard status file, which can find the target shard with a damaged shard status file and determine the Elasticsearch node corresponding to the target shard. By using a preset shard repair toolkit to repair the target shard with a damaged shard status file, the problem that the shard with a damaged shard status file cannot be repaired can be solved. The specific scheme is as follows:

[0005] In a first aspect, the present application discloses a method for repairing a damaged shard status file, comprising:

[0006] Find the target shard whose shard status file is damaged based on the preset shard search command, and determine the associated information corresponding to the target shard;

[0007] Determine the target Elasticsearch node where the target shard is located according to the association information, and shut down the target Elasticsearch node using a node host corresponding to the target Elasticsearch node;

[0008] After the target Elasticsearch node is shut down, a preset shard repair toolkit is uploaded to the node host so that the node host can use the preset shard repair toolkit to repair the target shard corresponding to the target Elasticsearch node.

[0009] Optionally, searching for a target shard whose shard status file is damaged based on a preset shard search command includes:

[0010] Using the preset shard search command to search for a shard list with abnormal status, and filtering out shards in the shard list with shard status as unallocated, to obtain unallocated shards;

[0011] Obtaining a shard abnormality reason corresponding to the abnormal state of the shard list, and judging whether the unallocated shard is a shard whose shard status file is damaged according to the shard abnormality reason;

[0012] If the unallocated shard is a shard whose shard status file is damaged, the unallocated shard is determined as the target shard, and a backup operation is performed on the file under the corresponding path of the target Elasticsearch node.

[0013] Optionally, shutting down the target Elasticsearch node by using a node host corresponding to the target Elasticsearch node includes:

[0014] Start a node host corresponding to the target Elasticsearch node, and use a preset process search command through the node host to search for a target Elasticsearch process corresponding to the target Elasticsearch node;

[0015] The target Elasticsearch process found is closed using a preset process closing command to shut down the target Elasticsearch node.

[0016] Optionally, before searching for the target shard whose shard status file is damaged based on the preset shard search command, the method further includes:

[0017] Switch the current Elasticsearch user account to the target account corresponding to the preset target user; the preset target user is a non-root user;

[0018] After detecting that the target account has been successfully switched, the step of searching for the target shard with a damaged shard status file based on a preset shard search command is triggered.

[0019] Optionally, the using the preset shard repair toolkit to repair the target shard corresponding to the target Elasticsearch node includes:

[0020] Determine whether the slice status file of the target slice meets the preset file format requirement;

[0021] If the shard status file does not meet the preset file format requirements, search for an initial status file with a preset file suffix in the target directory, and scan the target directory for a target data file with the same prefix as the initial status file; the target directory is the directory where the shard status file is located;

[0022] Performing an abnormality analysis on the initial state file and the target data file, and filtering out the target state file from the initial state file based on the abnormality analysis result;

[0023] Delete the transaction log file corresponding to the target shard, generate a new target transaction log file according to the first preset standard file structure, and record the universal unique identification code of the target transaction log file;

[0024] Utilizing the file information of the target status file and the universal unique identifier of the target transaction log file, and generating a repaired shard status file corresponding to the target shard according to a second preset standard file structure, and placing the repaired shard status file in the target directory.

[0025] Optionally, performing an abnormality analysis on the initial state file and the target data file, and filtering out the target state file from the initial state file based on the abnormality analysis result, includes:

[0026] Determine from the initial state files a first initial state file which has a file anomaly and a second initial state file corresponding to the target data file which has a file anomaly;

[0027] The first initial state file and the second initial state file are removed from the initial state file to obtain the target state file.

[0028] Optionally, after repairing the target shard corresponding to the target Elasticsearch node using the preset shard repair toolkit, the method further includes:

[0029] The Elasticsearch node corresponding to the target shard is started through the node host corresponding to the target Elasticsearch node and using a preset node startup command, and all shard information corresponding to the Elasticsearch node including the repaired shard status file is loaded.

[0030] In a second aspect, the present application provides a shard repair device for a damaged shard status file, comprising:

[0031] An associated information determination module, used to search for a target shard whose shard status file is damaged based on a preset shard search command, and determine associated information corresponding to the target shard;

[0032] A node shutdown module, used to determine the target Elasticsearch node where the target shard is located according to the association information, and shut down the target Elasticsearch node using the node host corresponding to the target Elasticsearch node;

[0033] The shard repair module is used to upload a preset shard repair toolkit to the node host after the target Elasticsearch node is shut down, so that the node host can use the preset shard repair toolkit to repair the target shard corresponding to the target Elasticsearch node.

[0034] In a third aspect, the present application provides an electronic device, including:

[0035] Memory, used to store computer programs;

[0036] A processor is used to execute the computer program to implement the aforementioned shard repair method for damaged shard status files.

[0037] In the present application, firstly, the target shard whose shard status file is damaged is searched based on the preset shard search command, and the associated information corresponding to the target shard is determined, then the target Elasticsearch node where the target shard is located is determined according to the associated information, and the target Elasticsearch node is shut down by the node host corresponding to the target Elasticsearch node, and finally, after the target Elasticsearch node is shut down, the preset shard repair toolkit is uploaded to the node host, so that the node host can repair the target shard corresponding to the target Elasticsearch node by using the preset shard repair toolkit. It can be seen that the present application can obtain the target shard whose shard status file is damaged by the preset shard search command, and by determining the associated information corresponding to the target shard, the Elasticsearch node corresponding to the target shard can be determined by using the associated information, and by uploading the preset shard repair toolkit to the node host, the shard whose shard status file is damaged can be repaired by using the preset shard repair toolkit, thereby reducing the risk of data loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0039] Figure 1 A flow chart of a method for repairing a damaged shard status file disclosed in the present application;

[0040] Figure 2 A flow chart of a method for repairing a damaged shard status file disclosed in the present application;

[0041] Figure 3 A schematic diagram of a shard file list disclosed in this application;

[0042] Figure 4 A schematic diagram of a sharding state disclosed in this application;

[0043] Figure 5 A schematic diagram of association information corresponding to a shard disclosed in this application;

[0044] Figure 6 A flowchart of a specific shard repair method disclosed in this application;

[0045] Figure 7 A specific fragment repair method flow chart disclosed in this application

[0046] Figure 8 A schematic diagram of a status file list disclosed in this application;

[0047] Fig. 9 A schematic diagram of the structure of a shard repair device for a damaged shard status file disclosed in the present application;

[0048] Fig.10 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Currently, Elasticsearch has a special tool to detect and repair data files, but its repair is only for some scenarios. The extreme scenario of damaged shard status files cannot be repaired, and the entire shard needs to be cleared and reset. To this end, the present application provides a shard repair method for damaged shard status files. By finding the target shard with damaged shard status files and determining the associated information corresponding to the target shard, the associated information can be used to determine the Elasticsearch node corresponding to the target shard, and by uploading the preset shard repair toolkit to the node host, the shard with damaged shard status files can be repaired.

[0051] See also Figure 1 As shown, an embodiment of the present invention discloses a method for repairing a damaged shard status file, comprising:

[0052] Step S11: searching for a target slice whose slice status file is damaged based on a preset slice search command, and determining association information corresponding to the target slice.

[0053] The basic process of the shard repair method for damaged shard status files provided by this application is as follows Figure 2 As shown, in this embodiment, first, in the list of files with abnormal status, a preset shard search command is used to search for the target shard whose shard status file is damaged, and after the target shard is found, the corresponding associated information is determined, which may specifically include: using a preset shard search command, such as GET _cat / shards?format=json command, to search for the file with abnormal status such as Figure 3 The shard list shown in the figure is filtered out. Figure 4The shard status shown is an unallocated shard, that is, the shard state (i.e., shard status) is UNASSIGNED (unallocated) to obtain the unallocated shard; use the POST _cluster / allocation / explain{ "index": "index", "shard": 1, "primary": true} command to obtain the shard exception cause corresponding to the abnormal state of the shard list, and determine whether the unallocated shard is a shard with a damaged shard status file based on the shard exception cause; if the unallocated shard is a shard with a damaged shard status file, the unallocated shard is determined as the target shard, and the files under the corresponding path of the target Elasticsearch node are backed up; accordingly, if all the unallocated shards are not shards with damaged shard status files, it means that the target shard with a damaged shard status file does not exist in the current shard list with abnormal status files, and the repair process of the shard with a damaged shard status file is stopped. It should be noted that if Figure 5 As shown, the above-mentioned associated information includes: the number, index name and node of the abnormal status shard. It should be noted that the Elasticsearch server needs to be started using a non-root user (i.e., a non-root user), and the files it generates also have specified user permissions. It must be run using the specified file, otherwise the newly generated file cannot be read. Therefore, before searching for the target shard with damaged shard status file based on the preset shard search command, it specifically also includes: switching the current Elasticsearch user account to the target account corresponding to the preset target user; the preset target user is a non-root user; after monitoring that the target account has been successfully switched to, trigger the step of searching for the target shard with damaged shard status file based on the preset shard search command. In this embodiment, by first finding the target shard with damaged shard status file and determining the associated information, the associated information can be used to locate the target shard to repair the target shard.

[0054] Step S12: determine the target Elasticsearch node where the target shard is located according to the association information, and shut down the target Elasticsearch node using the node host corresponding to the target Elasticsearch node.

[0055] In this embodiment, in order to prevent data loss or cluster confusion in the target Elasticsearch node and avoid meaningless re-sharding operations, before uploading the shard repair toolkit to the node host corresponding to the target Elasticsearch node, it is necessary to use the information of the node where the target shard is located in the association information to determine the target Elasticsearch node where the target shard with damaged shard status file is located, and use the node host corresponding to the above target Elasticsearch node to shut down the node. This step may specifically include: starting the node host corresponding to the target Elasticsearch node, and using the preset process search command through the node host, such as the ps aux | grep elasticsearch command to find the target Elasticsearch process corresponding to the target Elasticsearch node; using the preset process shutdown command, that is, the kill -SIGTERM pid command to shut down the found target Elasticsearch process to shut down the target Elasticsearch node. In this embodiment, by shutting down the target Elasticsearch node using the node host corresponding to the target Elasticsearch node, data loss or cluster confusion in the target node is avoided, and the reliability of the shard repair process is ensured.

[0056] Step S13: after the target Elasticsearch node is shut down, uploading a preset shard repair toolkit to the node host, so that the node host can use the preset shard repair toolkit to repair the target shard corresponding to the target Elasticsearch node.

[0057] In this embodiment, the process of repairing the target shard with damaged shard status file by using the shard repair toolkit needs to be performed in the JDK (Java Development Kit, i.e., a Java development environment) environment. In the JDK environment, execute the java -jar tools.jar {index uuid} {shard number} command, and use the shard repair toolkit to repair the corresponding target shard. After the target shard is repaired, it specifically includes: starting the Elasticsearch node corresponding to the target shard by using the preset node startup command, i.e., the bin / elasticsearch-d command, through the node host corresponding to the target Elasticsearch node, and after the target Elasticsearch node is fully started, using the shard allocation command, i.e., POST _cluster / reroute{"commands":[{"allocate_primary_shard":{"index":"test","shard":0,"node":"1.es1","accept_data_loss":true}}]} to reallocate the shard, and load all shard information corresponding to the Elasticsearch node including the repaired shard status file. In this embodiment, by using the shard repair toolkit to repair the target shard, the problem that the shard with a damaged shard status file cannot be repaired is solved, thereby reducing the risk of data loss.

[0058] It can be seen that the embodiment of the present application first finds the target shard with a damaged shard status file and determines the associated information, and can use the associated information to locate the target shard to repair the target shard. By shutting down the target Elasticsearch node using the node master corresponding to the target Elasticsearch node, data loss or cluster confusion in the target node is avoided, thereby ensuring the reliability of the shard repair process. By using the shard repair toolkit to repair the target shard, the problem that the shard with a damaged shard status file cannot be repaired is solved, thereby reducing the risk of data loss.

[0059] Based on the above embodiments, the present application discloses the overall process of using the shard repair toolkit to repair the target shard whose shard status file is damaged. In order to make the shard repair method disclosed in the present application more specific, the present application will elaborate on the process of using the shard toolkit to repair the target shard. Figure 6 As shown, the embodiment of the present application discloses a specific process of repairing a target shard whose shard status file is damaged by using a shard repair toolkit, including:

[0060] Step S21: determine whether the slice status file of the target slice meets the preset file format requirements, and obtain a corresponding determination result.

[0061] In this embodiment, Figure 7 As shown, in the specific process of using the shard repair toolkit to repair the target shard whose shard status file is damaged, it is first necessary to determine whether the file header, file tail, and various file contents in the middle of the shard status file of the target shard meet the preset file format requirements. In a specific implementation, if the shard status file does not meet the preset file format requirements, the initial status file with the file suffix of the preset suffix name under the target directory is searched, and the target data file with the same prefix as the initial status file is scanned from the target directory; the target directory is the directory where the shard status file is located; in another specific implementation, if the shard status file meets the preset file format requirements, the process of using the shard toolkit to repair the shard is stopped. The above detection steps reuse the code logic executed by the official Elasticsearch-shard. In this embodiment, by performing a secondary judgment on the shard status file of the target shard, the reliability of the file status of the obtained shard status file is improved.

[0062] Step S22: performing abnormality analysis on the initial state file and the target data file based on the judgment result, and filtering out the target state file from the initial state file based on the abnormality analysis result.

[0063] In this embodiment, an abnormality analysis is performed on the initial state file and the target data file, and the target state file is screened out from the initial state file based on the abnormality analysis result. Specifically, it may include: determining a first initial state file that has a file abnormality and a second initial state file corresponding to the target data file that has a file abnormality from the initial state file; removing the first initial state file and the second initial state file from the initial state file to obtain the target state file. The above-mentioned target data file is a file with the same prefix as the initial state file. In one specific embodiment, if the initial state file itself has a file abnormality, the above-mentioned initial state file that has a file abnormality is determined as the first initial state file and is removed from the initial state file; in another specific embodiment, if the initial state file itself does not have a file abnormality, but the target data file with the same prefix as the above-mentioned initial state file that does not have a file abnormality has an abnormality, it is determined as the second initial state file and is removed from the initial state file. Figure 8As shown, the above-mentioned initial state files are files with .si as the suffix, and these files are the status files of the segment data. When the shards are loaded, the shard status files are read. The shard status files contain information about the status of each segment, and these segment status files are needed to construct the shard status files; the target data file is a file with the same prefix as the file with .si as the suffix. In this implementation, the reliability of the shard repair process is ensured by screening out the first initial state file that has file anomalies itself and the second initial state file corresponding to the target data file with file anomalies.

[0064] Step S23: Delete the transaction log file corresponding to the target shard, generate a new target transaction log file according to the first preset standard file structure, and record the universal unique identification code of the target transaction log file.

[0065] The transaction log file in this embodiment is translog (a type of log file), and the universally unique identifier is UUID (Universally Unique Identifier). To ensure that the shard status is consistent with the UUID of translog, the translog corresponding to the target shard is deleted, and a new translog is regenerated. This record records the UUID corresponding to the new translog. This step refers to the translog cleanup step in Elasticsearch-shard, and generates a new translog according to the standard log file structure. In this embodiment, by deleting the existing transaction log file corresponding to the target shard and generating a new transaction log file, the shard status is guaranteed to be consistent with the universally unique identifier of the transaction log file, thereby ensuring the reliability of the shard repair process.

[0066] Step S24, using the file information of the target status file and the universal unique identifier of the target transaction log file, and generating a repaired shard status file corresponding to the target shard according to a second preset standard file structure, and placing the repaired shard status file in the target directory.

[0067] In this embodiment, a shard repair toolkit is used to generate a repaired shard status file according to a second preset standard file structure, i.e., a standard shard status file structure, using the file information of the target status file and the universal unique identifier of the target transaction log file, and the repaired shard status file is placed in the target directory. By using the shard repair toolkit to repair the target shard whose status file is damaged, the problem that the shard whose status file is damaged cannot be repaired is solved, thereby reducing the risk of data loss.

[0068] It can be seen that the present application improves the reliability of the file status of the obtained shard status file by performing a secondary judgment on the shard status file of the target shard, and ensures the reliability of the screened target status file by screening out the first initial status file which has file anomalies itself and the second initial status file corresponding to the target data file which has file anomalies, thereby ensuring the reliability of the shard repair process, and solves the problem that the shard with a damaged shard status file cannot be repaired by using a preset shard repair toolkit, thereby reducing the risk of data loss.

[0069] See also Fig. 9 As shown, an embodiment of the present invention discloses a shard repair device for a damaged shard status file, comprising:

[0070] The associated information determination module 11 is used to search for the target slice whose slice status file is damaged based on the preset slice search command, and determine the associated information corresponding to the target slice;

[0071] A node shutdown module 12 is used to determine the target Elasticsearch node where the target shard is located according to the association information, and shut down the target Elasticsearch node using the node host corresponding to the target Elasticsearch node;

[0072] The shard repair module 13 is used to upload a preset shard repair toolkit to the node host after the target Elasticsearch node is shut down, so that the node host can use the preset shard repair toolkit to repair the target shard corresponding to the target Elasticsearch node.

[0073] It can be seen that the embodiment of the present application first finds the target shard whose shard status file is damaged, and determines the corresponding associated information of the target shard. The associated information can be used to locate the target shard to repair the target shard. By shutting down the target Elasticsearch node using the node master corresponding to the target Elasticsearch node, data loss or cluster confusion in the target node is avoided, thereby ensuring the reliability of the shard repair process. By repairing the target shard using the shard repair toolkit, the problem that the shard with a damaged shard status file cannot be repaired is solved, thereby reducing the risk of data loss.

[0074] In some specific embodiments, the association information determination module 11 specifically includes:

[0075] A shard screening unit, configured to search a shard list with abnormal status by using the preset shard search command, and screen out shards in the shard list with shard status as unallocated, so as to obtain unallocated shards;

[0076] A fragment judgment unit, used for obtaining a fragment abnormality cause corresponding to the abnormal state of the fragment list, and judging whether the unallocated fragment is a fragment whose fragment status file is damaged according to the fragment abnormality cause;

[0077] The file backup unit is used to determine the unallocated shard as the target shard if the unallocated shard is a shard whose shard status file is damaged, and to perform a backup operation on the file under the path corresponding to the target Elasticsearch node.

[0078] In some specific embodiments, the node shutdown module 12 specifically includes:

[0079] A process search unit, used to start a node host corresponding to the target Elasticsearch node, and search for a target Elasticsearch process corresponding to the target Elasticsearch node through the node host using a preset process search command;

[0080] The node shutdown unit is used to shut down the target Elasticsearch process found by using a preset process shutdown command to shut down the target Elasticsearch node.

[0081] In some specific embodiments, the association information determination module 11 further includes:

[0082] An account switching unit, used to switch the current Elasticsearch user account to a target account corresponding to a preset target user; the preset target user is a non-root user;

[0083] The step triggering module is used to trigger the step of searching for the target shard whose shard status file is damaged based on the preset shard search command after detecting that the target account has been successfully switched.

[0084] In some specific embodiments, the fragment repair module 13 specifically includes:

[0085] A format determination unit, configured to determine whether the slice status file of the target slice meets a preset file format requirement;

[0086] A file scanning unit is used for searching for an initial state file with a preset file suffix in a target directory if the slice state file does not meet the preset file format requirement, and scanning the target data file with the same prefix as the initial state file from the target directory; the target directory is the directory where the slice state file is located;

[0087] A file screening submodule, used for performing an abnormality analysis on the initial state file and the target data file, and screening out the target state file from the initial state file based on the abnormality analysis result;

[0088] An identification code recording unit, used to delete the transaction log file corresponding to the target shard, generate a new target transaction log file according to a first preset standard file structure, and record a universal unique identification code of the target transaction log file;

[0089] A file generation unit is used to utilize the file information of the target status file and the universal unique identifier of the target transaction log file, and to generate a repaired shard status file corresponding to the target shard according to a second preset standard file structure, and to place the repaired shard status file in the target directory.

[0090] In some specific embodiments, the file screening submodule specifically includes:

[0091] A file determining unit, configured to determine, from the initial state files, a first initial state file which has a file anomaly and a second initial state file corresponding to the target data file which has a file anomaly;

[0092] The file elimination unit is used to eliminate the first initial state file and the second initial state file from the initial state file to obtain the target state file.

[0093] In some specific embodiments, the fragment repair module 13 further includes:

[0094] A node startup unit is used to start the Elasticsearch node corresponding to the target shard through the node host corresponding to the target Elasticsearch node and using a preset node startup command, and load all shard information corresponding to the Elasticsearch node including the repaired shard status file.

[0095] Furthermore, the present application also discloses an electronic device. Fig.10 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0096] Fig.10A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the method for repairing a damaged shard status file disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0097] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0098] In addition, the memory 22 as a carrier for storing resources may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0099] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the fragment repair method for the damaged fragment status file executed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0100] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned disclosed method for repairing a damaged shard status file is implemented. The specific steps of the method can be referred to the corresponding contents disclosed in the aforementioned embodiment, and will not be repeated here.

[0101] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0102] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0103] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0104] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0105] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for repairing a damaged shard status file, characterized in that: include: Find the target shard whose shard status file is damaged based on the preset shard search command, and determine the associated information corresponding to the target shard; Determine the target Elasticsearch node where the target shard is located according to the association information, and shut down the target Elasticsearch node using a node host corresponding to the target Elasticsearch node; After the target Elasticsearch node is shut down, a preset shard repair toolkit is uploaded to the node host so that the node host can use the preset shard repair toolkit to repair the target shard corresponding to the target Elasticsearch node.

2. The method for repairing a damaged shard status file according to claim 1, characterized in that: The step of searching for a target shard with a damaged shard status file based on a preset shard search command includes: Using the preset shard search command to search for a shard list with abnormal status, and filtering out shards in the shard list with shard status as unallocated, to obtain unallocated shards; Obtaining a shard abnormality reason corresponding to the abnormal state of the shard list, and judging whether the unallocated shard is a shard whose shard status file is damaged according to the shard abnormality reason; If the unallocated shard is a shard whose shard status file is damaged, the unallocated shard is determined as the target shard, and a backup operation is performed on the file under the corresponding path of the target Elasticsearch node.

3. The method for repairing a damaged shard status file according to claim 1, characterized in that: The shutting down the target Elasticsearch node by using the node host corresponding to the target Elasticsearch node includes: Start the node host corresponding to the target Elasticsearch node, and use the preset process search command through the node host to search for the target Elasticsearch process corresponding to the target Elasticsearch node; The target Elasticsearch process found is closed using a preset process closing command to shut down the target Elasticsearch node.

4. The method for repairing a damaged shard status file according to claim 1, characterized in that: Before searching for the target shard with damaged shard status file based on the preset shard search command, the method further includes: Switch the current Elasticsearch user account to the target account corresponding to the preset target user; the preset target user is a non-root user; After detecting that the target account has been successfully switched, the step of searching for the target shard with a damaged shard status file based on a preset shard search command is triggered.

5. The method for repairing a damaged shard status file according to any one of claims 1 to 4, characterized in that: The using the preset shard repair toolkit to repair the target shard corresponding to the target Elasticsearch node includes: Determine whether the slice status file of the target slice meets the preset file format requirement; If the shard status file does not meet the preset file format requirements, search for an initial status file with a preset file suffix in the target directory, and scan the target directory for a target data file with the same prefix as the initial status file; the target directory is the directory where the shard status file is located; Performing an abnormality analysis on the initial state file and the target data file, and filtering out the target state file from the initial state file based on the abnormality analysis result; Delete the transaction log file corresponding to the target shard, generate a new target transaction log file according to the first preset standard file structure, and record the universal unique identification code of the target transaction log file; Utilizing the file information of the target status file and the universal unique identifier of the target transaction log file, and according to a second preset standard file structure, a repaired shard status file corresponding to the target shard is generated, and the repaired shard status file is placed in the target directory.

6. The method for repairing a damaged shard status file according to claim 5, characterized in that: The performing abnormality analysis on the initial state file and the target data file, and filtering out the target state file from the initial state file based on the abnormality analysis result, comprises: Determine from the initial state files a first initial state file which has a file anomaly and a second initial state file corresponding to the target data file which has a file anomaly; The first initial state file and the second initial state file are removed from the initial state file to obtain the target state file.

7. The method for repairing a damaged shard status file according to claim 5, characterized in that: After repairing the target shard corresponding to the target Elasticsearch node using the preset shard repair toolkit, the method further includes: The Elasticsearch node corresponding to the target shard is started through the node host corresponding to the target Elasticsearch node and using a preset node startup command, and all shard information corresponding to the Elasticsearch node including the repaired shard status file is loaded.

8. A fragment repair device for a damaged fragment status file, characterized in that: include: An associated information determination module, used to search for a target shard whose shard status file is damaged based on a preset shard search command, and determine associated information corresponding to the target shard; A node shutdown module, used to determine the target Elasticsearch node where the target shard is located according to the association information, and shut down the target Elasticsearch node using the node host corresponding to the target Elasticsearch node; The shard repair module is used to upload a preset shard repair toolkit to the node host after the target Elasticsearch node is shut down, so that the node host can use the preset shard repair toolkit to repair the target shard corresponding to the target Elasticsearch node.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the method for repairing a damaged shard status file as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements a method for repairing a damaged shard status file as described in any one of claims 1 to 7.

Images