Method and device for updating disk information of data node and electronic equipment
By deploying extended services and native services in HDFS data nodes, disk information comparison and updates are solved, the problem of restarting the data node's disk update is reduced, and real-time update of disk information is achieved.
Patent Information
- Application Number
- CN202411299654.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-12-24
AI Technical Summary
In the HDFS distributed file system, the data node needs to be restarted after the disk is updated, resulting in temporary service interruption, which has a great impact.
Deploy extended services and native services in the data node, receive client disk information update requests through the extension services, obtain current and historical disk information, compare and update, and avoid restart operations.
It realizes that the updated disk takes effect during the operation of the data node, reduces the impact on the business and avoids interruptions caused by restarts.
Smart Images
Figure CN119179443A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of storage technology, and in particular to a method, device and electronic device for updating disk information of a data node. Background Art
[0002] HDFS (Hadoop Distributed File System) is a highly fault-tolerant distributed storage system that includes one or more data nodes. Usually, at least one disk is mounted on these data nodes for data storage. In actual applications, due to business needs or disk failures, it is inevitable to update the disks mounted on the data nodes, such as deleting some disks, replacing some disks, adding new disks, etc.
[0003] Currently, after updating the disk mounted on the data node, the data node needs to be restarted to make the updated disk take effect. During the restart of the data node, the business may be temporarily interrupted, which has a great impact on the business. Summary of the invention
[0004] In view of this, the present disclosure provides a disk information update method, disk information update device, electronic device, computer-readable storage medium and computer program product for a data node, which can make the updated disk effective during the operation of the data node and reduce the impact on the business.
[0005] In a first aspect, the present disclosure provides a method for updating disk information of a data node, wherein an extended service and a native service are deployed in the data node, and the native service is used to load disk information of each disk mounted on the data node into the storage of the data node; the method comprises:
[0006] In response to the extended service receiving a request for executing a disk information update task sent by a client, obtaining first disk information of each disk currently mounted on the data node and second disk information loaded into the storage by the native service before a current time point;
[0007] Compare the first disk information with the second disk information to determine the difference disk information between the first disk information and the second disk information;
[0008] Based on the differential disk information, the native service is called to update the second disk information in the storage to obtain updated second disk information.
[0009] In a second aspect, the present disclosure provides a disk information update device for a data node, wherein an extended service and a native service are deployed in the data node, and the native service is used to load disk information of each disk mounted on the data node into the storage of the data node; the device comprises:
[0010] an information acquisition module, configured to acquire, in response to the extended service receiving a request for executing a disk information update task sent by a client, first disk information of each disk currently mounted on the data node and second disk information loaded into the storage by the native service before a current time point;
[0011] A comparison module, used for comparing the first disk information with the second disk information to determine the difference disk information between the first disk information and the second disk information;
[0012] The information updating module is used to call the native service to update the second disk information in the storage based on the differential disk information to obtain the updated second disk information.
[0013] In a third aspect, the present disclosure provides an electronic device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the above method by executing the computer instructions.
[0014] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the above method.
[0015] In a fifth aspect, the present disclosure provides a computer program product, including computer instructions, and the computer instructions are used to enable a computer to execute the above method.
[0016] In the technical solutions of some embodiments of the present disclosure, by deploying an extension service in a data node, and after the extension service receives a request from a client to execute a disk information update task, the first disk information of each disk currently mounted on the data node and the second disk information loaded into the storage by the native service before the current time point are obtained, and the first disk information and the second disk information are compared to determine the difference disk information between the first disk information and the second disk information, and then based on the difference disk information, the native service is called to update the second disk information in the storage. In this way, the update of the disk information in the data node storage can be triggered without restarting the data node, so that the purpose of making the updated disk effective during the operation of the data node is achieved, and the impact of the disk information update in the storage on the business is reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0018] Figure 1 It is a schematic diagram of the architecture of the HDFS distributed file system provided by the present disclosure;
[0019] Figure 2 It is a schematic diagram of the architecture of data nodes in some technologies provided by the present disclosure;
[0020] Figure 3 is a schematic diagram of the architecture of a data node provided by an embodiment of the present disclosure;
[0021] Figure 4 It is a flowchart of a disk information updating method provided by an embodiment of the present disclosure;
[0022] Figure 5 is a module schematic diagram of a disk information updating device provided by some embodiments of the present disclosure;
[0023] Figure 6 It is a schematic diagram of the structure of an electronic device provided by some embodiments of the present disclosure. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.
[0025] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0026] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.
[0027] Herein, unless explicitly stated, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.
[0028] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.
[0029] It is understandable that before using the technical solutions disclosed in the various embodiments of the present disclosure, the types, scopes of use, usage scenarios, etc. of the information involved in the present disclosure should be informed to relevant users and their authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. The relevant users may include any type of right holders, such as individuals, enterprises, and groups.
[0030] For example, in response to receiving an active request from a user, a prompt message is sent to the relevant user to clearly prompt the relevant user that the operation requested to be performed will require obtaining and using the information of the relevant user, so that the relevant user can independently choose whether to provide information to software or hardware such as an electronic device, application, server or storage medium that executes the operation of the technical solution of the present disclosure based on the prompt message.
[0031] As an optional but non-limiting implementation, in response to receiving an active request from a relevant user, a prompt message is sent to the relevant user, for example, in the form of a pop-up window, in which the prompt message may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide information to the electronic device.
[0032] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.
[0033] The disk in the present disclosure may refer to a physical storage device or a disk volume divided from a physical storage device. Figure 1 , which is a schematic diagram of the architecture of the HDFS distributed file system. Figure 1In the example, the HDFS distributed file system includes a client 11, a name node 12, a data node 13, and a rack 14. Each rack 14 can be used to accommodate one or more data nodes 13.
[0034] The data node 13 is used to store files in the form of data blocks. The size of a single data block defaults to 128M (configurable). Each file can be divided into one or more data blocks and stored in the data node 13. After the file is divided into data blocks, each data block can be copied to multiple data nodes 13 for redundant storage, so as to prevent problems such as data loss caused by the failure of a single data node 13. Specifically, under normal circumstances, each data block can be copied into three copies A, B, and C, where copy A and copies B and C are stored in data nodes 13 of different racks 14, and copies B and C are stored in different data nodes 13 of the same rack 14. The name node 12 is used to manage the mapping relationship between files and data blocks, as well as the mapping relationship between data blocks and data nodes 13.
[0035] The client 11 can read files from the data node 13 or store files in the data node 13. Specifically, when the client 11 reads a file from the data node 13, it can first access the name node 12 to obtain the data node 13 corresponding to each data block of the file, as well as the storage location of each data block in the data node 13. Based on the information obtained from the name node 12, the client 11 can read the file from the data node 13. Correspondingly, when the client 11 saves a file to the data node 13, it can first divide the file into multiple data blocks, and then send the information of these data blocks to the name node 12. The name node 12 assigns a corresponding data node 13 and a storage location in the data node 13 to each data block. Based on the storage location assigned by the name node 13, the client 11 can save each data block to the data node 13.
[0036] Combined with reference Figure 2 , which is a schematic diagram of the architecture of the data node 13 in some technologies. Figure 2 In the example, the data node is mounted with multiple disks. Each disk can be mounted to one of the operating system file directories of the data node 13. After the disk is mounted successfully, data can be read and written in the disk by accessing the operating system file directory mounted on the disk. For example Figure 2 In the example, disk 1 can be mounted to the operating system file directory / disk1, disk 2 can be mounted to the operating system file directory / disk2, and so on. After the disks are mounted, data can be read and written in disk 1 by accessing the operating system file directory / disk1, and data can be read and written in disk 2 by accessing the operating system file directory / disk2.
[0037] Typically, the local configuration file / etc / hadoop / conf / hdfs-site.xml of the data node 13 may include the dfs.datanode.data.dir parameter. The dfs.datanode.data.dir parameter may be used to configure the operating system file directory mounted on each disk. For example, assuming Figure 2 In the example, disks 1 to n are mounted in the operating system file directories / disk1, / disk2, ..., / diskn, respectively. Then, in the local configuration file hdfs-site.xml of the data node 13, the parameter dfs.datanode.data.dir = / disk1, / disk2, ..., / diskn.
[0038] A native service is deployed in the data node 13. The native service is used to read the configured operating system file directory from the local configuration file hdfs-site.xml during the restart of the data node 13, and load the read operating system file directory into the memory of the data node 13. The data node 13 reads and writes data in the mounted disk based on the operating system file directory loaded into the memory. For example Figure 2 In the example, after the operating system file directories / disk1, / disk2, ..., / diskn are loaded into the memory of data node 13, data node 3 can read and write data in disk 1 by accessing / disk1. In short, only after the operating system file directory mounted on a disk is loaded into the memory of data node 13, the disk will take effect, and then data node 13 can read and write data in the disk.
[0039] At present, in some technologies, only when the data node 13 is restarted, the native service is triggered to load the operating system file directory configured in the local configuration file into the memory of the data node 13. During the loading process, the operating system file directory that exists in the local configuration file but does not exist in the memory will be loaded into the memory, the operating system file directory that does not exist in the local configuration file but exists in the memory will be deleted from the memory, and the operating system file directory that exists in both the local configuration file and the memory can continue to be stored in the memory.
[0040] In some technologies, after the native service updates the operating system file directory in the memory, it can continue to load the data block information in each disk mounted on the data node 13 into the memory based on the new operating system file directory in the memory, and report this data block information to the name node 12, so that the name node 12 can manage the data stored in the data node 13.
[0041] It is understandable that in actual applications, due to business needs or disk failures, it is inevitable to update the disk mounted on the data node 13. Normally, when a disk mounted on an operating system file directory is replaced, the operating system file directory configured in the local configuration file of the data node 13 may not change. For example, when disk 1 mounted under the operating system file directory / disk1 is damaged, the new disk n1 is used to replace the original disk 1. In this case, although the disk mounted under the operating system file directory / disk1 is changed from disk 1 to disk n1, the operating system file directory configured in the local configuration file has not changed. However, when a disk is deleted from the disk mounted on the data node 13, or a new disk is mounted for the data node, the operating system file directory configured in the local configuration file will also be updated accordingly. For example, due to the need for business expansion, a new disk n2 needs to be mounted on the data node 13. At this time, the operating system file directory corresponding to disk n2 needs to be added in the local configuration file. For another example, when disk 1 is damaged, if disk 1 is directly deleted from the disk mounted on data node 13 (ie, not replaced), then the operating system file directory corresponding to disk 1 also needs to be deleted from the local configuration file.
[0042] After the operating system file directory in the local configuration file is changed, in order to make the operating system file directory in the memory consistent with the operating system file directory in the local configuration file, some technologies will trigger the native service in the data node 13 to reload the operating system file directory in the local configuration file by restarting the data node 13. However, during the restart of the data node 13, temporary interruption of the business may be caused, which has a greater impact on the business.
[0043] In view of this, the present disclosure provides a new data node architecture and a disk information update method based on the data node architecture, which can make the updated disk effective during the operation of the data node and reduce the impact on the business. Figure 3 , is a schematic diagram of the architecture of a data node 33 provided in one embodiment of the present disclosure. Figure 3In the data node 33, an extended service and a native service are deployed. Among them, the native service is used to load the disk information of each disk mounted on the data node into the storage of the data node 33. Here, the storage of the data node 33 mainly refers to the memory of the data node 33. The disk information includes but is not limited to the operating system file directory corresponding to each disk mounted on the data node 33, the data block information in each disk, the remaining disk space of each disk, etc. The extended service may include a disk information update interface that can be called by the client 31. The maintenance personnel can call the disk information update interface provided by the extended service through the client 31, and send a request to the extended service to execute the disk information update task. After the extended service receives the request to execute the disk information update task sent by the client 31, the data node 33 can execute the disk information update method provided by the present application by running the program code in the extended service.
[0044] In some embodiments, the data node 33 and the client 31 communicate based on the Remote Procedure Call Protocol (RPC). On the client 31 side, a first command for calling the disk information update interface can be developed. The first command can be used to specify the IP and port of the data node to perform the disk information update task and the task to be performed. For example, the first command can be refreshconfigdatanode<host:ipc_port> <start>. Among them, host:ipc_port represents the IP and port of the data node to be executed for the disk information update task, and start represents the execution of the disk information update task. In this way, when the maintenance personnel need to execute the disk information update task in one of the data nodes 33, they can execute the above command and specify the IP and port of the data node 33 in the command, and then send a request to execute the disk information update task to the extension service in the corresponding data node 33.
[0045] The following uses the updating of the operating system file directory in the data node storage as an example to explain the disk information updating method. The disk information updating method is applied to the data node. Figure 4 , which is a flow chart of a disk information updating method provided by an embodiment of the present disclosure. Figure 4 In the method for updating disk information, the method comprises the following steps:
[0046] Step S401, in response to the extended service receiving a request for executing a disk information update task sent by the client 31, obtain the first disk information of each disk currently mounted on the data node 33, and the second disk information loaded into the storage by the native service before the current time point.
[0047] In this embodiment, the first operating system file directory read from the local configuration file of the data node 33 can be used as the first disk information. At the same time, by calling the native service, the second operating system file directory loaded into the storage by the native service before the current time point can be obtained as the second disk information.
[0048] Step S402: compare the first disk information with the second disk information to determine the difference disk information between the first disk information and the second disk information.
[0049] Specifically, the differential disk information mainly includes first differential disk information that exists in the first disk information but does not exist in the second disk information, and / or second differential disk information that does not exist in the first disk information but exists in the second disk information.
[0050] In this embodiment, the first difference disk information mainly includes the operating system file directory that exists in the first operating system file directory but does not exist in the second operating system file directory, and the second difference disk information mainly refers to the operating system file directory that does not exist in the first operating system file directory but exists in the second operating system file directory.
[0051] Step S403: Based on the differential disk information, the native service is called to update the second disk information in storage to obtain updated second disk information.
[0052] Specifically, based on the differential disk information, calling the native service to update the second disk information may include:
[0053] In the case where the difference disk information includes the first difference disk information, calling the native service to add the first difference disk information in the storage;
[0054] In case the difference disk information includes the second difference disk information, the second difference disk information is deleted from the storage.
[0055] This incremental update method can reduce the amount of data updates.
[0056] Specifically, in this embodiment, when the difference disk information includes the first difference disk information, the native service is called to add an operating system file directory that exists in the first operating system file directory but does not exist in the second operating system file directory to the storage; and when the difference disk information includes the second difference disk information, the native service is called to delete from the storage an operating system file directory that does not exist in the first operating system file directory but exists in the second operating system file directory.
[0057] In the above embodiment, in response to the extended service receiving the request for executing the disk information update task sent by the client 31, by determining the operating system file directory with differences between the first operating system file directory and the second operating system file directory, and then calling the native service to update the operating system file directory in the storage, it is possible to trigger the update of the operating system file directory in the storage without restarting the data node 33. That is, the update of the operating system file directory in the storage can be triggered during the operation of the data node 33, avoiding the problem of business interruption caused by restarting the data node 33, thereby reducing the business impact.
[0058] In some embodiments, after calling the native service to update the second disk information in step S403, the method of the present disclosure may further include:
[0059] Scan each disk currently mounted on the data node 33 to obtain data block information in each disk;
[0060] Call native services to load data block information into storage.
[0061] In this way, the problem of long-term inconsistency between the data block information in the data node storage and the data block information in the disk can be avoided. It should be noted that in the above embodiment, all disks currently mounted on the data node 33 are scanned, and the disk mounted by the newly added operating system file directory in the storage may not be scanned separately. In this way, when a disk mounted by a certain operating system file directory changes, the data block information in the new disk mounted by the operating system file directory can be synchronized to the storage in time, or when the data block information in a disk changes, the data block information in the hard disk can be synchronized to the data node storage in time.
[0062] In some embodiments, after the data block information is acquired, the data block information may also be synchronized to the name node 32 so that the name node can manage the data in the data node 33 based on the latest data block information.
[0063] It should be noted that, based on the above steps S401 to S403, other disk information other than the operating system file directory that has been loaded into the storage can also be updated. For example, in step S401, in response to receiving a request for executing a disk information update task sent by the client 31, the first remaining space information in each disk currently mounted on the data node 33 and the second remaining space information loaded into the storage by the native service before the current time point can be obtained. In step S402, the first remaining space information and the second remaining space information are compared to determine the difference remaining space information between the first remaining space information and the second remaining space information. In step S403, the native service is called to update the second remaining space information in the storage based on the difference remaining space information.
[0064] The present disclosure does not limit the comparison method of the first disk information and the second disk information, the specific form of the difference disk information, and the update method of the second disk information in storage.
[0065] In summary, in the technical solutions of some embodiments of the present disclosure, by deploying an extended service in the data node 33, and after the extended service receives a request to execute a disk information update task sent by the client 31, the first disk information of each disk currently mounted on the data node and the second disk information loaded into the storage by the native service before the current time point are obtained, and the first disk information and the second disk information are compared to determine the difference disk information between the first disk information and the second disk information, and then based on the difference disk information, the native service is called to update the second disk information in the storage. In this way, it is possible to trigger the update of the disk information in the data node storage without restarting the data node 33, so as to achieve the purpose of making the updated disk effective during the operation of the data node, and reduce the impact of the disk information update in the storage on the business.
[0066] Furthermore, through the above steps S401 to S403, it can be understood that each time the second disk information in the storage is updated, it is only necessary to execute the disk information update task once. For example, after updating the disk mounted on the data node 33 and updating the operating system file directory of the local configuration file, it is only necessary to execute the disk information update task once to complete the update of the operating system file directory in the storage. If at this time, the maintenance personnel initiate the disk information update task multiple times through the client 31, on the one hand, it will cause a waste of network resources, etc., and on the other hand, the simultaneous execution of multiple disk information update tasks may also cause problems such as abnormal disk information updates in the storage. In view of this, in combination with reference to Figure 3 In some embodiments, the extended service has a first task lock; after the extended service receives the client disk information update task, the method of the present disclosure may further include:
[0067] Determine whether the first task lock is in an occupied state;
[0068] If yes, a response indicating that the disk information update task has been completed is returned to the client 31;
[0069] If not, the first task lock is occupied and the disk information update task is triggered.
[0070] Specifically, if the first task lock is in an occupied state, it may indicate that a disk information update task is already being executed. In this case, there is no need to execute the currently received disk information update task, that is, generate a response representing the end of the task and return the response to the client 31; if the first task is not in an occupied state, it may indicate that no disk information update task is being executed. In this case, the currently received disk information update task is directly executed.
[0071] In the above embodiment, the execution of the disk information update task is controlled by the first task lock, so that problems such as waste of network resources and abnormal disk information update caused by the simultaneous execution of multiple disk information update tasks can be avoided.
[0072] In some embodiments, occupying the first task lock and triggering the execution of the disk information update task may include:
[0073] In the process of occupying the first task lock, creating a disk information update process for executing the disk information update task;
[0074] After the creation of the disk information update process is completed, the first task lock is released, and an execution response indicating that the task has started to be executed is returned to the client.
[0075] Specifically, this process can be called asynchronous execution of the disk information update task. That is, after the disk information update process is created, a response is returned to the client 31 during the execution of the disk information update task. In this way, it is possible to avoid returning a response to the client 31 only after the disk information update task is completed, thereby avoiding network resource consumption caused by a long-term communication link between the data node 33 and the client 31.
[0076] In some embodiments, the extended service also includes a status query interface that can be called by the client 31. After the maintenance personnel call the disk information update interface through the client 31, they can also call the status query interface provided by the extended service through the client 31 to instruct the data node 33 to perform a status query task. The status query task is used to query the execution status of the disk information update task. Specifically, on the client 31 side, a second command that calls the status query interface can be developed. The second command can be used to specify the IP and port of the data node to execute the status query task, as well as the operation to be performed by the extended service. For example, the first command can be refreshconfigdatanode<host:ipc_port> <status>. Among them, host:ipc_port represents the IP and port of the data node to be executed for the status query task, and status represents the execution of the status query task. In this way, when the maintenance personnel need to execute the status query task in one of the data nodes 33, they can execute the above second command and specify the IP and port of the data node 33 in the second command, and then send a request to execute the status query task to the extension service in the corresponding data node 33.
[0077] In response to the extended service receiving the request for executing the status query task sent by the client 31, the task execution result of the disk information update task can be queried, and the task execution result includes at least one of the task start time and the task end time of the disk information update task. In this way, the client 31 can timely understand the execution status of the disk information update task.
[0078] In some embodiments, the task execution result also includes the second disk information that has changed in the storage, such as the operating system file directory that has been added or deleted in the storage.
[0079] Further, the execution status query task mainly queries the task execution result of the most recent disk information update task. However, the most recent disk information update task may still be in the process of creating the disk information update process (i.e., the most recent disk information update task has not yet started to be executed). In this case, in order to avoid returning the task execution result of the last executed disk information update task to the client 31, the execution of the status query task can be controlled by the first task lock.
[0080] Specifically, after the extended service receives the request for executing the status query task sent by the client 31, it can be determined whether the first task lock is in an occupied state. If the first task lock is in an occupied state, it means that the disk information update process corresponding to the most recent disk information update task is being created (because the first task lock will be released after the disk information update process is created). In this case, the execution of the status query task can be suspended; if the first task lock is not in an occupied state, it means that the most recent disk information update task is being executed or has been completed. In this case, the first task lock can be occupied, and it can be determined whether the disk information update task has been completed; if the disk information update task has been completed, the task start time and task end time of the disk information update task are obtained as the task execution result. Here, during the execution of the status query task, occupying the first task lock can prevent the execution of a new disk information update task during the execution of the status query task.
[0081] In the above embodiment, by controlling the execution of the disk information update task and the status query task at the same time through the first task lock, it can be ensured that the acquired task execution result is always the task execution result of the most recent disk information update task, thereby improving the accuracy of the task execution result acquired by the client 31.
[0082] Combined with reference Figure 3 In some embodiments, similar to the extended service, in order to avoid data conflicts between multiple operations executed by calling the native service, the native service may have a second task lock. Before calling the native service to update the second disk information in the storage, it may be determined whether the second task lock is in an occupied state. If not, the second task lock is occupied, and the native service is called to update the second disk information in the storage. If so, the update operation of the second disk information is suspended until the second task lock is released.
[0083] Corresponding to the disk information updating method, the present disclosure also provides a disk information updating device for a data node. Figure 5 , which is a module diagram of a disk information updating device provided in some embodiments of the present disclosure. Figure 5 In the embodiment, the disk information updating device comprises:
[0084] The information acquisition module 501 is used to obtain the first disk information of each disk currently mounted on the data node and the second disk information loaded into the storage by the native service before the current time point in response to the extended service receiving the request for executing the disk information update task sent by the client;
[0085] A comparison module 502 is used to compare the first disk information with the second disk information to determine the difference disk information between the first disk information and the second disk information;
[0086] The information updating module 503 is used to call the native service to update the second disk information in storage based on the differential disk information to obtain the updated second disk information.
[0087] In some embodiments, the differential disk information includes first differential disk information that exists in the first disk information but does not exist in the second disk information, and / or second differential disk information that does not exist in the first disk information but exists in the second disk information; the information updating module 503 is specifically used to:
[0088] In the case where the difference disk information includes the first difference disk information, calling the native service to add the first difference disk information in the storage;
[0089] In a case where the difference disk information includes second difference disk information, and the second difference disk information is deleted from storage.
[0090] In some embodiments, after calling the native service to update the second disk information in storage, the information updating module 503 is further used to:
[0091] Scan each disk currently mounted on the data node to obtain the data block information in each disk;
[0092] Call native services to load data block information into storage.
[0093] In some embodiments, the extended service is also used to perform a status query task, and the information acquisition module 501 further includes:
[0094] In response to the extended service receiving a request for executing a status query task sent by the client, the task execution result of the disk information update task is queried, and the task execution result includes at least one of a task start time and a task end time of the disk information update task.
[0095] In some embodiments, the extended service has a first task lock;
[0096] After the extended service receives the request to execute the disk information update task, the information acquisition module 501 is specifically used to:
[0097] Determine whether the first task lock is in an occupied state;
[0098] If yes, a response indicating the completion of the disk information update task is returned to the client;
[0099] If not, the first task lock is occupied and the disk information update task is triggered.
[0100] In some embodiments, the information acquisition module 501 is specifically used to:
[0101] In the process of occupying the first task lock, creating a disk information update process for executing the disk information update task;
[0102] After the creation of the disk information update process is completed, the first task lock is released, and an execution response indicating that the task has started to be executed is returned to the client.
[0103] In some embodiments, after the extended service receives a request to execute a status query task, the information acquisition module 501 is specifically used to:
[0104] Determine whether the first task lock is in an occupied state;
[0105] If not, occupy the first task lock and determine whether the disk information update task has been completed;
[0106] If the disk information update task has been completed, the task start time and task end time of the disk information update task are obtained as the task execution result;
[0107] If the disk information update task is not completed, the task start time of the disk information update task is obtained as the task execution result.
[0108] In some embodiments, the native service has a second task lock;
[0109] Before calling the native service to update the second disk information in storage, the information updating module 503 is specifically used to:
[0110] It is determined whether the second task lock is in an occupied state. If not, the second task lock is occupied, and the native service is called to update the second disk information in the storage.
[0111] The disk information updating device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.
[0112] The device disclosed in the present invention has the same beneficial effects as the above method, which will not be described in detail here.
[0113] The present disclosure also provides an electronic device having the above Figure 5 The disk information update device shown.
[0114] Combined with reference Figure 6 , is a schematic diagram of the structure of an electronic device provided by some embodiments of the present disclosure. Figure 6 As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.
[0115] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.
[0116] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.
[0117] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0118] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.
[0119] The electronic device further comprises a communication interface 30 for the electronic device to communicate with other devices or a communication network.
[0120] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.
[0121] A part of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present disclosure through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.
[0122] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.< / status> < / start>
Claims
1. A method for updating disk information of a data node, characterized in that: The data node is deployed with an extended service and a native service, and the native service is used to load disk information of each disk mounted on the data node into the storage of the data node; the method includes: In response to the extended service receiving a request for executing a disk information update task sent by a client, obtaining first disk information of each disk currently mounted on the data node and second disk information loaded into the storage by the native service before a current time point; Compare the first disk information with the second disk information to determine the difference disk information between the first disk information and the second disk information; Based on the differential disk information, the native service is called to update the second disk information in the storage to obtain updated second disk information.
2. The method according to claim 1, characterized in that: The difference disk information includes first difference disk information that exists in the first disk information but does not exist in the second disk information, and / or second difference disk information that does not exist in the first disk information but exists in the second disk information; The calling the native service to update the second disk information in the storage based on the differential disk information includes: In a case where the difference disk information includes first difference disk information, calling the native service to add the first difference disk information in the storage; In case the difference disk information includes second difference disk information, the second difference disk information is deleted from the storage.
3. The method according to claim 2, characterized in that After calling the native service to update the second disk information in the storage, the method further includes: Scan each disk currently mounted on the data node to obtain data block information in each disk; The native service is called to load the data block information into the storage.
4. The method according to claim 1, characterized in that: The method further comprises: In response to the extended service receiving the request for executing the status query task sent by the client, querying the task execution result of the disk information update task, the task execution result including at least one of the task start time and the task end time of the disk information update task.
5. The method according to claim 4, characterized in that The extended service has a first task lock; After the extended service receives the request to execute the disk information update task, the method further includes: Determining whether the first task lock is in an occupied state; If yes, a response indicating that the disk information update task has been completed is returned to the client; If not, the first task lock is occupied and the disk information update task is triggered to be executed.
6. The method according to claim 5, characterized in that The occupying the first task lock and triggering the execution of the disk information update task includes: In the process of occupying the first task lock, creating a disk information update process for executing the disk information update task; After the creation of the disk information update process is completed, the first task lock is released, and an execution response indicating that the task has started to be executed is returned to the client.
7. The method according to claim 5, characterized in that After the extended service receives the request to execute the status query task, the method further includes: Determining whether the first task lock is in an occupied state; If not, occupy the first task lock and determine whether the disk information update task is completed; If the disk information update task has been completed, the task start time and task end time of the disk information update task are obtained as the task execution result; If the disk information update task is not completed, the task start time of the disk information update task is obtained as the task execution result.
8. The method according to claim 5, characterized in that The native service has a second task lock; Before calling the native service to update the second disk information in the storage, the method further includes: Determine whether the second task lock is in an occupied state; if not, occupy the second task lock, and call the native service to update the second disk information in the storage.
9. A disk information updating device for a data node, characterized in that: The data node is deployed with an extended service and a native service, and the native service is used to load disk information of each disk mounted on the data node into the storage of the data node; the device includes: an information acquisition module, configured to acquire, in response to the extended service receiving a request for executing a disk information update task sent by a client, first disk information of each disk currently mounted on the data node and second disk information loaded into the storage by the native service before a current time point; A comparison module, used for comparing the first disk information with the second disk information to determine the difference disk information between the first disk information and the second disk information; The information updating module is used to call the native service to update the second disk information in the storage based on the differential disk information to obtain the updated second disk information.
10. An electronic device, characterized in that: include: The memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the disk information updating method of any one of claims 1 to 8 by executing the computer instructions.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the disk information updating method according to any one of claims 1 to 8.
12. A computer program product, characterized in that The method comprises computer instructions for causing a computer to implement the disk information updating method according to any one of claims 1 to 8.