A method and apparatus for cross-cluster file reading

Abstract

This application discloses a cross-cluster file reading method and apparatus. The method is applied to a processing device and includes: receiving a file reading request from a target cluster, the file reading request including a file number corresponding to the file; searching for the file using the file number; identifying the cluster identifier of the found file, each cluster having a corresponding cluster identifier used to identify the cluster corresponding to the file; determining whether the cluster identifier corresponds to the target cluster; if not, reading the file from the cluster corresponding to the file. This application solves the problem in the prior art where files cannot be read between different clusters by receiving a file reading request from a processing device, searching for the file using the file number, and determining whether it is a cross-cluster file reading using the cluster identifier.

Description

Technical Field

[0001] This application relates to the field of file processing technology, and in particular to a method and apparatus for cross-cluster file reading. Background Technology

[0002] With the continuous advancement of technology, people have increasingly higher requirements for server quality. To meet these demands, cluster services have emerged. A cluster service refers to a large computer service system composed of several independent computers connected by a high-speed communication network, with files from different cluster services stored in isolation.

[0003] Existing technologies cannot read files across clusters, so how to achieve file reading between different clusters has become a problem that those skilled in the art urgently need to solve. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a cross-cluster file reading method and apparatus to solve the problem that existing technologies cannot perform cross-cluster file reading.

[0005] This application discloses a cross-cluster file reading method, which is applied to a processing device and includes the following steps:

[0006] Receive a file read request from the target cluster, wherein the file read request includes the file number corresponding to the file;

[0007] Use the file number to locate the file;

[0008] Identify the cluster identifier of the found file. Each cluster has a corresponding cluster identifier, which is used to identify the cluster to which the file belongs.

[0009] Determine whether the cluster identifier corresponds to the target cluster;

[0010] If not, then read the file from the cluster corresponding to the file.

[0011] Optionally, the cross-cluster file reading method further includes:

[0012] Get the file to be stored;

[0013] Add the cluster identifier of the cluster corresponding to the file to be stored to the file to be stored;

[0014] A universally unique identifier is generated using the filename of the file to be stored;

[0015] The universally unique identifier is used as the file number of the file to be stored.

[0016] Optionally, storing the file to be stored as the file number using the universally unique identifier includes:

[0017] The universally unique identifier is used as the file number of the file to be stored, and the file to be stored is stored in the database of the cluster corresponding to the file to be stored or in the storage medium of the cluster corresponding to the file to be stored.

[0018] Optionally, the step of using the file number to locate the file includes:

[0019] The file is located in the database and / or storage medium using the file number.

[0020] Optionally, reading the file from the cluster corresponding to the file includes:

[0021] Call the corresponding interface for cross-cluster file reading;

[0022] The file is read from the cluster corresponding to the file using the interface.

[0023] Optionally, determining whether the cluster identifier corresponds to the target cluster includes:

[0024] If so, then call the corresponding interface for reading files within the same cluster;

[0025] The file is read from the target cluster using the interface described above.

[0026] This application also discloses a cross-cluster file reading device, which includes the following modules:

[0027] A receiving module is used to receive file read requests from the target cluster, wherein the file read request includes the file number corresponding to the file;

[0028] The search module is used to search for the file using the file number;

[0029] The identification module is used to identify the cluster identifier of the found file. Each cluster has a corresponding cluster identifier, which is used to identify the cluster to which the file belongs.

[0030] The determination module determines whether the cluster identifier corresponds to the target cluster;

[0031] If not, then read the file from the cluster corresponding to the file.

[0032] Optionally, the cross-cluster file reading device further includes:

[0033] The acquisition module is used to acquire the file to be stored.

[0034] The cluster identifier adding module is used to add the cluster identifier of the cluster corresponding to the file to be stored to the file to be stored;

[0035] A universally unique identifier generation module is used to generate a universally unique identifier using the filename of the file to be stored.

[0036] The storage module is used to store the file to be stored as the file number of the file to be stored, using the universally unique identifier.

[0037] This application also provides an electronic device, including a memory and a processor, wherein:

[0038] The memory is used to store computer programs;

[0039] The processor is used to execute the computer program to implement the above-described cross-cluster file reading method.

[0040] This application also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described cross-cluster file reading method.

[0041] Compared with the prior art, this application has the following beneficial effects:

[0042] This application receives a file read request from a target cluster, locates the file using its file number, identifies the cluster identifier of the found file, and reads the file from the cluster corresponding to the file. Existing technologies isolate files between different cluster services using methods such as domain names. Because different clusters have different domain names, files cannot be read between different clusters. The method provided in this application is applied to a processing device. The target cluster cannot access other clusters, but the processing device can access the clusters provided by the processed device. After the processing device reads a file, it can send the file to the target cluster to achieve cross-cluster file reading within the target cluster. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 A flowchart of a cross-cluster file reading method provided in this application;

[0045] Figure 2 Flowchart of another cross-cluster file reading method provided in this application;

[0046] Figure 3 Flowchart of another cross-cluster file reading method provided in this application;

[0047] Figure 4 This is a schematic diagram of a cross-cluster file reading device provided in this application. Detailed Implementation

[0048] As described above, file reading is currently not possible between different clusters.

[0049] Research has revealed that in existing cluster services, files from different clusters are stored in isolation using methods such as domain names. For example, a file stored in the first cluster may be stored in the database or storage medium corresponding to the first cluster. This file is isolated by domain name; only the domain name corresponding to the first cluster can access the database or storage medium. The second cluster, because its domain name is different from the first cluster, cannot access the database or storage medium corresponding to the first cluster and therefore cannot read this file. This application uses a processing device to provide services to multiple clusters. The processing device has permission to access the databases and storage media corresponding to the multiple clusters being served. While physical isolation exists, the processing device enables file reading across the clusters.

[0050] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0051] Understandably, this method can be applied to processing devices capable of cross-cluster file reading, such as terminal devices or servers capable of data processing. This method can be executed independently by a terminal device or server, or it can be applied in network scenarios where a terminal device and a server communicate, executing in cooperation. The terminal device can be a computer, mobile phone, or other similar device. The server can be understood as an application server or a web server; in actual deployment, this server can be a standalone server or a cluster server.

[0052] Figure 1 A flowchart of a cross-cluster file reading method provided in this application is included, the method comprising the following steps:

[0053] S101: Receives file read requests from the target cluster.

[0054] Cluster service refers to a large computer service system composed of several independent computers using a high-speed communication network, where files from different cluster services are stored in isolation.

[0055] The processing device can provide services to multiple clusters; for example, it can provide services to three clusters: Cluster 1, Cluster 2, and Cluster 3. The processing device has permissions to read the databases corresponding to Cluster 1, Cluster 2, and Cluster 3, as well as to read the storage media of these three clusters, and possesses the relevant keys. If Cluster 1 wants to read file A, it sends a file read request to the processing device; that is, Cluster 1 is the target cluster.

[0056] The processing device can receive file read requests from the target cluster, and the file read request includes the file number of the file.

[0057] S102: Find files using file numbers.

[0058] The processing device can provide services to multiple clusters. For example, the processing device can provide services to three clusters: the first cluster, the second cluster, and the third cluster. The processing device receives a file read request from the first cluster, and the file read request includes the file number A corresponding to the file.

[0059] The processing device searches for file number A in three clusters: the first cluster, the second cluster, and the third cluster. The processing device can search in the databases corresponding to the first, second, and third clusters, or it can search in the storage media corresponding to the first, second, and third clusters. The search process can be performed simultaneously, such as searching in the databases corresponding to the three clusters at the same time, or it can be performed sequentially; both should fall within the scope of protection of this application.

[0060] S103: Identify the cluster identifier of the found file.

[0061] Each cluster receiving services from the processing equipment has a corresponding cluster identifier. Taking the above example, the cluster identifier for the first cluster is tenantA, the cluster identifier for the second cluster is tenantB, and the cluster identifier for the third cluster is tenantC. Of course, the cluster identifier can also be other letters, numbers, or combinations of letters and numbers. The cluster identifier is used to identify the cluster to which a file belongs. If a file's cluster identifier is tenantB, then this file corresponds to the second cluster.

[0062] After the processing device finds the file with file number A, it identifies the cluster identifier corresponding to the file with file number A.

[0063] S104: Determine if the cluster identifier corresponds to the target cluster.

[0064] Taking the above scenario as an example, if the processing device determines that the cluster identifier corresponding to file number A is tenantB, it means that file number A corresponds to the second cluster, the target cluster for sending the file read request is the first cluster, and the cluster identifier of the first cluster is tenantA. The processing device determines that the cluster identifier tenantB of file number A does not correspond to the cluster identifier tenantA of the target cluster. In this case, S105 is executed. If the processing device determines that the cluster identifier corresponding to file number A is tenantA, it means that file number A corresponds to the first cluster, the target cluster for sending the file read request is also the first cluster, and the processing device determines that the cluster identifier tenantA of file number A corresponds to the cluster identifier of the target cluster. In this case, S106 is executed.

[0065] S105: Read the file from the cluster corresponding to the file.

[0066] Taking the above scenario as an example, if the processing device determines that the cluster identifier corresponding to file number A does not correspond to the cluster identifier of the target cluster, it means that file number A does not correspond to the target cluster. The processing device then reads file number A from the second cluster corresponding to the identified cluster identifier tenantB. After reading file number A from the second cluster, the processing device can return file number A with cluster identifier tenantB to the first cluster, which is the target cluster. After returning to the target cluster, the target cluster reads file number A with cluster identifier tenantB.

[0067] S106: End cross-cluster file reading.

[0068] When the processing device recognizes that the cluster identifier of a file corresponds to the target cluster, it reads the file as a file within the same cluster. The target cluster can then read the file from the storage medium or database corresponding to the target cluster.

[0069] This application provides services to multiple clusters through processing equipment. It searches for files by file number, identifies the cluster identifier of the found file, and determines the cluster to which the file belongs based on the cluster identifier. The method provided in this application does not use domain names for physical file isolation; instead, it uses cluster identifiers for logical file isolation. The cluster identifier allows files from different clusters to be independent while enabling cross-cluster file reading.

[0070] When performing cross-cluster file reading, it is necessary to identify the cluster identifier, which can be added to the file when it is stored in the cluster. To better achieve cross-cluster file reading, this application provides another flowchart for a cross-cluster file reading method, which includes the following steps:

[0071] S201: Obtain the file to be stored.

[0072] When a cluster needs to store a file, it can first obtain the file to be stored. This cluster can be any cluster of receiving and processing devices providing services.

[0073] S202: Add the cluster identifier of the cluster corresponding to the file to be stored.

[0074] For example, the processing device provides services to three clusters: Cluster 1, Cluster 2, and Cluster 3. Cluster 1 needs to store file B. After Cluster 1 obtains file B, it adds the cluster identifier tenantA corresponding to Cluster 1 to file B.

[0075] S203: Generate a universally unique identifier using the filename of the file to be stored.

[0076] Taking the above situation as an example, the file name of file B is B, and the first cluster can use the file name of file B to generate a universally unique identifier.

[0077] A Universally Unique Identifier (UUID) is a software-defined standard, a concept proposed by the International Organization for Standardization (ISO). A UUID is a 128-bit value, which can be converted to hexadecimal. Each 4 bits represent one hexadecimal digit, resulting in 32 hexadecimal digits separated by four hyphens ("-") in an 8-4-4-4-12 sequence. This value can be calculated using a specific algorithm.

[0078] For example, the first cluster can convert the filename of file B into a universally unique identifier for file B: 12345678-abcd-abcd-abcd-12345678abcd.

[0079] S204: Store the file to be stored using the universally unique identifier as the file number of the file to be stored.

[0080] After the cluster converts the filename of the file to be stored into a universally unique identifier, the cluster can save the file as its file number.

[0081] Taking the above scenario as an example, after obtaining the universally unique identifier of file B, the first cluster stores the universally unique identifier of file B as a file number. The first cluster can either store the universally unique identifier of file B as a file number in the database corresponding to the first cluster, or it can store the universally unique identifier of file B as a file number in the storage medium corresponding to the first cluster.

[0082] When storing data, the cluster identifier and unique identifier can be combined to form a file number, which represents the file's identifier. For example, when storing file B, the cluster identifier tenantA corresponding to file B can be placed before the unique identifier of file B and separated by an underscore ("_"). Then, the file number corresponding to file B would be tenantA_12345678-abcd-abcd-abcd-12345678abcd.

[0083] The method provided in this application converts the filename into a universally unique identifier when storing files, and uses this universally unique identifier as the file number. This ensures that each file has a unique file number within the same cluster, thus avoiding the problem of finding multiple files with the same name when reading files across clusters. During the storage process, a cluster identifier is added to the file to be stored. This cluster identifier identifies the cluster to which the file belongs. The cluster identifier can be combined with the unique identifier to form the file number, thereby preventing different clusters from having the same file number.

[0084] Considering practical application scenarios, this application also provides a flowchart of another cross-cluster file reading method, which includes the following steps:

[0085] S301: Receives file read requests from the target cluster.

[0086] S302: Find files using file numbers.

[0087] Here, we again use the example of a processing device providing services to three clusters: Cluster 1, Cluster 2, and Cluster 3. Specifically, file information for Cluster 1 is stored in Database 1, file information for Cluster 2 is stored in Database 2, and file information for Cluster 3 is stored in Database 3. The databases use domain name isolation to isolate files; that is, the domain name corresponding to Cluster 1 can access Database 1, while the domain names corresponding to Cluster 2 and Cluster 3 cannot access Database 1. Using Cluster 1 as the target cluster, Cluster 1 sends a file read request to the processing device, requesting the file with the file number 12345678-abcd-abcd-abcd-12345678abcd.

[0088] After receiving a file read request from the first cluster, the processing device searches for the file information with file number 12345678-abcd-abcd-abcd-12345678abcd in the first, second, and third databases. This file information includes the file's address.

[0089] S303: Identify the cluster identifier of the found file.

[0090] After the processing device finds the file information with file number 12345678-abcd-abcd-abcd-12345678abcd, it identifies the cluster identifier of the file.

[0091] S304: Determine if the cluster identifier corresponds to the target cluster.

[0092] After the processing device identifies the cluster identifier of the file, it judges the cluster identifier. If the identified cluster identifier is tenantA, that is, the file corresponds to the first cluster, then S305 is executed; if the identified cluster identifier is not tenantA, that is, the file does not correspond to the target cluster and the file belongs to another cluster, then S307 is executed.

[0093] S305: Call the corresponding interface for reading files within the same cluster.

[0094] The processing device can call the interface corresponding to file reading in the same cluster, which connects to the relevant code for file reading in the same cluster.

[0095] S306: Use the interface to read the file in the target cluster.

[0096] Specifically, the processing device could return information about the file with file number 12345678-abcd-abcd-abcd-12345678abcd to the first cluster. The first cluster would then access the first database, locate the file with file number 12345678-abcd-abcd-abcd-12345678abcd, and read the file.

[0097] S307: Call the corresponding interface for cross-cluster file reading.

[0098] The processing device can call the corresponding interface for cross-cluster file reading, which connects to the relevant code for cross-cluster file reading.

[0099] S308: Use the interface to read files from the cluster corresponding to the file.

[0100] Here, we take the file with file number 12345678-abcd-abcd-abcd-12345678abcd and cluster identifier tenantB as an example.

[0101] Specifically, the processing device can use the address of the file with the file number 12345678-abcd-abcd-abcd-12345678abcd found in the file information to read the file at the corresponding address in the second cluster. After the processing device reads the file, it returns the file to the first cluster, adding the cluster identifier tenantB corresponding to the second cluster. This process completes the first cluster reading the file with file number 12345678-abcd-abcd-abcd-12345678abcd from the second cluster, thus completing the cross-cluster file reading.

[0102] After identifying the cluster identifier of a file, this application can determine whether the cluster identifier of the file corresponds to the target cluster. Different interfaces are used to implement different file reading methods, ensuring that the code of different services is fully decoupled and that the code of different methods is independent of each other. This can adapt to more complex scenarios and minimize the amount of code modification.

[0103] Figure 4 This application provides a schematic diagram of the structure of a cross-cluster file reading device 400, which includes:

[0104] The receiving module 401 is used to receive a file read request from the target cluster, wherein the file read request includes the file number corresponding to the file.

[0105] The search module 402 is used to search for the file using the file number.

[0106] The identification module 403 is used to identify the cluster identifier of the found file. Each cluster has a corresponding cluster identifier, which is used to identify the cluster corresponding to the file.

[0107] The determination module 404 determines whether the cluster identifier corresponds to the target cluster;

[0108] If not, then read the file from the cluster corresponding to the file.

[0109] Cross-cluster file reading devices may also include:

[0110] The acquisition module is used to acquire the file to be stored.

[0111] The cluster identifier adding module is used to add the cluster identifier of the cluster corresponding to the file to be stored to the file to be stored;

[0112] A universally unique identifier generation module is used to generate a universally unique identifier using the filename of the file to be stored.

[0113] The storage module is used to store the file to be stored as the file number of the file to be stored, using the universally unique identifier.

[0114] In practical applications, the computer-readable storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium, storing computer-readable program code that, when executed by a processor, implements the cross-cluster file reading method disclosed in the above embodiments.

[0115] Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0116] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0117] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0118] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0119] In some embodiments, the electronic device may be a mobile phone, tablet computer, desktop computer, laptop computer, ultra-mobile personal computer (UMPC), handheld computer, netbook, personal digital assistant (PDA), wearable electronic device, smartwatch, etc. This application does not impose any special restrictions on the specific form of the above-mentioned electronic devices.

[0120] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0121] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0122] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A cross-cluster file reading method, applied to a processing device, characterized in that, include: Get the file to be stored; Add the cluster identifier of the cluster corresponding to the file to be stored to the file to be stored; A universally unique identifier is generated using the filename of the file to be stored; The universally unique identifier is used as the file number of the file to be stored, and the file to be stored is stored in the database of the cluster corresponding to the file to be stored or in the storage medium of the cluster corresponding to the file to be stored; Receive a file read request from the target cluster, wherein the file read request includes the file number corresponding to the file; Use the file number to locate the file; Identify the cluster identifier of the found file. Each cluster has a corresponding cluster identifier, which is used to identify the cluster to which the file belongs. Determine whether the cluster identifier corresponds to the target cluster; If not, then call the corresponding interface for cross-cluster file reading; use the interface to read the file from the cluster corresponding to the file. If so, the corresponding interface for reading files in the same cluster is called; the file is then read from the target cluster using the interface.

2. The method according to claim 1, characterized in that, The process of using the file number to locate the file includes: The file is located in the database and / or storage medium using the file number.

3. A cross-cluster file reading device, characterized in that, include: The acquisition module is used to acquire the file to be stored. The cluster identifier adding module is used to add the cluster identifier of the cluster corresponding to the file to be stored to the file to be stored; A universally unique identifier generation module is used to generate a universally unique identifier using the filename of the file to be stored. The storage module is used to use the universally unique identifier as the file number of the file to be stored and to store the file to be stored in the database of the cluster corresponding to the file to be stored or in the storage medium of the cluster corresponding to the file to be stored. A receiving module is used to receive file read requests from the target cluster, wherein the file read request includes the file number corresponding to the file; The search module is used to search for the file using the file number; The identification module is used to identify the cluster identifier of the found file. Each cluster has a corresponding cluster identifier, which is used to identify the cluster to which the file belongs. The determination module determines whether the cluster identifier corresponds to the target cluster; If not, then call the corresponding interface for cross-cluster file reading; use the interface to read the file from the cluster corresponding to the file. If so, the corresponding interface for reading files in the same cluster is called; the file is then read from the target cluster using the interface.

4. An electronic device, characterized in that, Includes memory and processor, wherein: The memory is used to store computer programs; The processor is configured to execute the computer program to implement the cross-cluster file reading method as described in claim 1 or 2.

5. A computer-readable storage medium, characterized in that, Used to store computer programs, wherein the computer programs, when executed by a processor, implement the cross-cluster file reading method as described in claim 1 or 2.

Images