A method and apparatus for creating a fusion file

Abstract

The application discloses a fusion file creation method and device, and the method applied to a lib layer comprises the following steps: receiving a fusion file creation request, obtaining file creation information and attribute information from the fusion file creation request; creating a fusion message, and setting the file creation information and the attribute information in the fusion message; sending the fusion message to an mds, obtaining the file creation information and the attribute information from the fusion message by the mds, and creating metadata; receiving a message sent by the mds and indicating that the metadata creation is completed, and sending the message to a protocol layer; writing file data according to the metadata by the protocol layer, so as to confirm that the creation of the fusion file is completed; obtaining an md5 value of the fusion file, and sending the md5 value to the mds in an asynchronous mode. According to the technical scheme disclosed by the application, the metadata is created by message fusion, and the md5 value is updated by asynchronous operation, so that the creation of the fusion file is realized, the interaction times between the lib layer and the mds are reduced, and the creation efficiency of the fusion file is effectively improved.

Description

Technical Field

[0001] This application relates to the field of distributed systems technology, and more specifically, to a method and apparatus for creating fused files. Background Technology

[0002] In current distributed systems, depending on the application scenario, distributed systems can be further subdivided into distributed object systems and distributed file systems, etc. In protocol convergence scenarios (such as S3 / NAS (network attached storage) convergence), it is permissible to upload data from the distributed object system to the distributed file system using a special file method, thereby achieving a special file creation (i.e., realizing the creation of converged files).

[0003] For merged files, in order to enable the S3 protocol to access these files in the distributed file system, some special attributes need to be added to these files. Under the current protocol fusion state, the creation of merged files requires the lib layer to initiate a merged file creation and several attribute setting operations with the mds (metadata server), which increases the number of interactions between the lib layer and the mds and greatly reduces the efficiency of merged file creation.

[0004] In conclusion, how to improve the efficiency of creating fusion files is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a method and apparatus for creating fusion files, so as to improve the efficiency of creating fusion files.

[0006] To achieve the above objectives, this application provides the following technical solution:

[0007] A method for creating merged files, applied to the lib layer, includes:

[0008] Receive a fusion file creation request sent by the protocol layer, and obtain file creation information and attribute information from the fusion file creation request;

[0009] Create a fusion message, and set the file creation information and each of the attribute information in the fusion message;

[0010] The fusion message is sent to the MDS, which then retrieves the file creation information and attribute information from the fusion message and creates metadata.

[0011] The system receives a message from the MDS indicating that the metadata creation is complete and sends it to the protocol layer. The protocol layer then writes file data based on the metadata to confirm the completion of the fusion file creation.

[0012] Obtain the MD5 value of the fused file and send the MD5 value to the MDS asynchronously.

[0013] Preferably, setting each of the attribute information in the fusion message includes:

[0014] The attribute information is converted into a string type, encrypted and encoded, and then set in the fusion message.

[0015] Preferably, the MD5 value is sent to the MDS in an asynchronous manner, including:

[0016] When the lib layer and the mds perform permission update operations, the md5 value and the permission change message corresponding to the permission update operation are sent to the mds, and the mds updates the md5 value of the fused file.

[0017] Preferably, after obtaining the MD5 value of the fused file, the method further includes:

[0018] The MD5 value of the fused file is stored in a cache.

[0019] Preferred options also include:

[0020] If an exception occurs during the asynchronous transmission of the MD5 value to the MDS, the MD5 value will be transmitted to the MDS synchronously.

[0021] A method for creating merged files, applied to MDS, includes:

[0022] Receive a fusion message sent by the lib layer; the fusion message contains file creation information and attribute information obtained by the lib layer from the fusion file creation request;

[0023] The file creation information and attribute information are obtained from the fusion message. Metadata is created based on the file creation information and attribute information. A message indicating that the metadata creation is complete is sent to the lib layer. The lib layer sends the message indicating that the metadata creation is complete to the protocol layer. The protocol layer writes file data based on the metadata to confirm the completion of the fusion file creation.

[0024] Receive the MD5 value of the fusion file sent asynchronously by the lib layer.

[0025] Preferably, when creating metadata based on the file creation information and each of the attribute information, the method further includes:

[0026] The member structure is set; the member structure is set in the metadata using a pointer method.

[0027] Allocate memory space for the member structure to store the attribute information of each member.

[0028] Preferably, when creating metadata based on the file creation information and each of the attribute information, the method further includes:

[0029] Acquire a distributed lock and use the distributed lock to lock the process of creating the metadata.

[0030] A fusion file creation apparatus, applied to the lib layer, includes:

[0031] The first receiving module is used to receive the fusion file creation request sent by the protocol layer, and to obtain file creation information and attribute information from the fusion file creation request;

[0032] A creation module is used to create a fusion message, and to set the file creation information and each of the attribute information in the fusion message;

[0033] The first sending module is used to send the fusion message to the MDS, whereby the MDS obtains the file creation information and each attribute information from the fusion message and creates metadata.

[0034] The second receiving module is used to receive the message from the MDS indicating that the metadata creation is complete and send it to the protocol layer, whereby the protocol layer writes file data based on the metadata to confirm the completion of the fusion file creation.

[0035] The first acquisition module is used to acquire the MD5 value of the fused file and send the MD5 value to the MDS in an asynchronous manner.

[0036] A fusion file creation apparatus, applied to MDS, includes:

[0037] The third receiving module is used to receive the fusion message sent by the lib layer; the fusion message contains file creation information and attribute information obtained by the lib layer from the fusion file creation request;

[0038] The second acquisition module is used to acquire the file creation information and each attribute information from the fusion message, create metadata based on the file creation information and each attribute information, and send a message that the metadata creation is complete to the lib layer. The lib layer sends the message that the metadata creation is complete to the protocol layer, and the protocol layer writes file data based on the metadata to confirm the completion of the fusion file creation.

[0039] The fourth receiving module is used to receive the MD5 value of the fusion file sent asynchronously by the lib layer.

[0040] This application provides a method and apparatus for creating a fused file. The method applied to the lib layer includes: receiving a fused file creation request sent by the protocol layer, and obtaining file creation information and attribute information from the fused file creation request; creating a fused message, and setting the file creation information and attribute information in the fused message; sending the fused message to the MDS, where the MDS obtains the file creation information and attribute information from the fused message and creates metadata; receiving a message from the MDS indicating that metadata creation is complete and sending it to the protocol layer, where the protocol layer writes file data according to the metadata to confirm the completion of fused file creation; obtaining the MD5 value of the fused file and sending the MD5 value to the MDS asynchronously.

[0041] The technical solution disclosed in this application involves the lib layer obtaining file creation information and attribute information from the fusion file creation request sent by the protocol layer. This information is then set in the created fusion message. The fusion message is then sent to the MDS, where the MDS creates metadata based on the fusion message. This message fusion method sends a large amount of creation information to the MDS simultaneously, reducing the number of interactions between the lib layer and the MDS and simplifying the overall fusion file creation process. After the MDS creates the metadata and sends a message indicating completion to the lib layer, which in turn sends it to the protocol layer, the protocol layer can then write the file data. At this point, the protocol layer confirms the completion of the fusion file creation. Subsequently, the lib layer obtains the MD5 value of the fusion file and sends it to the MDS asynchronously, further reducing the number of interactions between the lib layer and the MDS. As can be seen, this application uses message fusion to create metadata and asynchronous operations to update the MD5 value to achieve fusion file creation, significantly reducing the number of interactions between the lib layer and the MDS, multiplying the efficiency of fusion file creation, and improving the user experience. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0043] Figure 1 A flowchart illustrating a method for creating a fusion file applied to the lib layer, as provided in this application embodiment;

[0044] Figure 2 A flowchart illustrating a method for creating a fusion file as provided in this application embodiment;

[0045] Figure 3 A flowchart illustrating a method for creating fusion files for MDS, provided as an embodiment of this application;

[0046] Figure 4 A schematic diagram of a fusion file creation apparatus applied to the lib layer provided in this application embodiment;

[0047] Figure 5 A schematic diagram of a fusion file creation device applied to MDS provided in this application embodiment;

[0048] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0049] For merged files, to enable the S3 protocol to access these files in the distributed file system, additional special attributes need to be added to these files (such as UID (user ID), GID (group ID), special file types, and MD5 (MD5 Message-Digest Algorithm) values). Under the current protocol merging state, for the creation of merged files, the lib layer needs to initiate one merged file creation and several attribute setting operations to the mds (metadata server) (the number of attribute setting operations corresponds to the number of special attributes), thus increasing the number of interactions between the lib layer and the mds and significantly reducing the efficiency of merged file creation.

[0050] Therefore, this application provides a method and apparatus for creating fusion files to improve the efficiency of fusion file creation.

[0051] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0052] See Figure 1 and Figure 2 ,in, Figure 1 This document illustrates a flowchart of a fusion file creation method applied to the lib layer, as provided in an embodiment of this application. Figure 2 This document illustrates a flowchart of a fusion file creation method provided in an embodiment of this application. The fusion file creation method provided in this embodiment, applied to the lib layer, may include:

[0053] S11: Receive the fusion file creation request sent by the protocol layer, and obtain the file creation information and attribute information from the fusion file creation request.

[0054] Fusion file creation is a special file creation operation under protocol fusion. A fusion file is data stored as a file that has been uploaded to the file storage system via the fusion protocol.

[0055] In this application, when a fusion file needs to be created, the protocol layer can send a fusion file creation request to the lib layer. The protocol layer refers to the protocol used by the client, such as NAS or S3 protocols. The lib layer (or lib library) is an intermediate library file between the distributed file system's MDS and OSD (Object-based Storage Device) and the upper-layer application, providing various interfaces to the upper-layer application. The MDS is the metadata service in the distributed file system.

[0056] Upon receiving a merged file creation request, the lib layer can collect all data related to file creation information and attribute settings from the request, thus obtaining the file creation information and attribute information of the merged file. Specifically, the file creation information may include the file upload type and protocol fusion type, while the attribute information may include special attributes such as uid and gid that need to be added to the merged file. It should be noted that the attribute information mentioned here does not include the MD5 value; the MD5 value can only be confirmed after the merged file data is written, and it does not need to be written during the metadata creation stage.

[0057] S12: Create a fusion message, setting the file creation information and various attribute information in the fusion message.

[0058] The lib layer can create fusion messages, and the obtained file creation information and attribute information can be set in the fusion messages created by the lib layer to achieve the fusion of file creation information and attribute information.

[0059] S13: Send the fusion message to the MDS, which then retrieves the file creation information and attribute information from the fusion message and creates metadata.

[0060] After the lib layer sets the file creation information and various attribute information in the fusion message, it can send the fusion message to the mds. The mds can add a new processing interface. After receiving the fusion message from the lib layer, it can use this new processing interface to process the fusion message to obtain the file creation information, specifically the protocol fusion type and file upload type. Furthermore, the mds can use this new processing interface or other existing interfaces to process the fusion message and obtain the various attribute information.

[0061] After obtaining file creation information and attribute information from the fusion message, MDS can create metadata based on this information. Metadata is data that records the attributes of a file or directory; it is descriptive data.

[0062] In the MDS, when creating metadata, a member structure can be added to store various attributes of the merged file. This member structure is set in the metadata as a pointer and defaults to NULL. Memory space is only allocated to the added member structure to store the attribute information when it is confirmed that the metadata is being created for the merged file (i.e., when the metadata is created based on the file creation information and various attribute information). This ensures that memory space is only allocated to the member structure when the merged file is created, thus avoiding waste of metadata space.

[0063] In addition, when creating metadata based on file creation information and various attribute information, MDS can also acquire a distributed lock and use the acquired distributed lock to lock the metadata creation process, so as to achieve data consistency between different MDS and between MDS and clients during the file creation process.

[0064] S14: Receive the message from MDS indicating that metadata creation is complete and send it to the protocol layer. The protocol layer then writes file data based on the metadata to confirm the completion of the fusion file creation.

[0065] After creating the metadata, the MDS can send a message indicating that metadata creation is complete to the lib layer. Correspondingly, the lib layer can receive this message from the MDS. Furthermore, the lib layer can forward this message to the protocol layer.

[0066] After receiving the metadata creation message, the protocol layer can write file data to the underlying distributed file system based on the metadata. In this application, the underlying distributed file system generally refers to the lib layer and mds. Specifically, the protocol layer can write file data to the lib layer based on the metadata, and the lib layer will then write the file data to the disk via OSD. After writing the file data, the protocol layer confirms that the fused file creation is complete; that is, the upper layer considers the creation of the fused file to be finished.

[0067] The above process enables the creation of metadata based on message fusion and distributed locks, so as to facilitate data writing based on metadata and realize the creation of fused files.

[0068] S15: Obtain the MD5 value of the merged file and send the MD5 value to the MDS asynchronously.

[0069] The MD5 hash of a merged file needs to be set during its application. However, the MD5 hash can only be obtained after the merged file is created and the data is written. Therefore, it cannot be sent to the MDS (Medium Data Storage) for attribute setting along with other attributes during merged file creation. Generally, the MD5 hash of a file can be set through the `setattr` interface in the lib layer. However, in this operation, the MDS revokes all client permissions to the merged file before setting the attribute. Even with optimization, the creation of a merged file still involves one creation and one attribute setting operation, and the attribute setting operation also involves permission revokement, which significantly impacts the performance of merged file creation. Therefore, in this application, after the merged file is created and the data is written, the lib layer can obtain the MD5 hash of the merged file and save it. Specifically, the protocol layer can generate the MD5 hash of the merged file and send it to the MDS so that the MDS can store the MD5 hash. Sending the MD5 hash of the merged file to the MDS signifies that the merging file creation is complete at the underlying level.

[0070] After the lib layer obtains and saves the MD5 value of the merged file, it can send the MD5 value of the merged file to the MDS asynchronously instead of synchronously, thereby effectively improving the efficiency of merging file creation.

[0071] The above achieves the updating of MD5 values ​​based on delayed asynchronous operations, eliminating the need to specifically set up an interaction between the lib layer and the MDS for the MD5 value. This reduces the number of interactions between the lib layer and the MDS, effectively improving the efficiency of creating fused files while ensuring performance.

[0072] As described above, this application uses message fusion and delayed asynchronous operation to update the MD5 value for creating fused files. Compared to existing methods that require multiple interactions between the lib layer and MDS during fused file creation, this application only requires the lib layer to fuse the messages together and send them to the MDS, and then uses an asynchronous method to update the MD5 value. Therefore, it can greatly reduce the number of interactions between the lib layer and MDS, significantly improve the efficiency of fused file creation, and thus enhance the user experience.

[0073] The technical solution disclosed in this application involves the lib layer obtaining file creation information and attribute information from the fusion file creation request sent by the protocol layer. This information is then set in the created fusion message. The fusion message is then sent to the MDS, where the MDS creates metadata based on the fusion message. This message fusion method sends a large amount of creation information to the MDS simultaneously, reducing the number of interactions between the lib layer and the MDS and simplifying the overall fusion file creation process. After the MDS creates the metadata and sends a message indicating completion to the lib layer, which in turn sends it to the protocol layer, the protocol layer can then write the file data. At this point, the protocol layer confirms the completion of the fusion file creation. Subsequently, the lib layer obtains the MD5 value of the fusion file and sends it to the MDS asynchronously, further reducing the number of interactions between the lib layer and the MDS. As can be seen, this application uses message fusion to create metadata and asynchronous operations to update the MD5 value to achieve fusion file creation, significantly reducing the number of interactions between the lib layer and the MDS, multiplying the efficiency of fusion file creation, and improving the user experience.

[0074] This application provides a method for creating a fusion file, which sets various attribute information in a fusion message, and may include:

[0075] Each attribute information is converted into a string type, encrypted and encoded, and then set in the fusion message.

[0076] In this application, when setting the attribute information in the fusion message, the lib layer can convert the attribute information into a string type, and can perform encryption and encoding operations on the string-type attribute information (specifically, it can encrypt first and then encode, or encode first and then encrypt, etc.). Then, the result obtained after the aforementioned operations can be set in the fusion message.

[0077] Among these, string type and encryption can ensure the security of attribute information, while encoding operations can facilitate transmission.

[0078] Accordingly, after receiving the fusion message, the MDS can decrypt and decode the encrypted attribute information in the fusion message to obtain the original attribute information.

[0079] This application provides a method for creating a fused file, which asynchronously sends the MD5 value to the MDS, and may include:

[0080] When the lib layer and mds perform permission update operations, the MD5 value and the permission change message corresponding to the permission update operation are sent to the mds, and the mds updates the MD5 value of the merged file.

[0081] In this application, when the lib layer sends the MD5 value to the mds asynchronously, specifically when the lib layer and mds perform a permission update operation (which is asynchronous compared to the creation of the merged file), the lib layer sends the MD5 value of the merged file and the permission change message corresponding to the permission update operation (this permission change message may include a specific permission change flag) to the mds together. That is, when sending the permission change message to the mds, the MD5 value of the merged file is sent along with it, without requiring separate interaction between the lib layer and mds for the MD5 value of the merged file.

[0082] Upon receiving a permission change message and the MD5 value of the merged file, the MDS can update the MD5 value of the merged file. Specifically, if the MDS detects that the file is a merged file and its MD5 value needs to be updated (e.g., the merged file is missing an MD5 value), then the MDS updates the MD5 value of the merged file. Additionally, upon receiving a permission change message, the MDS can also perform permission update operations corresponding to that message.

[0083] The method for creating a fusion file provided in this application embodiment may further include, after obtaining the MD5 value of the fusion file:

[0084] Store the MD5 value of the merged file in a cache.

[0085] In this application, after obtaining the MD5 value of the fused file, the lib layer can store the MD5 value of the fused file in a cache so that when the fused file is sent to the MDS asynchronously later, the MD5 value of the fused file can be quickly retrieved from the cache and then sent to the MDS asynchronously.

[0086] The fusion file creation method provided in this application embodiment may further include:

[0087] If an exception occurs during the asynchronous sending of the MD5 value to the MDS, the MD5 value should be sent to the MDS synchronously.

[0088] In this application, since asynchronous operations pose a risk of data loss, a dedicated attribute setting interface can be provided in the lib layer. If an exception occurs during the asynchronous sending of the MD5 value to the MDS, the dedicated attribute setting interface can be used to send the MD5 value to the MDS synchronously, thereby achieving synchronous updates of the MD5 value, preventing data loss, and ensuring the reliability of the MD5 value update.

[0089] This application also provides a method for creating fused files, applied to MDS, see [link to relevant documentation]. Figure 3 It illustrates a flowchart of a method for creating a fusion file applied to MDS, provided by an embodiment of this application, which may include:

[0090] S31: Receive the fusion message sent by the lib layer; the fusion message contains the file creation information and attribute information obtained by the lib layer from the fusion file creation request;

[0091] S32: Obtain file creation information and attribute information from the fusion message, create metadata based on the file creation information and attribute information, and send a message to the lib layer indicating that the metadata creation is complete. The lib layer then sends the message indicating that the metadata creation is complete to the protocol layer, and the protocol layer writes file data based on the metadata to confirm the completion of the fusion file creation.

[0092] S33: Receive the MD5 value of the fusion file sent asynchronously by the lib layer.

[0093] The technical solution disclosed in this application involves the lib layer obtaining file creation information and attribute information from the fused file creation request, setting this information in the created fused message, and then sending the fused message to the MDS. The MDS creates metadata based on the fused message, thus sending a large amount of creation information to the MDS together through message fusion, reducing the number of interactions between the lib layer and the MDS and simplifying the overall fused file creation process. After the MDS creates the metadata and sends the metadata creation completion message to the lib layer, which then sends it to the protocol layer, the protocol layer can write the file data. At this point, the protocol layer can confirm the completion of the fused file creation. Afterward, the lib layer can obtain the MD5 value of the fused file and send it to the MDS asynchronously, further reducing the number of interactions between the lib layer and the MDS. As can be seen from the above, this application uses message fusion to create metadata and asynchronous operation to update the MD5 value to achieve fused file creation, significantly reducing the number of interactions between the lib layer and the MDS, multiplying the efficiency of fused file creation, and improving the user experience.

[0094] The fusion file creation method provided in this application embodiment may further include, when creating metadata based on file creation information and various attribute information:

[0095] Set the member structure; the member structure is set in the metadata using pointer methods;

[0096] Allocate memory space for the member structure to store the attribute information.

[0097] The fusion file creation method provided in this application embodiment may further include, when creating metadata based on file creation information and various attribute information:

[0098] Acquire a distributed lock and use it to lock the process of creating metadata.

[0099] This application also provides a fusion file creation apparatus, applied to the lib layer, see [link to relevant documentation]. Figure 4 It shows a schematic diagram of a fusion file creation apparatus applied to the lib layer according to an embodiment of this application, which may include:

[0100] The first receiving module 41 is used to receive the fusion file creation request sent by the protocol layer and obtain file creation information and attribute information from the fusion file creation request;

[0101] Create module 42, which is used to create fusion messages and set file creation information and various attribute information in the fusion message;

[0102] The first sending module 43 is used to send the fusion message to the MDS, and the MDS obtains the file creation information and attribute information from the fusion message and creates metadata;

[0103] The second receiving module 44 is used to receive the message that the metadata creation is complete sent by the MDS and send it to the protocol layer. The protocol layer writes file data according to the metadata to confirm the completion of the creation of the fusion file.

[0104] The first acquisition module 45 is used to acquire the MD5 value of the fused file and send the MD5 value to the MDS in an asynchronous manner.

[0105] The technical solution disclosed in this application involves the lib layer obtaining file creation information and attribute information from the fusion file creation request sent by the protocol layer. This information is then set in the created fusion message. The fusion message is then sent to the MDS, where the MDS creates metadata based on the fusion message. This message fusion method sends a large amount of creation information to the MDS simultaneously, reducing the number of interactions between the lib layer and the MDS and simplifying the overall fusion file creation process. After the MDS creates the metadata and sends a message indicating completion to the lib layer, which in turn sends it to the protocol layer, the protocol layer can then write the file data. At this point, the protocol layer confirms the completion of the fusion file creation. Subsequently, the lib layer obtains the MD5 value of the fusion file and sends it to the MDS asynchronously, further reducing the number of interactions between the lib layer and the MDS. As can be seen, this application uses message fusion to create metadata and asynchronous operations to update the MD5 value to achieve fusion file creation, significantly reducing the number of interactions between the lib layer and the MDS, multiplying the efficiency of fusion file creation, and improving the user experience.

[0106] This application provides a fusion file creation apparatus, wherein the creation module 42 may include:

[0107] The fusion unit is used to convert various attribute information into string type, and then encrypt and encode it before setting it in the fusion message.

[0108] This application provides a fusion file creation apparatus, wherein the first acquisition module 45 may include:

[0109] The sending unit is used to send the MD5 value and the permission change message corresponding to the permission update operation to the MDS when the lib layer and MDS perform permission update operations, so that the MDS can update the MD5 value of the merged file.

[0110] The fusion file creation apparatus provided in this application embodiment may further include:

[0111] The storage module is used to store the MD5 value of the merged file in a cache after obtaining the MD5 value of the merged file.

[0112] The fusion file creation apparatus provided in this application embodiment may further include:

[0113] The second sending module is used to send the MD5 value to the MDS synchronously if an exception occurs during the asynchronous sending of the MD5 value to the MDS.

[0114] This application also provides a fusion file creation apparatus, applied to MDS, see [link to relevant documentation]. Figure 5It shows a schematic diagram of a fusion file creation apparatus for MDS provided in an embodiment of this application, which may include:

[0115] The third receiving module 51 is used to receive the fusion message sent by the lib layer; the fusion message contains file creation information and attribute information obtained by the lib layer from the fusion file creation request;

[0116] The second acquisition module 52 is used to acquire file creation information and attribute information from the fusion message, create metadata based on the file creation information and attribute information, and send a message to the lib layer that the metadata creation is complete. The lib layer sends the message that the metadata creation is complete to the protocol layer, and the protocol layer writes file data based on the metadata to confirm the completion of the fusion file creation.

[0117] The fourth receiving module 53 is used to receive the MD5 value of the fusion file sent asynchronously by the lib layer.

[0118] The technical solution disclosed in this application involves the lib layer obtaining file creation information and attribute information from the fusion file creation request sent by the protocol layer. This information is then set in the created fusion message. The fusion message is then sent to the MDS, where the MDS creates metadata based on the fusion message. This message fusion method sends a large amount of creation information to the MDS simultaneously, reducing the number of interactions between the lib layer and the MDS and simplifying the overall fusion file creation process. After the MDS creates the metadata and sends a message indicating completion to the lib layer, which in turn sends it to the protocol layer, the protocol layer can then write the file data. At this point, the protocol layer confirms the completion of the fusion file creation. Subsequently, the lib layer obtains the MD5 value of the fusion file and sends it to the MDS asynchronously, further reducing the number of interactions between the lib layer and the MDS. As can be seen, this application uses message fusion to create metadata and asynchronous operations to update the MD5 value to achieve fusion file creation, significantly reducing the number of interactions between the lib layer and the MDS, multiplying the efficiency of fusion file creation, and improving the user experience.

[0119] The fusion file creation apparatus provided in this application embodiment may further include:

[0120] The configuration module is used to set the member structure; the member structure is set in the metadata using pointer methods.

[0121] The allocation module is used to allocate memory space for member structures to store attribute information.

[0122] The fusion file creation apparatus provided in this application embodiment may further include:

[0123] The third acquisition module is used to acquire a distributed lock when creating metadata based on file creation information and various attribute information, and to lock the metadata creation process using the distributed lock.

[0124] This application also provides an electronic device, see [link to relevant documentation] Figure 6 It shows a schematic diagram of the structure of an electronic device provided in an embodiment of this application, which may include:

[0125] Memory 61 is used to store computer programs;

[0126] When the processor 62 executes the computer program stored in the memory 61, it can implement the steps of any of the above-described methods for creating a fusion file applied to the lib layer and / or any of the above-described methods for creating a fusion file applied to the mds.

[0127] This application also provides a readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps of any of the above-described methods for creating fusion files applied to the lib layer and / or any of the above-described methods for creating fusion files applied to the mds layer.

[0128] The readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0129] The descriptions of the relevant parts of the fusion file creation method for MDS, the fusion file creation apparatus for the lib layer, the fusion file creation apparatus for MDS, the electronic device, and the readable storage medium provided in the embodiments of this application can be found in the detailed description of the relevant parts of the fusion file creation method for the lib layer provided in the embodiments of this application, and will not be repeated here.

[0130] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that the elements inherent in a process, method, article, or apparatus that includes a list of elements are included. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Additionally, portions of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0131] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for creating fused files, characterized in that, Applied to the lib layer, including: The system receives a fusion file creation request sent by the protocol layer and obtains file creation information and attribute information from the fusion file creation request. The file creation information includes file upload type and protocol fusion type. The attribute information consists of special attributes added to the fusion file. Create a fusion message, and set the file creation information and each of the attribute information in the fusion message; The fusion message is sent to the MDS, which then retrieves the file creation information and attribute information from the fusion message and creates metadata. The system receives a message from the MDS indicating that the metadata creation is complete and sends it to the protocol layer. The protocol layer then writes file data based on the metadata to confirm the completion of the fusion file creation. Obtain the MD5 value of the fused file and send the MD5 value to the MDS asynchronously; Sending the MD5 value to the MDS asynchronously includes: When the lib layer and the mds perform permission update operations, the md5 value and the permission change message corresponding to the permission update operation are sent to the mds, and the mds updates the md5 value of the fused file.

2. The method for creating a fused file according to claim 1, characterized in that, Setting the attribute information in the fusion message includes: The attribute information is converted into a string type, encrypted and encoded, and then set in the fusion message.

3. The method for creating a fused file according to claim 1, characterized in that, After obtaining the MD5 value of the fused file, the process also includes: The MD5 value of the fused file is stored in a cache.

4. The method for creating a fused file according to claim 1, characterized in that, Also includes: If an exception occurs during the asynchronous transmission of the MD5 value to the MDS, the MD5 value will be transmitted to the MDS synchronously.

5. A method for creating fused files, characterized in that, Applied to MDS, including: The system receives a fusion message sent by the lib layer. The fusion message contains file creation information and attribute information obtained by the lib layer from the fusion file creation request. The file creation information includes the file upload type and protocol fusion type. The attribute information consists of special attributes added to the fusion file. The file creation information and attribute information are obtained from the fusion message. Metadata is created based on the file creation information and attribute information. A message indicating that the metadata creation is complete is sent to the lib layer. The lib layer sends the message indicating that the metadata creation is complete to the protocol layer. The protocol layer writes file data based on the metadata to confirm the completion of the fusion file creation. Receive the MD5 value of the fusion file sent asynchronously by the lib layer; The MD5 value of the fused file is sent asynchronously by the lib layer, including: when the lib layer and mds perform permission update operations, the MD5 value and the permission change message corresponding to the permission update operation are sent to the mds together, and the mds updates the MD5 value of the fused file.

6. The method for creating a fused file according to claim 5, characterized in that, When creating metadata based on the file creation information and each of the attribute information, the method further includes: The member structure is set; the member structure is set in the metadata using a pointer method. Allocate memory space for the member structure to store the attribute information of each member.

7. The method for creating a fused file according to claim 5, characterized in that, When creating metadata based on the file creation information and each of the attribute information, the method further includes: Acquire a distributed lock and use the distributed lock to lock the process of creating the metadata.

8. A fusion file creation apparatus, characterized in that, Applied to the lib layer, including: The first receiving module is used to receive the fusion file creation request sent by the protocol layer, and to obtain file creation information and attribute information from the fusion file creation request; A creation module is used to create a fusion message, and to set the file creation information and various attribute information in the fusion message; the file creation information includes the file upload type and protocol fusion type; the attribute information is information on special attributes added to the fusion file. The first sending module is used to send the fusion message to the MDS, whereby the MDS obtains the file creation information and each attribute information from the fusion message and creates metadata. The second receiving module is used to receive the message from the MDS indicating that the metadata creation is complete and send it to the protocol layer, whereby the protocol layer writes file data based on the metadata to confirm the completion of the fusion file creation. The first acquisition module is used to acquire the MD5 value of the fused file and send the MD5 value to the MDS in an asynchronous manner. The first acquisition module includes: The sending unit is used to send the MD5 value and the permission change message corresponding to the permission update operation to the MDS when the lib layer and MDS perform permission update operations, so that the MDS can update the MD5 value of the merged file.

9. A fusion file creation device, characterized in that, Applied to MDS, including: The third receiving module is used to receive fusion messages sent by the lib layer; the fusion message contains file creation information and attribute information obtained by the lib layer from the fusion file creation request; the file creation information includes file upload type and protocol fusion type; the attribute information is information on special attributes added to the fusion file. The second acquisition module is used to acquire the file creation information and each attribute information from the fusion message, create metadata based on the file creation information and each attribute information, and send a message that the metadata creation is complete to the lib layer. The lib layer sends the message that the metadata creation is complete to the protocol layer, and the protocol layer writes file data based on the metadata to confirm the completion of the fusion file creation. The fourth receiving module is used to receive the MD5 value of the fusion file sent asynchronously by the lib layer; The MD5 value of the fused file sent asynchronously by the lib layer includes: when the lib layer and mds perform permission update operations, the MD5 value and the permission change message corresponding to the permission update operation are sent to the mds together, and the mds updates the MD5 value of the fused file.

Images