A file system, method, device and storage medium for computing power network

Abstract

The present application provides a file system, method, device and storage medium of a computing power network, which belongs to the technical field of data processing. The system includes: the file metadata management module obtains the metadata information of any file on each cloud and each edge node of the computing power network and stores it in the target database, and obtains the task information associated with any file from the task manager of the computing power network, determines the duplicate version, derivative version and blood relationship of any file and stores it in the target database; the multi-cloud file management module obtains the metadata information, duplicate version, derivative version and blood relationship of any file on each cloud and each edge node of the computing power network in the target database, and generates a traceable file view. The present application aims to facilitate the management and search of files in the computing power network.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of computing power networks, and specifically, to a file system, method, device, and storage medium of a computing power network. Background Art

[0002] The computing power network is a new type of information infrastructure with computing as the center and the network as the foundation, integrating multiple elements such as network, cloud, data, intelligence, security, edge, end, and chain. It uses cloud-network integration technology and new network technology to deeply integrate edge computing nodes, cloud computing nodes, and various network resources including wide area networks, reducing the complexity of management and control of edge computing nodes, and coordinating with the computing and storage resources of cloud computing nodes and the network resources of the wide area network through centralized control or distributed scheduling methods to form a new generation of information infrastructure, providing customers with overall computing power services including computing, storage, and connection, and providing flexible and schedulable on-demand services based on business characteristics.

[0003] In the computing network scenario, as tasks are distributed to various clouds or edge nodes for execution, input files, output files, and other data files are scattered on the storage of various clouds or edge nodes. The same file may exist on multiple clouds and have multiple versions. As the tasks and files of the computing network accumulate, not only will a lot of storage space be wasted, causing the cost of the computing network to increase significantly, but it will also be difficult to manage and find any file. Summary of the invention

[0004] The embodiments of the present application provide a file system, method, device, and storage medium for a computing power network, which are intended to facilitate the management and search of files in the computing power network.

[0005] In a first aspect, an embodiment of the present application provides a file system of a computing network, the system comprising a file metadata management module, a multi-cloud file manager module and a target database, wherein: The file metadata management module is used to obtain metadata information of any file on each cloud and each edge node of the computing network and store it in the target database, and obtain task information associated with any file from the task manager of the computing network, determine the duplicate version, derivative version and blood relationship of any file and store it in the target database; The multi-cloud file management module is used to obtain metadata information, duplicate versions, derivative versions and blood relationships of any file on each cloud and each edge node of the computing power network in the target database, and generate a traceable file view.

[0006] Optionally, the metadata information of any file includes basic information, modification records, tag lists, access records, and migration records, where: The basic information includes file name, file size, creation time, last modification time, and checksum; The modification record includes a version list and basic information of each version; The tag list includes custom tags and tags generated by the storage system on each cloud and each edge node; The access record includes an access record list, and any access record in the access record list includes an access time and an access source; The migration record includes a migration record list, and any migration record in the migration record list includes source storage, target storage, migration time, and migration time consumption.

[0007] Optionally, task information associated with any file, including the source and usage list; The generation source is the task that generates the file; The usage list includes all tasks that use the file.

[0008] Optionally, the system further comprises: The multi-cloud file plug-in layer is used to provide the file metadata management module with a plug-in for accessing the file services of each cloud of the computing power network, and the plug-in is used to access the OpenAPI of each cloud of the computing power network.

[0009] Optionally, the system further comprises: A file management agent is deployed in a data center that does not have an OpenAPI in the computing network; The file management agent is used to obtain metadata information of any file in the data center and report it to the file metadata management module.

[0010] Optionally, the multi-cloud file management module is further used to: In response to a multi-condition search request, returning a target file and a target version corresponding to the multi-condition search request; The search conditions of the multi-condition search request include at least one of a file name, a tag, a modification time, a generation source, any task using the file, and a migration record.

[0011] Optionally, the multi-cloud file management module is further used to: Monitor whether any file meets the preset integration conditions; When any file meets the integration condition, the multi-cloud file plug-in layer is called to integrate the file in each cloud and each edge node of the computing power network.

[0012] In a second aspect, an embodiment of the present application provides a file processing method for a computing network, the method comprising: Obtaining metadata information of any file on each cloud and each edge node of the computing network and storing it in a target database, obtaining task information associated with any file from a task manager of the computing network, determining duplicate versions, derivative versions, and lineage relationships of any file and storing them in the target database; The metadata information, duplicate versions, derivative versions and lineage relationships of any file on each cloud and each edge node of the computing network are obtained in the target database, and a traceable file view is generated.

[0013] In a third aspect, an embodiment of the present application provides a computer device, comprising: at least one processor, and a memory, wherein the memory stores a computer program that can be run on the processor, wherein when the processor executes the computer program, the file processing method of the computing power network described in the first aspect of the embodiment is executed.

[0014] In a fourth aspect, an embodiment of the present application provides a non-volatile readable storage medium, wherein the non-volatile readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the file processing method of the computing power network described in the first aspect of the embodiment is executed.

[0015] Beneficial effects: The system provided in this embodiment includes a file Metadata management module, a multi-cloud file manager module and a target database. The file Metadata management module obtains metadata information of any file on each cloud and each edge node of the computing power network and stores it in the target database, and obtains task information associated with any file from the task manager of the computing power network, determines the duplicate version, derivative version and blood relationship of any file, and stores it in the target database.

[0016] The multi-cloud file management module obtains metadata information, duplicate versions, derivative versions and lineage relationships of any file on each cloud and each edge node of the computing network in the target database, and generates a traceable file view.

[0017] By obtaining metadata and task information of all files in each cloud and edge node of the computing network, it is possible to determine the duplicate versions, derivative versions, and lineage relationships of any file, and then generate a traceable file view, which can facilitate the management and search of distributed files on the computing network, and avoid the problem of files being scattered in the storage systems of various clouds or edge nodes, making it difficult to find files and versions when viewing files. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 It is a structural diagram of a file system of a computing network proposed in an embodiment of the present application; Figure 2 It is a structural diagram of a file system of a computing network proposed in an embodiment of the present application; Figure 3 It is a flowchart of the steps of the file processing method of the computing network provided by an embodiment of the present application; Figure 4 It is an execution diagram of a file processing method of a computing network proposed in an embodiment of the present application; Figure 5 It is an execution diagram of a file processing method of a computing network proposed in an embodiment of the present application; Figure 6 is a schematic diagram of a computer device provided by an embodiment of the present application; Figure 7 It is a schematic diagram of a non-volatile readable storage medium proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] Computing power network: A new type of information infrastructure with computing as the center and network as the foundation, integrating multiple elements such as network, cloud, data, intelligence, security, edge, end, and chain. It uses cloud-network integration technology and new network technology to deeply integrate edge computing nodes, cloud computing nodes, and various network resources including wide area networks, reducing the complexity of management and control of edge computing nodes, and coordinating with the computing and storage resources of cloud computing nodes and the network resources of wide area networks through centralized control or distributed scheduling methods to form a new generation of information infrastructure, providing customers with overall computing power services including computing, storage, and connection, and providing flexible and schedulable on-demand services based on business characteristics.

[0022] In the computing network scenario, as tasks are distributed to various clouds or edge nodes for execution, input files, output files, and other data files are scattered on the storage of various clouds or edge nodes. The same file may exist on multiple clouds and have multiple versions.

[0023] As the tasks and files of the computing network accumulate, not only will a lot of storage space be wasted, causing the cost of the computing network to rise sharply, but it will also be difficult to find the desired file and version when viewing the file, and it will be difficult to obtain the desired input file when executing a new task.

[0024] Therefore, the embodiment of the present application provides a file system of a computing network, which is convenient for managing and searching distributed files on the computing network, avoiding the problem that files are scattered in the storage systems of various clouds or edge nodes, making it difficult to find files and versions when viewing files. Reference Figure 1 , showing a structural schematic diagram of a file system of a computing power network provided in an embodiment of the present application, wherein the file system of the computing power network includes a file metadata management module, a multi-cloud file manager module, and a target database.

[0025] The file metadata management module is used to obtain metadata information of any file on each cloud and each edge node of the computing power network and store it in the target database.

[0026] For example, the metadata information of any file includes basic information, modification records, tag lists, access records, and migration records.

[0027] The basic information includes the file name, file size, creation time, last modification time and a check code, such as MD5.

[0028] The modification record includes a version list and basic information of each version.

[0029] The tag list includes custom tags and tags generated by the storage system on each cloud and each edge node.

[0030] The access record includes an access record list, and any access record in the access record list includes an access time and an access source.

[0031] The migration record includes a migration record list, and any migration record in the migration record list includes source storage, target storage, migration time, and migration time consumption.

[0032] After the file Metadata management module obtains the metadata information of any file on each cloud and each edge node of the computing network, the file Metadata management module can also obtain the task information associated with any file from the task manager of the computing network to determine the duplicate version, derivative version and blood relationship of any file.

[0033] The task information associated with any file includes a generation source and a use list, wherein the generation source is the task that generates the file; and the use list includes all tasks that use the file.

[0034] After the file metadata management module collects the metadata information of any file and the task information associated with any file, it can perform version association analysis and lineage analysis on any file, determine the duplicate versions and derivative versions of any file on each cloud and each edge node of the computing network, and deduce the generation process of each version to determine the lineage relationship of any file.

[0035] Specifically, a duplicate version of any file refers to substantially identical versions of the same file on various clouds and edge nodes of the computing network. For example, multiple files with the same file size and the same checksum such as MD5 are duplicate versions.

[0036] For example, file 1 is named xx1 in the storage system of cloud A of the computing network. Suppose that when cloud B of the computing network executes any task, file 1 on cloud A is copied, migrated and renamed to xx2. Cloud B obtains file 2 after executing the task based on file 1, but file 1 named xx2 is still stored in the storage system of cloud B. After comparison, file 1 named xx1 on cloud A and file 1 named xx2 on cloud B have the same file size and the same checksum. Therefore, although file 1 named xx1 and file 1 named xx2 have different file names, they are actually duplicate versions.

[0037] A derivative version of any file means that the same file is transferred to multiple clouds, the file is modified on each cloud, and the modified files have the same source but different contents, such as different file sizes or different checksums such as MD5.

[0038] For example, file 1 is named xx1 in the storage system of cloud A of the computing network. Assuming that cloud B and cloud C of the computing network each need file 1 in the process of executing tasks, cloud B and cloud C each request to migrate file 1 to themselves. Cloud B and cloud C modify file 1 respectively. Although the source of file 1 in the storage systems of cloud B and cloud C is file 1 in cloud A, the file size or checksum is different after modification. Therefore, the modified file 1 in cloud B and cloud C is a derivative version of file 1 in cloud A.

[0039] The blood relationship of any file refers to: the generation process of each version of the file, the parent-child relationship and brother relationship between versions, and other association relationships. For example, after executing a task based on file 1 and file 2, file 3 and file 4 are generated. Then file 3 has a parent-child relationship with file 1 and file 2, and file 3 has a brother relationship with file 4.

[0040] The metadata information of any file acquired by the file metadata management module and the duplicate version, derivative version and blood relationship of any file are all stored in the target database.

[0041] The target database may be selected in actual applications according to the requirements of the actual applications, such as Hbase, etc., which is not limited in this embodiment.

[0042] The multi-cloud file management module is used to obtain metadata information, duplicate versions, derivative versions and blood relationships of any file on each cloud and each edge node of the computing power network in the target database, and generate a traceable file view.

[0043] Specifically, the multi-cloud file management module obtains the metadata information, duplicate versions, derivative versions and blood relationship of any file from the target database, and the generated traceable file view can be presented in different forms, such as a tree structure. In the tree structure, an original file can be used as a root node, and the description information of the node includes the metadata information and duplicate versions of the file, as well as the metadata information of each duplicate version; the lower-level nodes of the root node can be the derivative versions corresponding to the original file and the files with blood relationships.

[0044] In the actual implementation process, other methods may be used, and multiple presentation forms may be combined, such as a presentation method that combines a list, a directory, and a tree structure.

[0045] The system provided in this embodiment includes a file Metadata management module, a multi-cloud file manager module and a target database. The file Metadata management module obtains metadata information and task information of all files in each cloud and each edge node of the computing network. The duplicate versions, derivative versions and blood relationships of any file can be determined and stored in the target database. Then the multi-cloud file manager module generates a traceable file view, which can facilitate the management and search of distributed files on the computing network, avoiding the problem that files are scattered in the storage systems of various clouds or edge nodes, making it difficult to find files and versions when viewing files.

[0046] Reference Figure 2, showing a structural diagram of a file system of a computing power network provided in an embodiment of the present application. In a feasible implementation manner, the file system of the computing power network includes a file metadata management module, a multi-cloud file manager module, a target database, a multi-cloud file plug-in layer, and a file management agent.

[0047] Specifically, the multi-cloud file plug-in layer is used to provide the file metadata management module with a plug-in for accessing the file services of each cloud of the computing power network, and the plug-in is used to access the OpenAPI of each cloud of the computing power network.

[0048] The computing network usually includes cloud platforms from different vendors, and the access interface configurations of cloud platforms from different vendors may also be different. Therefore, in order to facilitate access to the file services of each cloud in the computing network, plug-ins for the file services corresponding to each cloud in the computing network are configured in the multi-cloud file plug-in layer. For example, in the multi-cloud file plug-in layer, plug-in 1 for the file service of cloud A, plug-in 2 for the file service of cloud B... The OpenAPI of any cloud can be accessed through the plug-in for the file service of that cloud.

[0049] The multi-cloud file plug-in layer can provide a unified interface for the file metadata management module to perform read and write operations on each cloud in the computing network by integrating the plug-ins of the file services of each cloud.

[0050] However, in addition to the cloud platform, the computing network usually also includes some local storage data centers, which do not have OpenAPI data centers. Therefore, in order for the file metadata management module to obtain the metadata information of the files in the data center, the file management Agent can be deployed on the data center in the computing network that does not have OpenAPI.

[0051] The file management agent is used to obtain the metadata information of any file in the data center where it is deployed, and then report the obtained metadata information of any file to the file metadata management module.

[0052] When the file metadata management module obtains the metadata information of any file in the computing network, it can read the metadata information of any file on each cloud and each edge node of the computing network through the multi-cloud file plug-in layer, and receive the metadata information of any file in the data center reported by the file management agent deployed on any data center.

[0053] The metadata information of any file includes basic information, modification records, tag lists, access records, and migration records.

[0054] The basic information includes the file name, file size, creation time, last modification time and verification code, such as MD5; the modification record includes a version list and basic information of each version; the tag list includes custom tags and tags generated by the storage system on each cloud and each edge node; the access record includes an access record list, and any access record in the access record list contains the access time and access source; the migration record includes a migration record list, and any migration record in the migration record list contains the source storage, target storage, migration time and migration duration.

[0055] After the file Metadata management module obtains the metadata information of any file on each cloud and each edge node of the computing network, the file Metadata management module obtains the task information associated with any file from the task manager of the computing network; the task information associated with any file includes a generation source and a usage list, the generation source is the task that generates the file; the usage list includes all tasks that use the file.

[0056] After the file metadata management module collects the metadata information of any file and the task information associated with any file, it can perform version analysis and lineage analysis on any file, determine the duplicate versions and derivative versions of any file on each cloud and edge node of the computing network, and deduce the generation process of each version to determine the lineage relationship of any file.

[0057] The metadata information of any file acquired by the file metadata management module, the task information associated with the file, and the duplicate version, derivative version and blood relationship of any file generated by analysis are all stored in the target database.

[0058] The multi-cloud file management module obtains metadata information, duplicate versions, derivative versions and lineage relationships of any file on each cloud and each edge node of the computing network in the target database, and generates a traceable file view. The generated traceable file view can be presented in different forms, such as a tree structure or a presentation method that combines lists, directories and tree structures.

[0059] The multi-cloud file management module also provides a multi-condition search function, such as searching by file name, tag, modification time record, generation source, associated task, migration record, etc., so that users can quickly find the desired files and versions.

[0060] Specifically, a human-computer interaction interface can be configured in the file system of the computing power network. Users can view the traceable file view of any file on the human-computer interaction interface, and can also enter multi-condition search requests for the target file and target version on the human-computer interaction interface.

[0061] The multi-cloud file management module responds to a multi-condition search request and returns a target file and a target version corresponding to the multi-condition search request; the search conditions of the multi-condition search request include at least one of a file name, a tag, a modification time, a generation source, any task using the file, and a migration record.

[0062] The multi-cloud file management module can also monitor whether any file meets the preset integration conditions; when any file meets the integration conditions, the multi-cloud file plug-in layer is called to integrate the file in each cloud and each edge node of the computing power network.

[0063] In a feasible implementation, the preset integration condition for any file may be whether the number of duplicate versions of any file exceeds a duplication threshold. The duplication threshold may be set according to the actual application of the computing power network, such as being set to 5.

[0064] When the multi-cloud file management module detects that the number of duplicate versions of any file exceeds a duplicate threshold, a duplicate integration strategy is generated. The duplicate integration strategy may be to retain the file version with the most recent last modification time and delete the remaining duplicate versions.

[0065] The multi-cloud file management module determines the cloud corresponding to the duplicate version to be deleted according to the metadata information of the duplicate version to be deleted, and then calls the multi-cloud file plug-in layer to access the cloud corresponding to the duplicate version to be deleted. For example, a deletion request corresponding to the duplicate version to be deleted is sent, and the deletion request can carry the storage location of the file version with the most recent last modification time, so that the cloud can call or migrate the file when it still needs the file later.

[0066] In a feasible implementation, the preset integration condition for any file may be that the number of storage areas for any file exceeds a storage threshold, and the storage threshold may be set according to the actual application of the computing power network, such as being set to 5.

[0067] When the multi-cloud file management module detects that the number of storage areas for any file exceeds the storage threshold, a storage integration strategy is generated. The repeated integration strategy may be to randomly keep one storage area of ​​the file unchanged and delete the files in the remaining storage areas. For example, if the storage area of ​​the file is the storage area corresponding to the source of the file after the computing network executes the task, the area may be kept unchanged.

[0068] In a feasible implementation, the preset integration condition for any file may be that the number of derivative versions of any file exceeds a derivative threshold, and the derivative threshold may be set according to the actual application of the computing power network, such as being set to 10.

[0069] When the multi-cloud file management module detects that the number of derivative versions of any file exceeds a derivative threshold, a derivative integration strategy is generated. The derivative integration strategy may be to package all derivative versions of the file and place them in a target storage location.

[0070] In the actual implementation process, the integration conditions and integration strategies of the multi-cloud file management module can be determined according to the needs of the actual application, and this embodiment does not limit this.

[0071] This system not only obtains metadata information and task information of all files in each cloud and edge node of the computing network, determines the duplicate versions, derivative versions and blood relationships of any file, and then generates a traceable file view, but also facilitates the management and search of distributed files on the computing network, avoiding the problem that files are scattered in the storage systems of various clouds or edge nodes, making it difficult to find files and versions when viewing files; it also saves storage costs by integrating and optimizing the duplicate versions and numerous derivative versions of any file.

[0072] Reference Figure 3 , showing a step flow chart of a file processing method of a computing power network in an embodiment of the present application, the method can be applied to the file processing system of the computing power network, and the file system of the computing power network includes a file metadata management module, a multi-cloud file manager module, a target database, a multi-cloud file plug-in layer and a file management Agent.

[0073] The method may specifically include the following steps: S101: Obtain metadata information of any file on each cloud and each edge node of the computing network and store it in a target database.

[0074] Specifically, when obtaining the metadata information of any file in the computing network, the file metadata management module can read the metadata information of any file on each cloud and each edge node of the computing network through the multi-cloud file plug-in layer, and receive the metadata information of any file in the data center reported by the file management agent deployed on any data center, and then store it in the target database. The target database can be selected in actual application according to the needs of the actual application, such as Hbase.

[0075] The metadata information of any file includes basic information, modification records, tag lists, access records, and migration records.

[0076] Basic information includes file name, file size, creation time, last modification time, and verification code, such as MD5.

[0077] The revision history includes a list of versions and basic information about each version.

[0078] The tag list includes custom tags and tags generated by the storage system on each cloud and each edge node.

[0079] The access record includes an access record list, and any access record in the access record list includes an access time and an access source.

[0080] The migration record includes a migration record list, and any migration record in the migration record list includes source storage, target storage, migration time, and migration consumption time.

[0081] S102: Obtain task information associated with any file from the task manager of the computing network, determine duplicate versions, derivative versions and blood relationships of any file, and store them in the target database.

[0082] The file metadata management module can obtain task information associated with any file from the task manager of the computing power network; the task information associated with any file includes a generation source and a usage list, the generation source is the task that generates the file; the usage list includes all tasks that use the file.

[0083] After the file metadata management module collects the metadata information of any file and the task information associated with any file, it performs version association analysis and lineage relationship analysis on any file, determines the duplicate versions and derivative versions of any file on each cloud and edge node of the computing network, and deduce the generation process of each version to determine the lineage relationship of any file.

[0084] A duplicate version of any file is a substantially identical version of the same file on each cloud and each edge node of the computing network. For example, multiple files with the same file size and the same checksum such as MD5 are duplicate versions.

[0085] A derivative version of any file is a file that is delivered to multiple clouds and modified on each cloud. The modified files have the same source but different contents, such as different file sizes or different checksums such as MD5.

[0086] The blood relationship of any file is the generation process of each version of the file, and the parent-child relationship and brother relationship between the versions.

[0087] The metadata information of any file obtained by the file metadata management module and the duplicate versions, derivative versions and lineage relationships of any file are all stored in the target database.

[0088] S103: Obtain metadata information, duplicate versions, derivative versions, and lineage relationships of any file on each cloud and each edge node of the computing network in the target database, and generate a traceable file view.

[0089] Specifically, the multi-cloud file management module obtains metadata information, duplicate versions, derivative versions, and lineage relationships of any file from the target database.

[0090] Then the multi-cloud file management module can generate a traceable file view, which can be presented in different forms, such as a tree structure or a presentation method that combines a list, a directory, and a tree structure.

[0091] In a feasible implementation manner, the method further includes: S104: The multi-cloud file management module responds to the multi-condition search request and returns the target file and target version corresponding to the multi-condition search request; the search conditions of the multi-condition search request include at least one of the file name, label, modification time, generation source, any task using the file, and migration record.

[0092] After the multi-cloud file management module generates a traceable file view, it can support users' multi-condition search functions, such as searching by file name, tag, modification time record, source, associated tasks, migration records, and other conditions, so that users can quickly find the desired files and versions.

[0093] For example, a human-computer interaction interface can be configured in the file system of the computing network. Users can view the traceable file view of any file on the human-computer interaction interface, and can also enter multi-condition search requests for target files and target versions on the human-computer interaction interface; the multi-cloud file management module responds to the multi-condition search request and returns the target file and target version corresponding to the multi-condition search request; thereby supporting users to find target files and target versions through multi-condition search.

[0094] S105: The multi-cloud file management module can also monitor whether any file meets the preset integration conditions; when any file meets the integration conditions, the multi-cloud file plug-in layer is called to integrate the file in each cloud and each edge node of the computing network.

[0095] For example, the preset integration condition for any file may be whether the number of duplicate versions of any file exceeds a duplicate threshold. When the multi-cloud file management module detects that the number of duplicate versions of any file exceeds the duplicate threshold, a duplicate integration strategy is generated. The duplicate integration strategy may be to retain the file version with the most recent last modification time and delete the remaining duplicate versions.

[0096] The preset integration condition for any file may be that the number of storage areas for any file exceeds a storage threshold. When the multi-cloud file management module detects that the number of storage areas for any file exceeds the storage threshold, a storage integration strategy is generated. The repeated integration strategy may be to randomly keep one storage area of ​​the file unchanged and delete the files in the remaining storage areas.

[0097] The preset integration condition for any file may be that the number of derivative versions of any file exceeds a derivative threshold. When the multi-cloud file management module detects that the number of derivative versions of any file exceeds the derivative threshold, a derivative integration strategy is generated. The derivative integration strategy may be to package all derivative versions of the file and place them in a target storage location.

[0098] Reference Figure 4 , showing an execution diagram of a file processing method of a computing network in an embodiment of the present application. When establishing a traceable file view, first, the file Metadata management module calls the OpenAPI of any cloud through the multi-cloud file plug-in layer to obtain the metadata information of any file.

[0099] Then, the file management agent deployed in the data center reports the metadata information of any file to the file metadata management module.

[0100] Then, the file metadata management module obtains the task information associated with any file in the task manager of the computing network.

[0101] Next, the file metadata management module saves the collected metadata information of any file in the target database.

[0102] Then, the file metadata management module performs version association analysis and lineage relationship analysis on any file.

[0103] Next, the file metadata management module stores any file version relationship and lineage relationship in the target database.

[0104] Then, the multi-cloud file management module obtains metadata information, duplicate versions, derivative versions and lineage relationships of any file from the target database to generate a traceable file view.

[0105] Reference Figure 5 , showing an execution diagram of a file processing method of a computing network in an embodiment of the present application. In the process of integrating files, the multi-cloud file management module determines whether any file in the target database meets the preset integration conditions.

[0106] Then, the multi-cloud file management module generates an integration strategy when the integration conditions are met.

[0107] Next, the multi-cloud file management module calls the multi-cloud file plug-in layer, and performs the file integration operation on the cloud through the multi-cloud file plug-in layer.

[0108] For example, the preset integration condition for any file may be whether the number of duplicate versions of any file exceeds a duplicate threshold. When the multi-cloud file management module detects that the number of duplicate versions of any file exceeds the duplicate threshold, a duplicate integration strategy is generated. The duplicate integration strategy may be to retain the file version with the most recent last modification time and delete the remaining duplicate versions.

[0109] The preset integration condition for any file may be that the number of storage areas for any file exceeds the storage threshold. When the multi-cloud file management module detects that the number of storage areas for any file exceeds the storage threshold, a storage integration strategy is generated. The repeated integration strategy may be to randomly keep one storage area of ​​the file unchanged and delete the files in the remaining storage areas.

[0110] The preset integration condition for any file may be that the number of derivative versions of any file exceeds a derivative threshold. When the multi-cloud file management module detects that the number of derivative versions of any file exceeds the derivative threshold, a derivative integration strategy is generated. The derivative integration strategy may be to package all derivative versions of the file and place them in a target storage location.

[0111] The system and method provided by this embodiment have at least the following effects: 1. Collect metadata information of any file from various clouds, edge nodes, and data centers in the computing network, and then analyze the version relationship and lineage relationship of the file, provide a traceable file view, support multi-condition search, and view the duplicate and derivative versions of the file in each cloud, as well as the lineage relationship of each version, so as to quickly find the target file and target version.

[0112] 2. Automatically identify duplicate and derivative versions of any file on each cloud and edge node, and reduce overall storage costs through storage consolidation and optimization.

[0113] Reference Figure 6 , shows a schematic diagram of a computer device provided in an embodiment of the present application, including: at least one processor 601, and a memory 602, wherein the memory 602 stores a computer program that can be run on the processor 601, wherein the processor 601 executes the file processing method of the computing power network described in the embodiment when executing the computer program.

[0114] When the processor 601 executes the computer program, the file processing method of the computing power network described in the embodiment is executed, including: obtaining the metadata information of any file on each cloud and each edge node of the computing power network and storing it in the target database, and obtaining the task information associated with any file from the task manager of the computing power network, determining the duplicate version, derivative version and blood relationship of any file and storing it in the target database; obtaining the metadata information, duplicate version, derivative version and blood relationship of any file on each cloud and each edge node of the computing power network in the target database, and generating a traceable file view.

[0115] It also includes executing: the multi-cloud file management module responds to the multi-condition search request and returns the target file and target version corresponding to the multi-condition search request; the search conditions of the multi-condition search request include at least one of the file name, label, modification time, generation source, any task using the file, and migration record; it also includes executing: the multi-cloud file management module can also monitor whether any file meets the preset integration conditions; when any file meets the integration conditions, the multi-cloud file plug-in layer is called to integrate the file in each cloud and each edge node of the computing power network.

[0116] Reference Figure 7 , showing a schematic diagram of a non-volatile readable storage medium provided in an embodiment of the present application, wherein the non-volatile readable storage medium 700 stores a computer program 701, wherein the computer program 701, when executed by a processor, executes the file processing method of the computing power network described in the embodiment.

[0117] When the computer program 701 is executed by the processor to perform the file processing method of the computing power network described in the embodiment, it specifically includes the following executions: obtaining the metadata information of any file on each cloud and each edge node of the computing power network and storing it in the target database, and obtaining the task information associated with any file from the task manager of the computing power network, determining the duplicate version, derivative version and blood relationship of any file and storing it in the target database; obtaining the metadata information, duplicate version, derivative version and blood relationship of any file on each cloud and each edge node of the computing power network in the target database, and generating a traceable file view.

[0118] It also includes executing: the multi-cloud file management module responds to the multi-condition search request and returns the target file and target version corresponding to the multi-condition search request; the search conditions of the multi-condition search request include at least one of the file name, label, modification time, generation source, any task using the file, and migration record.

[0119] It also includes execution: the multi-cloud file management module can also monitor whether any file meets the preset integration conditions; when any file meets the integration conditions, the multi-cloud file plug-in layer is called to integrate the file in each cloud and each edge node of the computing power network.

[0120] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0121] It should be understood by those skilled in the art that the embodiments of the present application can be provided as methods, devices or computer program products. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0122] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0123] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0124] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0125] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0126] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0127] Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A file system for a computing network, characterized in that: The system includes a file metadata management module, a multi-cloud file manager module, and a target database, wherein: The file metadata management module is used to obtain metadata information of any file on each cloud and each edge node of the computing network and store it in the target database, and obtain task information associated with any file from the task manager of the computing network, determine the duplicate version, derivative version and blood relationship of any file and store it in the target database; The multi-cloud file management module is used to obtain metadata information, duplicate versions, derivative versions and blood relationships of any file on each cloud and each edge node of the computing power network in the target database, and generate a traceable file view.

2. The system according to claim 1, characterized in that The metadata information of any file includes basic information, modification records, tag lists, access records, and migration records, among which: The basic information includes file name, file size, creation time, last modification time, and checksum; The modification record includes a version list and basic information of each version; The tag list includes custom tags and tags generated by the storage system on each cloud and each edge node; The access record includes an access record list, and any access record in the access record list includes an access time and an access source; The migration record includes a migration record list, and any migration record in the migration record list includes source storage, target storage, migration time, and migration time consumption.

3. The system according to claim 1, characterized in that Task information associated with any file, including the source and usage list; The generation source is the task that generates the file; The usage list includes all tasks that use the file.

4. The system according to claim 1, characterized in that The system further comprises: The multi-cloud file plug-in layer is used to provide the file metadata management module with a plug-in for accessing the file services of each cloud of the computing power network, and the plug-in is used to access the OpenAPI of each cloud of the computing power network.

5. The system according to claim 1, characterized in that The system further comprises: A file management agent is deployed in a data center that does not have an OpenAPI in the computing network; The file management agent is used to obtain metadata information of any file in the data center and report it to the file metadata management module.

6. The system according to claim 1, characterized in that The multi-cloud file management module is also used for: In response to a multi-condition search request, returning a target file and a target version corresponding to the multi-condition search request; The search conditions of the multi-condition search request include at least one of a file name, a tag, a modification time, a generation source, any task using the file, and a migration record.

7. The system according to claim 4, characterized in that The multi-cloud file management module is also used for: Monitor whether any file meets the preset integration conditions; When any file meets the integration condition, the multi-cloud file plug-in layer is called to integrate the file in each cloud and each edge node of the computing power network.

8. A file processing method for a computing network, characterized in that: The method comprises: Obtaining metadata information of any file on each cloud and each edge node of the computing network and storing it in a target database, obtaining task information associated with any file from a task manager of the computing network, determining duplicate versions, derivative versions, and lineage relationships of any file and storing them in the target database; The metadata information, duplicate versions, derivative versions and lineage relationships of any file on each cloud and each edge node of the computing network are obtained in the target database, and a traceable file view is generated.

9. A computer device, characterized in that: include: At least one processor, and a memory, wherein the memory stores a computer program that can be run on the processor, wherein the processor executes the file processing method of the computing power network as described in claim 8 when executing the computer program.

10. A non-volatile readable storage medium, characterized in that: The non-volatile readable storage medium stores a computer program, wherein the computer program, when executed by a processor, executes the file processing method of the computing power network described in claim 8.

Images