Digital networking scheduling system and method for file type digital object

Abstract

The invention provides a file type digital object-oriented data networking scheduling system and method, relates to the technical field of data networking, and aims to solve the problem of heterogeneous access of multi-protocol and multi-type data resources in a data networking environment. The system comprises a client access layer, a core scheduler and a data warehouse interface, wherein the client access layer is used for calling a request initiated by a user; the data warehouse interface is used for accessing different data warehouses; the core scheduler is used for acquiring metadata of the target file type digital object according to a calling request, and determining a transmission strategy according to a transmission type field in the metadata of the target file type digital object, and writing the body of the target file type digital object from a data warehouse to a storage position corresponding to the position field according to the transmission strategy.

Description

Technical Field

[0001] The present application relates to the field of digital networking technology, and in particular to a digital networking scheduling system and method for file-type digital objects. Background Art

[0002] A digital object (DO) refers to a digital entity that can be uniquely identified and manipulated in cyberspace. A digital object typically consists of a data entity and its associated metadata, and is assigned a globally unique and persistent identifier. Under the Digital Object Architecture (DOA) framework, a digital object's identifier (e.g., Handle or DOI) is associated with the object's storage location through a registry, enabling data location and retrieval.

[0003] Traditional digital object models focus on abstract definitions and are inadequate for file data encapsulation, lacking clear descriptions of file formats, sizes, and transmission methods. Furthermore, current scientific data sharing and scheduling processes face challenges such as high data heterogeneity, complex protocol adaptation, inconsistent identifier resolution, and low automation. However, much data acquisition currently relies on manual processes, requiring users to manually parse data identifiers to locate storage locations, select transmission protocols, and download files. This is time-consuming and error-prone, hindering data interoperability and acquisition efficiency and hindering the automation of large-scale scientific data sharing.

[0004] Therefore, how to solve the problem of heterogeneous access to multi-protocol and multi-type data resources in a digital network environment is a technical problem that needs to be solved urgently. Summary of the Invention

[0005] In view of the above problems, embodiments of the present application provide a digital network scheduling system and method for file-type digital objects to overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect of an embodiment of the present application, a data network scheduling system for file-type digital objects is disclosed, the system comprising: The client access layer is used for user-initiated call requests, wherein the call request includes a target identifier and a location field, wherein the target identifier represents the identifier of the target file-type digital object to be accessed, and the location field represents the desired storage location; wherein the metadata of the file-type digital object includes an identifier, a transmission type field, and a file content type field; a core scheduler configured to obtain metadata of the target file-type digital object according to the call request, determine a transmission strategy based on a transmission type field in the metadata of the target file-type digital object, and write the body of the target file-type digital object from a data repository to a storage location corresponding to the location field according to the transmission strategy, wherein the transmission strategy includes a transmission protocol and a transmission method; The data warehouse interface is used to access different data warehouses, wherein the data warehouses store the ontology of file-type digital objects.

[0007] Optionally, the transmission type field includes a transmission protocol and a transmission method; and the core scheduler includes: A transmission type loading module is used to send a retrieval request to the data provider based on the target identifier, obtain the metadata of the target file-type digital object, and load the corresponding transmission processing class instance based on the transmission protocol-transmission method corresponding to the transmission type field in the metadata of the target file-type digital object. The transmission processing class instance represents the transmission strategy.

[0008] Optionally, the transmission type loading module is also used to determine whether the identifier in the metadata of the target file type digital object is consistent with the target identifier in the call request; and if consistent, determine the file type information based on the file content type field in the metadata of the target file type digital object.

[0009] Optionally, the location field includes a prefix flag and a path, the prefix flag is used to indicate a storage mode, and the path is used to indicate a storage location; the core scheduler includes: The storage manager module is used to determine the path scheduling process according to the prefix flag and the path, and write the body of the target file-type digital object into the storage location according to the transmission strategy and the path scheduling process.

[0010] Optionally, the prefix flag includes a local flag, a distributed file system flag, and a cloud flag; the path scheduling process includes a local download processing process, a distributed file system upload process, and a cloud storage upload process; and the storage manager module is specifically configured to execute: In the case where the prefix flag is a local flag, starting a local download processing flow, and downloading the body of the target file-type digital object to the local storage location specified by the path according to the transmission strategy; In the case where the prefix flag is a distributed file system flag, a distributed file system upload process is started, and the body of the target file-type digital object is uploaded to the storage location of the distributed file system specified by the path according to the transmission strategy; In the case where the prefix mark is a cloud mark, the cloud storage upload process is started, and the body of the target file-type digital object is uploaded to the storage location of the cloud storage platform specified by the path according to the transmission strategy.

[0011] Optionally, the storage manager module is further configured to determine the storage location based on the path and an identifier in the metadata of the target file-type digital object.

[0012] Optionally, the core scheduler further includes: The network card server entry module monitors the client access layer through a cross-language remote service call framework or a hypertext transfer protocol server, and is used to receive a call request from the client access layer.

[0013] Optionally, the client access layer is constructed based on a cross-language remote service call framework, and the client access layer includes: File download interface, used by users to initiate call requests.

[0014] Optionally, the client access layer further includes at least one of the following interfaces: Download progress query interface, used to query the current progress of the calling task corresponding to the target identifier; The storage path query interface is used to return the actual storage location of the target file-type digital object if the call is successful.

[0015] A second aspect of the embodiments of the present application discloses a data network scheduling method for file-type digital objects, which is applied to the data network scheduling system for file-type digital objects described in the first aspect of the embodiments of the present application. The method comprises: Obtaining a call request initiated by a user, the call request including a target identifier and a location field, wherein the target identifier represents an identifier of a target file-type digital object to be accessed, and the location field represents a desired storage location; wherein metadata of the file-type digital object includes an identifier, a transmission type field, and a file content type field; Acquire metadata of the target file-type digital object according to the call request, and determine a transmission strategy according to a transmission type field in the metadata of the target file-type digital object, wherein the transmission strategy includes a transmission protocol and a transmission method; According to the transmission strategy, the body of the target file-type digital object is written from the data warehouse to the storage location corresponding to the location field.

[0016] The embodiments of the present application include the following advantages: In an embodiment of the present application, a digital network scheduling system for file-type digital objects achieves a comprehensive description of file data by encapsulating it in a standardized manner as a file-type digital object. The metadata of the file-type digital object includes an identifier, a transmission type field, and a file content type field. This eliminates ambiguity in file data access and improves retrieval and parsing accuracy. Furthermore, for different types of file-type digital objects, the digital object identification resolution system can automatically map the identifier in the call request to a corresponding transmission strategy, which includes a transmission protocol and a transmission method. Based on this transmission strategy, the file data (the main body of the file-type digital object) is written from the data warehouse to the storage location corresponding to the location field. This enables flexible scheduling of heterogeneous data in a digital network environment with multiple transmission protocols and multiple types of data resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 This is a structural diagram of a data network scheduling system for file-type digital objects provided by an embodiment of the present application; Figure 2 This is a schematic diagram of the interaction flow of a file-type digital object-oriented digital network scheduling system provided by an embodiment of the present application; Figure 3 This is a flowchart of the steps of a data network scheduling method for file-type digital objects provided by an embodiment of the present application. DETAILED DESCRIPTION

[0019] To make the above-mentioned purposes, features, and advantages of this application more clearly understood, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of this application.

[0020] Reference Figure 1 As shown, Figure 1 : This is a schematic diagram of the structure of a data network scheduling system for file-type digital objects provided by an embodiment of the present application. The data network scheduling system for file-type digital objects specifically includes: The client access layer is used for user-initiated call requests, wherein the call requests include a target identifier and a location field. The target identifier represents the identifier of the target file-type digital object to be accessed, and the location field represents the desired storage location. The metadata of the file-type digital object includes an identifier, a transmission type field, and a file content type field.

[0021] A core scheduler is used to obtain the metadata of the target file-type digital object according to the call request, determine the transmission strategy according to the transmission type field in the metadata of the target file-type digital object, and write the body of the target file-type digital object from the data warehouse to the storage location corresponding to the location field according to the transmission strategy. The transmission strategy includes a transmission protocol and a transmission method.

[0022] The data warehouse interface is used to access different data warehouses, wherein the data warehouses store the ontology of file-type digital objects.

[0023] In this embodiment of the present application, the file data provided by the data provider is uniformly encapsulated as a file-type digital object in the digital network. This file-type digital object uses an identifier as an index and uses specific fields to achieve self-describing encapsulation of the file data. Specifically, the core fields of the file-type digital object include the identifier, the transmission type field, and the file content type field. These fields constitute the metadata of the file-type digital object, thus addressing the lack of detailed description of the data in general digital objects.

[0024] The identifier can be used to globally locate the target file-type digital object (target file data). In a digital network environment, the identifier is assigned by the identity resolution system. This system uses this identifier as a reference for requests to locate and control access to the file object. The file content type field (contentType) describes the type of file data. For example, application / pdf represents a PDF (Portable Document Format) document, and image / png represents a PNG (Portable Network Graphics) image. The file content type field allows the system to clearly understand the file's data format so that it can take appropriate action. For example, after the transfer is complete, the subsequent parsing or preview strategy can be determined based on the file content type field. The transfer type field (transferType) describes the file's transfer method (i.e., indicates how to obtain the file data). The transfer type field allows the system to programmatically understand how to obtain the file, providing a basis for subsequent automated scheduling.

[0025] The client access layer is a unified interface that supports multilingual clients. Through the client access layer, users can invoke the system's scheduling services using various languages or tool-line commands. Specifically, when initiating a call request, users only need to provide two core parameters: a target identifier and a location field to implement scheduling. The target identifier represents the identifier of the target file-type digital object to be accessed, and the corresponding target file-type digital object can be retrieved through the target identifier; the location field represents the desired storage location, such as a local storage path, a cloud storage address, or a distributed file system storage address.

[0026] The core scheduler is used to implement the specific calling process of file-type digital objects. The core scheduler can retrieve the target file-type digital object based on the target identifier in the scheduling request, and obtain the metadata of the target file-type digital object. The metadata includes an identifier, a transmission type field, and a file content type field. The core scheduler can determine the specific transmission strategy (i.e., the transmission protocol and transmission method) through the transmission type field, and then start data transmission to transmit the body of the target file-type digital object (i.e., the actual file data content) to the storage location specified by the user through the location field.

[0027] The system can access different data repositories through the data repositories interface to retrieve the corresponding file-type digital object ontology from the data repositories and write it to the storage location corresponding to the location field. The data repositories store the ontology of the file-type digital object and also declare the supported transmission types based on the service capabilities.

[0028] Through the above implementation process, the digital network scheduling system for file-type digital objects achieves a full description of file data by encapsulating file data into file-type digital objects in a standardized manner, including an identifier, a transmission type field, and a file content type field in the metadata of the file-type digital objects. This eliminates ambiguity in the file data access process and improves retrieval and parsing accuracy. In addition, for different types of file-type digital objects, the digital object identification resolution system can automatically map the identifier in the call request to the corresponding transmission strategy, which includes a transmission protocol and a transmission method, so that the file data (the body of the file-type digital object) is written from the data warehouse to the storage location corresponding to the location field according to the transmission strategy. This achieves flexible scheduling of heterogeneous data in a digital network environment with multiple transmission protocols and multiple types of data resources.

[0029] The following describes the "client access layer" and "core scheduler" in detail in two parts.

[0030] 1. Client access layer: In an optional embodiment, the client access layer is constructed based on a cross-language remote service call framework, and the client access layer includes: a file download interface for users to initiate call requests.

[0031] In this embodiment, a unified, cross-language SDK (Software Development Kit), or client access layer, is provided to facilitate access to the system's scheduling functionality by various clients. The client access layer encapsulates the system's network communication details and provides a simple, easy-to-use calling interface, allowing users to access the scheduling service in a familiar programming environment, much like calling a native library.

[0032] The cross-language remote service call framework refers to the Apache Thrift framework, which is designed to simplify communication between different programming languages. Specifically, based on this cross-language remote service call framework, the service interface is described using an interface definition language, and client and server code in multiple languages is generated to form the client access layer. The core scheduler implements the specific logic by integrating the client access layer's interfaces. In this way, clients in different languages call SDK methods (client access layer) in their respective languages, which are serialized and transmitted over the network via Apache Thrift, and the core scheduler ultimately completes the actual call process.

[0033] It is understandable that the client access layer can support different programming languages. No matter what programming language is used, the client access layer provides the user with consistent functions, that is, the provided interface names and parameters are consistent for different programming languages.

[0034] The client access layer includes a file download interface, which is used by users to initiate a call request, for example, to initiate a download request for a file-type digital object. In this case, the user (caller) only needs to provide the identifier and location fields of the target file-type digital object to be accessed. After calling this method, the client access layer forwards the call request to the core scheduler and waits for the return result. The return result typically includes the operation status and related information; for example, a successful call may return the identifier or path of the downloaded file.

[0035] Furthermore, the client access layer further includes at least one of the following interfaces: Download progress query interface, used to query the current progress of the calling task corresponding to the target identifier; The storage path query interface is used to return the actual storage location of the target file-type digital object if the call is successful.

[0036] In embodiments of the present application, when querying the current progress through the download progress query interface, the return value may be a percentage (0-100) or a status description. Because the core scheduler maintains the progress of each calling task, calling the download progress query interface requests the latest progress value from the core scheduler. In some embodiments, users can periodically call this download progress query interface to obtain real-time progress, thereby displaying a progress bar on the interface or determining when to process subsequent logic.

[0037] The storage path query interface is typically used after a download is complete (a successful call) to return the actual storage location of the target file-type digital object. For example, for a local download, this returns the absolute path of the local file; for a cloud download, it returns the URL of the file generated on the cloud. Through the storage path query interface, users can determine the location of the result file (i.e., the target file-type digital object), allowing them to perform subsequent operations such as opening and analysis.

[0038] It's understandable that the storage path query API will only return valid results if the download is complete and the file path is registered with the core scheduler. Otherwise, it may return a "not found" status. Therefore, it's usually necessary to wait for progress queries or callback notifications before calling this API.

[0039] In some embodiments, in addition to the above-mentioned main interfaces, the client access layer also includes auxiliary function interfaces, such as a download cancellation interface and a current active task list interface.

[0040] Through the above implementation process, a client access layer is built based on a cross-language remote service call framework, providing a set of concise service interfaces, such as file download interface, download progress query interface, storage path query interface, etc., so as to facilitate various clients to use the scheduling function of this system.

[0041] 2. Core scheduler: In an optional embodiment, the core scheduler further includes: a network card server entry module, which monitors the client access layer through a cross-language remote service call framework or a hypertext transfer protocol server, and is used to receive call requests from the client access layer.

[0042] In the embodiments of the present application, the network card server entry module is a unified access interface and serves as the entry point for all call requests. Specifically, the network card server entry module is responsible for receiving call requests (e.g., download requests, download progress query requests, etc.) from various interfaces within the client access layer. The network card server entry module monitors the client access layer via a cross-language remote service call framework (Apache Thrift framework) or a Hypertext Transfer Protocol server (HTTP Server) to receive call requests from the client access layer, thereby completing basic tasks such as call request reception, parsing, logging, and exception capture. Furthermore, the network card server entry module shields network communication details from external devices while providing a unified scheduling context internally, acting as a bridge between the external and internal interfaces.

[0043] In an optional embodiment, the transmission type field includes a transmission protocol and a transmission method; and the core scheduler includes: A transmission type loading module is used to send a retrieval request to the data provider based on the target identifier, obtain the metadata of the target file-type digital object, and load the corresponding transmission processing class instance based on the transmission protocol-transmission method corresponding to the transmission type field in the metadata of the target file-type digital object. The transmission processing class instance represents the transmission strategy.

[0044] The "transfer protocol" field indicates the transmission protocol used. Examples include the Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), and Digital Object Interface Protocol (DOIP). The "transfer method" field further specifies the transmission method. For example, "direct" indicates direct download.

[0045] For example, http:direct indicates that the file can be downloaded directly via HTTP, with the download URL provided by the data provider; doip:direct indicates that the file data will be retrieved directly via the DOIP protocol, with the data embedded in the DOIP response. Furthermore, composite transfer methods can also be described using the transfer type field. For example, ftp:auth indicates that FTP authentication is required before transfer.

[0046] In an embodiment of the present application, different file-type digital objects in the digital network may support different protocol transmissions. For the target file-type digital object to be accessed, its metadata includes a transmission type field. By parsing the transmission type field, the transmission strategy of the target file-type digital object (i.e., the transmission processing class instance) can be determined.

[0047] In practice, to implement a multi-protocol transport adaptation mechanism, a factory method can be used to enable the transport type loading module to support the selection of different transport strategies. Specifically, for each transport type field associated with a transport protocol and method combination, a corresponding transport handler class is implemented in the system. This class is abstracted into a unified interface or parent class to ensure that when a service is called, a corresponding transport handler class instance is automatically created given a transport type field, and its methods are called to complete data acquisition.

[0048] Furthermore, the transmission type loading module is also used to determine whether the identifier in the metadata of the target file type digital object is consistent with the target identifier in the call request; and if consistent, determine the file type information based on the file content type field in the metadata of the target file type digital object.

[0049] In this embodiment of the present application, after obtaining the metadata of the target file-type digital object, the transfer type loading module reads the identifier in the metadata and verifies whether the identifier in the metadata of the target file-type digital object is consistent with the target identifier in the call request to ensure that the data corresponds correctly. It then reads the file content type field in the metadata to determine the file type virtualization for logging or subsequent processing. For example, different post-processors (i.e., post-processing methods) can be selected for different types of file data.

[0050] Through the above implementation process, "strategy selection" and "strategy implementation" are separated. Through it, the system achieves integrated support for transmission methods such as HTTP and DOIP, meeting the data scheduling needs in a multi-protocol environment.

[0051] In an optional embodiment, the location field includes a prefix flag and a path, the prefix flag is used to indicate a storage mode, and the path is used to indicate a storage location; the core scheduler includes: The storage manager module is used to determine the path scheduling process according to the prefix flag and the path, and write the body of the target file-type digital object into the storage location according to the transmission strategy and the path scheduling process.

[0052] Among them, the path scheduling process refers to the routing and distribution process of data between different paths. Before starting data transmission, the storage manager module in the system prepares the storage location based on the location field in the call request.

[0053] Furthermore, the storage manager module is further configured to determine the storage location based on the path and an identifier in the metadata of the target file-type digital object.

[0054] In an embodiment of the present application, the storage location needs to be prepared according to the identifier in the metadata, that is, the "cache directory + identifier" is used to generate a path scheduling process to ensure that files of different call requests are stored separately to avoid conflicts.

[0055] Specifically, the prefix flag includes a local flag, a distributed file system flag, and a cloud flag; the path scheduling process includes a local download processing process, a distributed file system upload process, and a cloud storage upload process; the storage manager module is specifically used to perform the following items A-1 to A-3: Item A-1: When the prefix flag is a local flag, start the local download processing flow and download the body of the target file-type digital object to the local storage location specified by the path according to the transmission strategy.

[0056] Item A-2: When the prefix flag is a distributed file system flag, start the distributed file system upload process and upload the body of the target file-type digital object to the storage location of the distributed file system specified by the path according to the transmission strategy.

[0057] Item A-3: When the prefix mark is a cloud mark, the cloud storage upload process is started, and the body of the target file-type digital object is uploaded to the storage location of the cloud storage platform specified by the path according to the transmission strategy.

[0058] In an embodiment of the present application, the storage manager module and the transmission type loading module work together. After the transmission type loading module returns the transmission data (the body of the target file type digital object), the storage manager module selects the corresponding path scheduling process according to the prefix flag of the location field, and connects the data pipeline generated during the transmission process to the process.

[0059] Specifically, if the prefix flag is "local," the storage manager module executes the local download process described in item A-1, directly writing the target file-type digital object to the local storage location specified by the path, without requiring any additional processing. This local download process is typically used when the client and system are on the same host or within the same intranet environment, enabling the write to be completed with minimal latency.

[0060] If the prefix flag is the distributed file system flag (hdfs), the storage manager module executes the distributed file system upload process of item A-2. Specifically, uploading the body of the target file-type digital object to the storage location of the distributed file system specified by the path includes two implementation methods. One is to directly write to the distributed file system during the download process without landing locally, and the other is to first download to a local temporary file and then upload to the distributed file system.

[0061] If the prefix flag is a cloud flag, indicating that a third-party cloud storage service needs to be invoked, the storage manager module executes the cloud storage upload process of item A-3. Similarly, uploading the target file-type digital object to the storage location of the cloud storage platform specified by the path includes two implementation methods: one is to directly write to the cloud storage service during the download process without saving it locally, and the other is to first download it to a local temporary file and then upload it to the cloud storage service. The storage location of the cloud storage platform may include information such as the storage bucket name. In some embodiments, after uploading the target file-type digital object to the storage location of the cloud storage platform specified by the path, a cloud URL (Uniform Resource Locator) or confirmation information is obtained and returned to the user for subsequent access.

[0062] In some embodiments, if the prefix flag is the Science Data Center flag (scidatacenter), indicating that the storage location is a Science Data Center platform, the Science Data Center platform upload process is initiated to upload the target file-type digital object's body to the Science Data Center platform storage location specified by the path. Specifically, the target file-type digital object's body can be pushed to the Science Data Center platform via an Application Programming Interface (API) provided by the Science Data Center platform.

[0063] Through this implementation, the storage manager module and location fields enable direct translation from user intent to system action, enabling on-demand delivery. This fully embodies the user-centric design philosophy, where user needs are clearly communicated and the system is responsible for adapting and executing accordingly. For users, this simply requires adding a parameter to the download call, eliminating the need to worry about the complex underlying network and storage logic. For the system, this increases flexibility, making it easy to expand new storage backends (by simply adding new prefix flag branching logic).

[0064] The following combination Figure 2 The interactive process of the data network scheduling system for file-type digital objects is shown to illustrate the data network scheduling system for file-type digital objects provided by the present application. Figure 2 As shown in the figure, the specific interaction process is as follows: First, the client (user) initiates a call request through the file download interface of the client access layer. The call request includes a target identifier and a location field. Next, the network card server entry module of the core scheduler receives the call request, and the transmission type loading module sends a search request to the data provider based on the target identifier. For example, it sends a DIOP search request to the data network to obtain the metadata of the target file-type digital object and loads the corresponding transmission processing class instance (transmission strategy) based on the transmission protocol and transmission method corresponding to the transmission type field in the metadata of the target file-type digital object. Then, the storage manager module determines the path scheduling process based on the prefix flag and path of the location field and executes the data download. That is, according to the transmission strategy and path scheduling process, the target file-type digital object's body is written from the data warehouse to the storage location.

[0065] When the core scheduler begins the data call process, it can return a response to the client indicating that the task has started. At this point, the client can poll the client access layer's download progress query interface to query the current progress of the call task corresponding to the target identifier. Once the progress reaches 100% or the system notifies the task is complete, the client can call the storage path query interface to obtain the actual file storage location (location path). After obtaining the storage location, the client can access or further process the file data.

[0066] In this way, through the digital network scheduling system for file-type digital objects, users do not need to worry about how to parse the file-type digital object (FileDO) internally, how to select the transmission protocol, how to download and save, and can complete the entire process by simply calling the interface of the client access layer in sequence.

[0067] The embodiment of the application also provides a data network scheduling method for file-type digital objects, which is applied to the data network scheduling system for file-type digital objects described in the above embodiment. Figure 3 As shown, Figure 3 This is a flowchart of a method for scheduling file-type digital objects on a data network provided by an embodiment of the present application. The method includes steps 310 to 320: Step S310: Obtaining a call request initiated by a user, the call request including a target identifier and a location field, wherein the target identifier represents an identifier of a target file-type digital object to be accessed, and the location field represents a desired storage location; wherein metadata of a file-type digital object includes an identifier, a transmission type field, and a file content type field; Step S320: acquiring metadata of the target file-type digital object according to the call request, and determining a transmission strategy according to a transmission type field in the metadata of the target file-type digital object, wherein the transmission strategy includes a transmission protocol and a transmission method; Step S330: writing the body of the target file-type digital object from the data warehouse to the storage location corresponding to the location field according to the transmission strategy.

[0068] In the embodiment of the present application, file data is standardized and encapsulated into a file-type digital object. The metadata of the file-type digital object includes an identifier, a transmission type field, and a file content type field. This fully describes the file data, eliminates ambiguity in the file data access process, and improves the accuracy of retrieval and parsing. In addition, for different types of file-type digital objects, the digital object identification resolution system can automatically map the identifier in the call request to the corresponding transmission strategy, which includes a transmission protocol and a transmission method, so that the file data is written from the data warehouse to the storage location corresponding to the location field according to the transmission strategy. This achieves flexible scheduling of heterogeneous data in a digital network environment with multiple transmission protocols and multiple types of data resources.

[0069] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0070] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the systems and methods according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0071] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0073] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0074] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0075] The above is a detailed introduction to a digital network scheduling system and method for file-type digital objects provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data network scheduling system for file-type digital objects, characterized by: include: The client access layer is used for user-initiated call requests, wherein the call request includes a target identifier and a location field, wherein the target identifier represents the identifier of the target file-type digital object to be accessed, and the location field represents the desired storage location; wherein the metadata of the file-type digital object includes an identifier, a transmission type field, and a file content type field; a core scheduler configured to obtain metadata of the target file-type digital object according to the call request, determine a transmission strategy based on a transmission type field in the metadata of the target file-type digital object, and write the body of the target file-type digital object from a data repository to a storage location corresponding to the location field according to the transmission strategy, wherein the transmission strategy includes a transmission protocol and a transmission method; The data warehouse interface is used to access different data warehouses, wherein the data warehouses store the ontology of file-type digital objects.

2. The file-type digital object-oriented data network scheduling system according to claim 1, characterized in that: The transmission type field includes a transmission protocol and a transmission method; the core scheduler includes: A transmission type loading module is used to send a retrieval request to the data provider based on the target identifier, obtain the metadata of the target file-type digital object, and load the corresponding transmission processing class instance based on the transmission protocol-transmission method corresponding to the transmission type field in the metadata of the target file-type digital object. The transmission processing class instance represents the transmission strategy.

3. The file-oriented digital object data network scheduling system according to claim 2, characterized in that: The transmission type loading module is also used to determine whether the identifier in the metadata of the target file type digital object is consistent with the target identifier in the call request; and if consistent, determine the file type information based on the file content type field in the metadata of the target file type digital object.

4. The file-type digital object-oriented data network scheduling system according to claim 1 or 2, characterized in that: The location field includes a prefix flag and a path, the prefix flag is used to indicate a storage mode, and the path is used to indicate a storage location; the core scheduler includes: The storage manager module is used to determine the path scheduling process according to the prefix flag and the path, and write the body of the target file-type digital object into the storage location according to the transmission strategy and the path scheduling process.

5. The file-type digital object-oriented data network scheduling system according to claim 4 is characterized in that: The prefix flag includes a local flag, a distributed file system flag, and a cloud flag. The path scheduling process includes a local download processing process, a distributed file system upload process, and a cloud storage upload process. The storage manager module is specifically used to execute: In the case where the prefix flag is a local flag, starting a local download processing flow, and downloading the body of the target file-type digital object to the local storage location specified by the path according to the transmission strategy; In the case where the prefix flag is a distributed file system flag, a distributed file system upload process is started, and the body of the target file-type digital object is uploaded to the storage location of the distributed file system specified by the path according to the transmission strategy; In the case where the prefix mark is a cloud mark, the cloud storage upload process is started, and the body of the target file-type digital object is uploaded to the storage location of the cloud storage platform specified by the path according to the transmission strategy.

6. The file-type digital object-oriented data network scheduling system according to claim 4, characterized in that: The storage manager module is further configured to determine the storage location based on the path and an identifier in the metadata of the target file-type digital object.

7. The file-type digital object-oriented data network scheduling system according to claim 1 or 2, characterized in that: The core scheduler further includes: The network card server entry module monitors the client access layer through a cross-language remote service call framework or a hypertext transfer protocol server, and is used to receive a call request from the client access layer.

8. The file-oriented digital object data network scheduling system according to claim 1, characterized in that: The client access layer is built based on a cross-language remote service call framework and includes: File download interface, used by users to initiate call requests.

9. The file-type digital object-oriented data network scheduling system according to claim 8, characterized in that: The client access layer further includes at least one of the following interfaces: Download progress query interface, used to query the current progress of the calling task corresponding to the target identifier; The storage path query interface is used to return the actual storage location of the target file-type digital object if the call is successful.

10. A data network scheduling method for file-type digital objects, characterized in that: The method applied to the file-type digital object-oriented data network scheduling system according to any one of claims 1 to 9 comprises: Obtaining a call request initiated by a user, the call request including a target identifier and a location field, wherein the target identifier represents an identifier of a target file-type digital object to be accessed, and the location field represents a desired storage location; wherein metadata of the file-type digital object includes an identifier, a transmission type field, and a file content type field; Acquire metadata of the target file-type digital object according to the call request, and determine a transmission strategy according to a transmission type field in the metadata of the target file-type digital object, wherein the transmission strategy includes a transmission protocol and a transmission method; According to the transmission strategy, the body of the target file-type digital object is written from the data warehouse to the storage location corresponding to the location field.

Images