A file conversion engine based on cluster architecture intelligent scheduling and a building method

Abstract

The application discloses a file conversion engine based on intelligent scheduling of a cluster architecture and a building method, and relates to the technical field of file conversion engines. The file conversion engine is based on distributed file sharing storage and comprises a registration service center unit, a service gateway, an authentication center service unit, an authorization center service unit, a scheduling command center unit and a plurality of file conversion units. After a requestor sends a file conversion request, the file conversion engine can quickly convert through the corresponding units, thereby solving the technical problem of low conversion efficiency of a file conversion server in the prior art.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a file conversion engine based on cluster architecture intelligent scheduling and its construction method. Background Technology

[0002] Office automation involves the need for online previewing of various documents. Currently, most solutions on the market convert the source document to a uniform format before displaying it on the page when a user initiates a preview. This approach results in a significant difference between the preview and the original document, with low fidelity in document formatting, layout, and font reproduction. Furthermore, resource bottlenecks in the file conversion server lead to slow preview speeds and low efficiency, greatly degrading the user experience. A file conversion engine that can automatically identify resource bottlenecks in the file conversion server and automatically schedule a better file conversion service to improve the fidelity of the previewed document and increase preview speed can better meet the needs of online document previewing. Summary of the Invention

[0003] This invention provides a file conversion engine and its construction method based on intelligent scheduling of cluster architecture, in order to at least solve the technical problem of low conversion efficiency of file conversion servers in related technologies.

[0004] According to one aspect of the present invention, a file conversion engine based on cluster architecture intelligent scheduling and a method for building it are provided. The file conversion engine is based on distributed file shared storage and includes:

[0005] The registration service center unit is used for long-term connection and management and maintenance with the service gateway, authentication center unit, authorization center unit, dispatch and command center unit, and resource pool;

[0006] The service gateway is used to collect file conversion request traffic. After receiving a file conversion request, the service gateway will register it with the registration service center unit and also return the conversion result to the requester.

[0007] The authentication center service unit is used to authenticate the identity of the requester in the file conversion service request sent by the service gateway, and add the identity information to the file conversion request service after successful authentication. It is also used to receive the service processing level of the returned file conversion service request.

[0008] The authorization center service unit is used to authorize the identity of the requester in the file conversion service request sent by the authentication center service unit and determine the service processing level of the requester; after authorization is completed, the authorization data is added to the file conversion service request and sent.

[0009] The dispatch and command center unit is used to schedule file conversion service requests sent by the authorized center service unit, track the processing status of file conversion service requests in real time, and perform intelligent secondary scheduling, which is to schedule service node anomalies.

[0010] Multiple file conversion units, each performing only one type of file conversion task. The file conversion units perform conversions according to the scheduling tasks issued by the scheduling and command center unit and feed the conversion results back to the service gateway.

[0011] Optionally, it also includes an operation and maintenance monitoring backend system, which is used to visually display and manage the running status of the entire file conversion engine on a web page, providing a platform for manual processing.

[0012] Optionally, the scheduling and command center unit is also used to convert the local file storage path according to the URL of the file conversion service request, determine whether the source file to be converted in the file conversion service request exists locally, add the request to the conversion queue if it exists, otherwise download it, and notify the scheduling and command center unit to schedule the file conversion unit to process it after the download is successful.

[0013] Optionally, each unit of the file conversion engine is deployed and run based on Docker containerization and managed using Kubernetes.

[0014] Optionally, all file conversion requests first enter the service gateway.

[0015] According to another aspect of the present invention, a method for building a file conversion engine based on cluster architecture intelligent scheduling is also provided, including: building an underlying layer for distributed file sharing storage based on distributed technology, and deploying a registration service center unit, a service gateway, an authentication center service unit, an authorization center service unit, a scheduling command center unit and multiple file conversion units on the underlying layer respectively;

[0016] The registration service center unit is used for long-term connection and management and maintenance with the service gateway, authentication center unit, authorization center unit, dispatch and command center unit, and resource pool;

[0017] The service gateway is used to collect file conversion request traffic. After receiving a file conversion request, the service gateway will register with the registration service center unit and also return the conversion result to the requester.

[0018] The authentication center service unit is used to authenticate the identity of the requester in the file conversion service request sent by the service gateway, and add the identity information to the file conversion request service after successful authentication. It is also used to receive the service processing level returned for the file conversion service request.

[0019] The authorization center service unit is used to authorize the identity of the requester in the file conversion service request sent by the authentication center service unit and determine the service processing level of the requester; after authorization is completed, the authorization data is added to the file conversion service request and then sent.

[0020] The dispatch and command center unit is used to schedule file conversion service requests sent by the authorized center service unit, track the processing status of file conversion service requests in real time, and perform intelligent secondary scheduling, which is to schedule service node anomalies.

[0021] Each file conversion unit performs only one type of file conversion task. Each file conversion unit is assigned to a resource pool of different service levels. The file conversion unit performs conversion according to the scheduling task issued by the scheduling and command center unit and feeds back the conversion result to the service gateway.

[0022] Optionally, the system may also include an operation and maintenance monitoring backend system deployed on the underlying layer. The operation and maintenance monitoring backend system is used to visualize and manage the running status of the entire file conversion engine on a web page, providing a platform for manual processing.

[0023] Optionally, the underlying layer is used to provide shared file services for each unit of the file conversion engine, and to ensure that the files are distributed and highly available, enabling fast and consistent reading, writing, sharing, distribution, and verification of the files.

[0024] Optionally, the dispatch and command center unit is also used to convert the local file storage path according to the URL of the file conversion service request, determine whether the source file to be converted in the file conversion service request exists locally, add the request to the conversion queue if it exists, otherwise download it, and notify the dispatch and command center unit to schedule the file conversion unit to process it after the download is successful.

[0025] Optionally, each unit of the file conversion engine is deployed and run on Docker containers and managed using Kubernetes.

[0026] Compared with existing technologies, the present invention has the following advantages:

[0027] In this embodiment of the invention, the file conversion engine based on cluster architecture and intelligent scheduling is based on distributed file shared storage and includes: a registration service center unit, a service gateway, an authentication center service unit, an authorization center service unit, a scheduling and command center unit, and multiple file conversion units. File conversion requests can automatically identify resource bottlenecks on file conversion servers through the file conversion engine, that is, automatically identify which server resources are busy and which are idle, and then send the conversion request to the less busy server. The scheduling and command center unit automatically schedules the better file conversion unit for conversion. The fast conversion of the file conversion engine enables the online preview document to highly restore the format, layout, and font of the source document, while also solving the prominent problem of slow online document preview speed and improving the user experience. For example, some source documents, such as Word files, do not support being opened, viewed, or previewed directly in a browser. To open and view these files from a text-based system, the file must first be downloaded from the system, and then the corresponding software (such as Office) must be installed to open it, which requires two steps. However, if the file could be opened and viewed online directly through a web system or browser, the experience would be much better. Browsers are software that is installed by default after the user installs the operating system, so for the user, everything is installed and they can directly view Word files without having to download the file to their computer first. Attached Figure Description

[0028] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only one embodiment of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating a method for building a file conversion engine based on intelligent scheduling of a cluster architecture according to an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of the structure of a file conversion engine based on cluster architecture intelligent scheduling according to an embodiment of the present invention. Detailed Implementation

[0031] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0032] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] Example 1

[0035] According to an embodiment of the present invention, an embodiment of a method for building a file conversion engine based on intelligent scheduling of a cluster architecture is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0036] like Figure 1 This is a flowchart illustrating a method for building a file conversion engine based on intelligent scheduling of a cluster architecture according to an embodiment of the present invention, such as... Figure 1 As shown, the method for building a file conversion engine includes the following steps:

[0037] Step 1: Based on distributed technology, build the underlying layer for distributed file sharing storage. Specifically, the underlying layer provides shared file services for each unit of the file conversion engine and ensures that the files are distributed and highly available, so that the files will not be lost or damaged due to the sudden failure of some of the storage nodes (disks). Functionally, it mainly performs fast and consistent reading, writing, sharing, distribution and verification of files.

[0038] Step 2: Deploy the registration service center unit at the bottom layer. The registration service center unit is used to establish long-term connections and manage and maintain the service gateway, authentication center unit, authorization center unit, scheduling and command center unit, and resource pool.

[0039] Management and maintenance include module availability, current module load information, and service call addresses. Long connections are used to send real-time event notifications to each unit.

[0040] Step 3: Deploy a service gateway at the underlying level. All file conversion requests first enter the service gateway. The service gateway is used to collect file conversion request traffic and send the file conversion requests to the authentication service center unit for further processing. After receiving a file conversion request, the service gateway will register with the registration service center unit to facilitate information exchange between other units and the service gateway. The service gateway is also used to return the conversion result to the requester.

[0041] Step 4: Deploy the authentication center service unit at the underlying layer. The authentication center service unit is used to authenticate the requester's identity in the file conversion service request sent by the service gateway. After successful authentication, the identity information is added to the file conversion request service. If the authentication fails, the result is directly fed back to the service gateway. The authentication center service unit is also used to receive the service processing level returned for the file conversion service request.

[0042] The authentication method is adapted to the business scenario, selecting the most suitable authentication method. For example, some requesters provide JWT (JSON Web Token), and the authentication center uses JWT authentication; while some requesters have integrated with the OAuth 2.0 protocol, and the authentication center uses OAuth 2.0 authentication. After successful authentication, the identity information is included in the service request and sent to the subsequent authorization center service unit.

[0043] Step 5: Deploy the authorization center service unit at the bottom layer. The authorization center service unit is used to authorize the identity of the requester in the file conversion service request sent by the authentication center service unit and determine the service processing level of the requester; after authorization is completed, the authorization data is added to the file conversion service request and sent to the dispatch and command center.

[0044] Different response levels will be allocated different file conversion resources to ensure varying processing times and qualities. Another benefit of tiered request processing is that it maximizes the fulfillment of the needs of critical systems and key clients.

[0045] As an optional implementation, the service processing level classification is designed based on two factors: the importance of the customer and the importance of the business system. Customers are divided into four levels: Gold, Silver, Bronze, and General. Business systems are divided into two levels: Important and Non-Important. Therefore, a comprehensive service level score can be generated similar to the following:

[0046] Gold-level clients + key systems: 100

[0047] Gold-level clients + non-critical systems: 80

[0048] Silver Clients + Key Systems: 95

[0049] Silver-level clients + non-critical systems: 75

[0050] Bronze-level clients + key systems: 90

[0051] Bronze-level clients + non-critical systems: 70

[0052] General customers + critical systems: 85

[0053] General customers + non-critical systems: 65.

[0054] Step 6: The scheduling and command center unit is used to schedule file conversion service requests sent by the authorized center service unit, track the processing status of file conversion service requests in real time, and perform intelligent secondary scheduling, which is to schedule service node anomalies.

[0055] Specifically, the underlying layer maintains a message queue. The function and responsibility of the scheduling and command center unit is to maintain the position of each request in this message queue. The earlier the position, the higher the priority for processing. At the same time, based on the service level information of the file request, the resource pool allocation algorithm and the load balancing algorithm, the resource pool to process the request and the processing node in the pool are calculated.

[0056] The intelligent scheduling and command center is the core functional module of the file conversion engine. It constantly tracks the processing status of file conversion requests and performs intelligent secondary scheduling. Intelligent secondary scheduling involves handling service node anomalies. For example, if a request is pushed to a service node for processing, and that node unexpectedly stops working, the intelligent scheduling center will redirect the request to another available node.

[0057] As an optional embodiment, the dispatch and command center unit is also used to convert the local file storage path according to the URL of the file conversion service request, determine whether the source file to be converted in the file conversion service request exists locally, and add the request to the conversion queue if it exists to avoid repeated downloads and wasting resources; otherwise, if it does not exist, it will be downloaded, and after the download is successful, the dispatch and command center unit will be notified to schedule the file conversion unit to process it.

[0058] Step 7: Each file conversion unit performs only one type of file conversion task. Each file conversion unit is assigned to a resource pool of different service levels. The file conversion unit performs conversion according to the scheduling task issued by the scheduling and command center unit and feeds back the conversion result to the service gateway.

[0059] Specifically, each file conversion unit performs only one type of file conversion task. For example, converting an Office Word file to a PDF file is one type of file conversion task, while converting an Office Excel file to an HTML file is another type of file conversion service. These file conversion units are divided into resource processing pools of different service levels. Generally speaking, the higher the service level, the more conversion modules of each type are in the resource processing pool.

[0060] Each time, the file conversion unit retrieves a request to be converted from the conversion queue, calls the corresponding file conversion unit to perform the conversion based on the conversion type of the request, and notifies the scheduling center of the result regardless of whether the conversion is successful or not. If the conversion is successful, the converted file is uploaded to the unified file storage server.

[0061] By combining the file conversion unit with the scheduling center, the speed and success rate of file conversion can be improved more efficiently, reducing user waiting time.

[0062] As an optional embodiment, after receiving the intelligent dispatch center dispatch unit, if the file conversion unit identifies the source file as a compressed file such as rar or zip, it decompresses it, uploads the decompressed file to the unified file storage server and writes it into the decompression data table, and notifies the dispatch command center unit of the decompression result.

[0063] As an optional embodiment, the method for building a file conversion engine based on cluster architecture intelligent scheduling further includes step 8: deploying an operation and maintenance monitoring backend system on the underlying layer. The operation and maintenance monitoring backend system is used to visualize and manage the running status of the entire file conversion engine on a WEB page, providing a platform for manual processing, and allowing for certain manual intervention and error correction when necessary.

[0064] Specifically, the operation and maintenance monitoring backend system can fully present the complete link tracing process of a file conversion request, from entering the gateway service, then reaching various intermediate processing nodes such as the authentication center unit, authorization center unit, intelligent scheduling and command center unit, resource pool, and file conversion unit, and finally returning to the service gateway. Additionally, the processing process can be manually intervened; for example, when a high-level service request lacks processing resources in its corresponding resource pool, it can be easily transferred to an idle resource pool for processing via the interface.

[0065] As an optional implementation, each unit of the file conversion engine is deployed and run on Docker containers and managed using Kubernetes (K8S). K8S was chosen primarily because of its advantages:

[0066] Orchestrate containers across multiple hosts.

[0067] Make fuller use of hardware and maximize the effectiveness of the resources required for enterprise applications to run.

[0068] Deployment and updates of control and automation applications.

[0069] Mount and add the storage required to run stateful applications.

[0070] Dynamically scale containerized applications and their resources.

[0071] Declarative management of services ensures that deployed applications always function as you expect.

[0072] It features health checks and self-repair, as well as automatic placement, automatic startup, automatic replication, and automatic expansion.

[0073] Example 2

[0074] According to another aspect of the present invention, a file conversion engine based on cluster architecture intelligent scheduling is also provided. The file conversion engine based on cluster architecture intelligent scheduling is constructed using the aforementioned method for constructing a file conversion engine based on cluster architecture intelligent scheduling. The file conversion engine is based on distributed file sharing storage and includes:

[0075] The registration service center unit is used for long-term connections and management / maintenance with the service gateway, authentication center unit, authorization center unit, scheduling and command center unit, and resource pool. Management / maintenance includes module availability, current module load information, and service call addresses. Long-term connections are used to notify each unit of events in real time.

[0076] The service gateway is where all file conversion requests first enter. It collects file conversion request traffic and sends the requests to the authentication service center for further processing. Upon receiving a file conversion request, the service gateway registers it with the registration service center, facilitating information exchange between other units and the service gateway. The service gateway also returns the conversion result to the requester.

[0077] The authentication center service unit is used to authenticate the identity of the requester in the file conversion service request sent by the service gateway, and add the identity information to the file conversion request service after successful authentication; the authentication center service unit is also used to receive the service processing level returned for the file conversion service request.

[0078] The authentication method is adapted to the business scenario, selecting the most suitable authentication method. For example, some requesters provide JWT (JSON Web Token), and the authentication center uses JWT authentication; while some requesters have integrated with the OAuth 2.0 protocol, and the authentication center uses OAuth 2.0 authentication. After successful authentication, the identity information is included in the service request and sent to the subsequent authorization center service unit.

[0079] The authorization center service unit is used to authorize the identity of the requester in the file conversion service request sent by the authentication center service unit and determine the service processing level of the requester; after authorization is completed, the authorization data is added to the file conversion service request and sent to the dispatch and command center.

[0080] Different response levels will be allocated different file conversion resources to ensure varying processing times and qualities. Another benefit of tiered request processing is that it maximizes the fulfillment of the needs of critical systems and key clients.

[0081] As an optional implementation, the service processing level classification is designed based on two factors: the importance of the customer and the importance of the business system. Customers are divided into four levels: Gold, Silver, Bronze, and General. Business systems are divided into two levels: Important and Non-Important. Therefore, a comprehensive service level score can be generated similar to the following:

[0082] Gold-level clients + key systems: 100

[0083] Gold-level clients + non-critical systems: 80

[0084] Silver Clients + Key Systems: 95

[0085] Silver-level clients + non-critical systems: 75

[0086] Bronze-level clients + key systems: 90

[0087] Bronze-level clients + non-critical systems: 70

[0088] General customers + critical systems: 85

[0089] General customers + non-critical systems: 65.

[0090] The dispatch and command center unit is used to schedule file conversion service requests sent by the authorized center service unit, track the processing status of file conversion service requests in real time, and perform intelligent secondary scheduling, which is to schedule service node anomalies.

[0091] Specifically, the underlying layer maintains a message queue. The function and responsibility of the scheduling and command center unit is to maintain the position of each request in this message queue. The earlier the position, the higher the priority for processing. At the same time, based on the service level information of the file request, the resource pool allocation algorithm and the load balancing algorithm, the resource pool to process the request and the service node in the pool are calculated.

[0092] The intelligent scheduling and command center is the core functional module of the file conversion engine. It constantly tracks the processing status of file conversion requests and performs intelligent secondary scheduling. Intelligent secondary scheduling addresses service node anomalies. For example, if a request is pushed to a service node for processing, and that node unexpectedly stops working, the intelligent scheduling and command center will redirect the request to another available node.

[0093] As an optional embodiment, the dispatch and command center unit is also used to convert the local file storage path according to the URL of the file conversion service request, determine whether the source file to be converted in the file conversion service request exists locally, and add the request to the conversion queue if it exists to avoid repeated downloads and wasting resources; otherwise, if it does not exist, it will be downloaded, and after the download is successful, the dispatch and command center unit will be notified to schedule the file conversion unit to process it.

[0094] Multiple file conversion units, each performing only one type of file conversion task. The file conversion units perform conversions according to the scheduling tasks issued by the scheduling and command center unit and feed the conversion results back to the service gateway.

[0095] Specifically, each file conversion unit performs only one type of file conversion task. For example, converting an Office Word file to a PDF file is one type of file conversion task, while converting an Office Excel file to an HTML file is another type of file conversion service. These file conversion units are divided into resource processing pools of different service levels. Generally speaking, the higher the service level, the more conversion modules of each type are in the resource processing pool.

[0096] Each time, the file conversion unit retrieves a request to be converted from the conversion queue, calls the corresponding file conversion unit to perform the conversion based on the conversion type of the request, and notifies the scheduling center of the result regardless of whether the conversion is successful or not. If the conversion is successful, the converted file is uploaded to the unified file storage server.

[0097] By combining the file conversion unit with the scheduling center, the speed and success rate of file conversion can be improved more efficiently, reducing user waiting time.

[0098] As an optional embodiment, after receiving the intelligent dispatch center dispatch unit, if the file conversion unit identifies the source file as a compressed file such as rar or zip, it decompresses it, uploads the decompressed file to the unified file storage server and writes it into the decompression data table, and notifies the dispatch command center unit of the decompression result.

[0099] As an optional embodiment, the file conversion engine based on cluster architecture intelligent scheduling also includes an operation and maintenance monitoring backend system. The operation and maintenance monitoring backend system is used to visually display and manage the running status of the entire file conversion engine on a WEB page, providing a platform for manual processing and allowing for manual intervention and error correction when necessary.

[0100] Specifically, the operation and maintenance monitoring backend system can fully present the complete link tracing process of a file conversion request, from entering the gateway service, then reaching various intermediate processing nodes such as the authentication center unit, authorization center unit, intelligent scheduling and command center unit, resource pool, and file conversion unit, and finally returning to the service gateway. Additionally, the processing process can be manually intervened; for example, when a high-level service request lacks processing resources in its corresponding resource pool, it can be easily transferred to an idle resource pool for processing via the interface.

[0101] As an optional implementation, each unit of the file conversion engine is deployed and run on Docker containerization and managed using K8S (Kubernetes).

[0102] The working principle of the aforementioned file conversion engine is as follows: After receiving a file conversion request from the requester, the file conversion request first goes through the service gateway. The service gateway then sends the file conversion request to the authentication center service unit and the human service center unit. The authentication service center unit registers the file conversion request and authenticates the requester's identity for the file conversion service request sent by the service gateway. If authentication fails, the result is directly fed back to the service gateway. If authentication is successful, the identity information is added to the file conversion request service and sent to the scheduling and command center unit. Based on the file conversion service request sent by the authorization center service unit, the corresponding file conversion unit is invoked to perform the conversion. The file conversion unit feeds back the conversion result to the service gateway, which then feeds it back to the requester, thus completing the file conversion.

[0103] By deploying each unit service of the file conversion engine using a containerized approach, high availability of the cluster can be achieved, along with horizontal scaling capabilities. During peak business periods, the throughput of the entire file conversion service is intelligently increased; during off-peak periods, service nodes are appropriately reduced and some hosts are shut down to achieve energy conservation and green IT goals.

[0104] By deploying multiple dispatch and command centers, the intelligent dispatch terminal uses a load balancing algorithm to find idle conversion services or conversion services with small conversion queues (i.e., message queues), and initiates conversion requests to them. After the conversion service completes the conversion, it synchronously returns the conversion result to the dispatch terminal. The intelligent dispatch terminal previews the result if it is successful and redistributes it if it fails. This greatly improves the success rate of conversion and highly restores the file format and layout of the original file.

[0105] This invention is not limited to the specific embodiments described above. The above are merely preferred embodiments of this invention and are not intended to limit the invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

[0106] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0107] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The system embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interface, and the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0108] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0109] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0110] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0111] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A file conversion engine based on cluster architecture intelligent scheduling, characterized in that, The file conversion engine is based on distributed file sharing storage, comprising: A registration service center unit for long connection and management and maintenance with a service gateway, an authentication center unit, an authorization center unit, a dispatch command center unit, and a resource pool; The service gateway is used for collecting file conversion request traffic, and the service gateway will be registered to the registration service center unit after receiving a file conversion request, and is also used for returning the conversion result to the requester; The authentication center service unit is used for authenticating the identity of the requester of the file conversion service request sent by the service gateway, and adding the identity information into the file conversion request service after authentication, and is also used for receiving the service processing level of the returned file conversion service request; The authorization center service unit is used for authorizing the identity of the requester of the file conversion service request sent by the authentication center service unit, and determining the service processing level of the requester; and after authorization, the authorization data is added to the file conversion service request and sent; The dispatch command center unit is used for scheduling the task of the file conversion service request sent by the authorization center service unit, tracking the processing status of the file conversion service request in real time, and intelligently scheduling the secondary scheduling of the service node exception; A plurality of file conversion units, each of which only performs one type of file conversion task, performs conversion according to the scheduling task issued by the dispatch command center unit, and feeds back the conversion result to the service gateway.

2. The cluster architecture based intelligent scheduling file conversion engine of claim 1, wherein, It also includes an operation and maintenance monitoring background system, which is used for visual display and management of the running state of the entire file conversion engine on the WEB page, and provides a platform for manual processing.

3. The cluster architecture based intelligent scheduling file conversion engine of claim 1, wherein, The dispatch command center unit is also used for converting the local file storage path according to the url of the file conversion service request, judging whether the source file to be converted in the file conversion service request exists locally, when it exists, adding the request to the conversion queue, otherwise downloading, and notifying the dispatch command center unit to schedule the file conversion unit for processing after successful downloading.

4. The cluster architecture based intelligent scheduling file conversion engine of claim 1, wherein, Each unit of the file conversion engine is deployed and run based on Docker containerization, and managed using K8S.

5. The cluster architecture based intelligent scheduling file conversion engine of claim 1, wherein, All file conversion requests first enter the service gateway.

6. A method for building a file conversion engine based on intelligent scheduling of a cluster architecture, characterized in that, Comprising: Based on distributed technology, a bottom layer for distributed file sharing storage is built, and a registration service center unit, a service gateway, an authentication center service unit, an authorization center service unit, a dispatch command center unit, and a plurality of file conversion units are deployed on the bottom layer respectively; The registration service center unit is used for long connection and management and maintenance with the service gateway, the authentication center unit, the authorization center unit, the dispatch command center unit, and the resource pool; The service gateway is used for collecting file conversion request traffic, and the service gateway will be registered to the registration service center unit after receiving a file conversion request, and is also used for returning the conversion result to the requester; The authentication center service unit is used for authenticating the identity of the requester of the file conversion service request sent by the service gateway, and adding the identity information into the file conversion request service after authentication, and is also used for receiving the service processing level of the returned file conversion service request; The authorization center service unit is used for authorizing the identity of the requester of the file conversion service request sent by the authentication center service unit, and determining the service processing level of the requester; and after authorization, the authorization data is added to the file conversion service request and sent; The authorization center service unit is used for authorizing the identity of a requestor according to a file conversion service request sent by the authentication center service unit, and determining the service processing level of the requestor; After authorization, the authorization data is added to the file conversion service request and sent; The dispatching center unit is used for scheduling the file conversion service request sent by the authorization center service unit, tracking the processing state of the file conversion service request in real time, and intelligently scheduling the secondary scheduling of the service node exception; Each file conversion unit only performs one type of file conversion task, and each file conversion unit is divided into a resource pool of different service levels. The file conversion unit performs conversion according to the scheduling task issued by the dispatching center unit, and feeds back the conversion result to the service gateway.

7. The method of claim 6, wherein the method further comprises: An operation and maintenance monitoring background system is further deployed on the bottom layer, and the operation and maintenance monitoring background system is used for visually displaying and managing the running state of the entire file conversion engine on a WEB page, and providing a platform for manual processing.

8. The method of claim 6, wherein the method further comprises: The bottom layer is used for providing a shared file service for each unit of the file conversion engine, and guaranteeing that the file is distributed and highly available. The file is read, written, shared, distributed, and verified quickly and consistently.

9. The method of claim 6, wherein the method further comprises: The dispatching center unit is also used for converting a local file storage path according to a url of the file conversion service request, judging whether the source file to be converted in the file conversion service request exists locally, adding the request to a conversion queue when the source file exists, otherwise downloading the source file, and notifying the dispatching center unit to schedule a file conversion unit to process the file conversion service request after the source file is downloaded successfully.

10. The method of claim 6, wherein the method further comprises: Each unit of the file conversion engine is deployed and run based on Docker containerization, and is managed by using K8S.

Images