A cloud service network system based on high performance computing cluster

By introducing a cloud service network system into the high-performance computing cluster system, the scheduling service layer is used to realize communication between the website service layer, the database layer and the cluster, the problems of insufficient user operation complexity and security in the existing technology are solved, and the problem of improved The system applicability and data processing security are used to meet the data analysis needs of non-professional users.

CN117793105BActive Publication Date: 2024-11-19WESTLAKE UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311871020.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-29
Publication Date
2024-11-19
Estimated Expiration
2043-12-29

AI Technical Summary

Technical Problem

The existing high-performance computing cluster system has high requirements for use, poor applicability and broad spectrum, and insufficient security, making it difficult to meet the data processing and analysis needs of non-professional users.

Method used

A cloud service network system based on high-performance computing clusters is designed, including the website service layer, the scheduling service layer, the database layer and the cluster. Through the scheduling service layer, the communication between the website service layer, the database layer and the cluster is realized to avoid clusters and The database layer communicates directly to ensure data security and provides a user-friendly interactive interface. Users can process and analyze data without mastering Linux operating system or Slurm commands.

Benefits of technology

It improves the applicability and broad spectrum of the cloud service network system, reduces user operation complexity, enhances the security of data processing, and enables non-professional users to easily and efficiently use high-performance computing clusters for data analysis and processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117793105B_ABST
    Figure CN117793105B_ABST
Patent Text Reader

Abstract

The present application provides a cloud service network system based on a high-performance computing cluster. It includes a website service layer, a scheduling service layer, a database layer and a cluster, wherein the website service layer is connected to the scheduling service layer in communication, and the database layer responds to a user's request for task processing on the input directory and adds a record bar associated with the request content; the scheduling service layer is connected to the database layer in communication, and checks each record bar in the database layer at a preset time interval, and modifies the task status in each record bar based on the progress of the task; the cluster is connected to the scheduling service layer in communication, and is used to process the input directory, and respond to the instructions of the scheduling service layer, and feedback the task processing status to the scheduling service layer or transmit the output directory. In this way, the applicability and security of the cloud service network system are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of big data information processing technology, and specifically to a cloud service network system based on a high-performance computing cluster. Background Art

[0002] In a high-performance computing cluster, it is usually necessary to manage and schedule the computing nodes in the cluster to achieve fair sharing and optimal use of computing resources among multiple users. At this time, some special software is needed to manage and schedule the resources in the cluster, and Slurm is such a software.

[0003] Slurm can provide users with many computing resource management and job scheduling functions, such as node allocation, task submission, queue management, resource limits, priority management, etc. Users can submit tasks through the command line or Web interface, and then Slurm will automatically allocate resources and schedule jobs according to system load and user needs to ensure that each task can be completed on time.

[0004] However, in actual applications, if users want to analyze and process data based on high-performance computing clusters, they need to fully master the knowledge of the Linux operating system, write scripts by themselves, know all the commands of Slurm, and monitor the problems that Slurm may encounter during operation. Therefore, the current application of high-performance computing clusters has high requirements for users, and its applicability and broad spectrum are poor. In addition, the existing systems based on high-performance computing clusters have poor security and are vulnerable to attacks, resulting in data loss. Summary of the invention

[0005] This application is proposed to solve the above technical problems existing in the prior art. This application aims to provide a cloud service network system based on a high-performance computing cluster, which can provide the public with a cloud service network system with high applicability and security, and enable the public who are not familiar with high-difficulty technologies such as Linux operating system and Slurm to conveniently perform secure data processing and analysis based on the cloud service network system.

[0006] According to the first scheme of the present application, a cloud service network system based on a high-performance computing cluster is provided, wherein the cloud service network system is communicatively connected to a user terminal, and is at least used to receive data to be processed uploaded by a user and establish an input directory, as well as submitted requests for task processing on the input directory and obtaining an output directory, and comprises a website service layer, a scheduling service layer, a database layer and a cluster, wherein the website service layer is communicatively connected to the scheduling service layer, and is used to store the input directory and receive the output directory returned by the scheduling service layer, so that the user can obtain the output directory based on the user terminal; the database layer, in response to a request submitted by a user for task processing on the input directory The processing request adds a record item associated with the request content, and the record item at least includes the task status; the scheduling service layer is connected to the database layer for communication, checks each record item in the database layer at a preset time interval, and schedules the website service layer and the corresponding party in the cluster to perform the advancement of the task to the next task status based on the task status of each record item; based on the advancement of the task, the task status in each record item is modified; the cluster is connected to the scheduling service layer for communication, for performing task processing on the input directory, and in response to the instructions of the scheduling service layer, feedbacks the task processing status to the scheduling service layer or transmits the output directory.

[0007] Compared with the prior art, the beneficial effects of the embodiments of the present application are:

[0008] The cloud service network system provided by the embodiment of the present application has a scheduling service layer that is communicatively connected to the database layer, checks each record item in the database layer at a preset time interval, and schedules the website service layer and the counterpart in the cluster to perform the task advancement processing to the next task status based on the task status of each record item. The cluster is communicatively connected to the scheduling service layer, and is used to perform task processing on the input directory, and in response to the instructions of the scheduling service layer, feedback the task processing status to the scheduling service layer or transmit the output directory. In this way, the communication between the website service layer, the database layer and the cluster is realized through the scheduling service layer, thereby avoiding the communication between the cluster and the database layer, and can effectively ensure the security of data information.

[0009] In addition, users only need to upload the data to be processed and analyzed, and establish an input directory based on the cloud service network system to realize the processing and analysis of data based on the high-performance computing cluster. Users do not need to write scripts for the operation of the high-performance computing cluster, nor do they need to master the relevant knowledge of the Linux operating system, know all the commands of Slurm, or monitor the problems that Slurm may encounter during operation. Therefore, for general users who do not understand the relevant professional knowledge of high-performance computing clusters, they can use high-performance computing clusters conveniently and efficiently to complete data processing and analysis, which greatly improves the breadth and applicability of the cloud service network system.

[0010] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above description and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In the drawings, which are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. Similar reference numerals with letter suffixes or different letter suffixes may represent different examples of similar components. The accompanying drawings generally illustrate various embodiments by way of example and not by way of limitation, and together with the description and claims, are used to illustrate the disclosed embodiments. Such embodiments are illustrative and exemplary and are not intended to be exhaustive or exclusive embodiments of the present system or non-transitory computer-readable medium having instructions for implementing the method.

[0012] Figure 1 A structural diagram of a cloud service network system according to an embodiment of the present application is shown.

[0013] Figure 2 A flowchart is shown of the processing performed by the scheduling service layer in the cloud service network system according to an embodiment of the present application based on the status of the record strip in the queried database layer. DETAILED DESCRIPTION

[0014] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific implementation examples. The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation examples, but are not intended to limit the present application.

[0015] The words "first", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish. The words "include" or "comprises" and similar words used in this application mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of covering other elements. In this application, the arrows shown in the figures of each step are only examples of the execution order, not limitations. The technical solution of this application is not limited to the execution order described in the embodiments. The various steps in the execution order can be combined, decomposed, or the order can be changed, as long as the logical relationship of the execution content is not affected.

[0016] All terms (including technical terms or scientific terms) used in this application have the same meaning as those understood by ordinary technicians in the field to which this application belongs, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined herein. Systems known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, the systems should be considered as part of the specification.

[0017] Figure 1 The schematic diagram of the structure of the cloud service network system according to the embodiment of the present application is shown. The cloud service network system 100 is connected to the user terminal for communication, and is at least used to receive the data to be processed uploaded by the user and establish an input directory, as well as the submitted request for task processing on the input directory and obtaining the output directory, including a website service layer 101, a scheduling service layer 102, a database layer 103 and a cluster 104. The user terminal can be a notebook, a desktop computer, a tablet computer, a mobile phone, etc., which is not specifically limited.

[0018] Exemplarily, the cloud service network system 100 can be configured on the user terminal, and an interactive interface can be provided to the user on the user terminal so that the user can set user information, upload data to be processed, etc. based on the interactive interface. The cloud service network system 100 automatically establishes an input directory based on the user information set by the user, the uploaded data to be processed, etc. in response to the user's configuration operation. Based on the interactive interface provided by the user terminal, the user can submit a request for task processing on the input directory on the interactive interface. For example, the user clicks the submit button on the interactive interface, and the cloud service network system 100 automatically starts task processing in response to the user's task processing request for the submit operation. Of course, the user can also submit a request to obtain an output directory, and the cloud service network system 100 automatically provides the output directory to the user in response to the user's request to submit the output directory. This is only used as an exemplary explanation, and the specific method of the cloud service network system 100 communicating with the user terminal is not limited, as long as the cloud service network system 100 described in the embodiment of the present application can be provided to the user through the user terminal.

[0019] The website service layer 101 is in communication with the scheduling service layer 102, and is used to store the input directory and receive the output directory returned by the scheduling service layer 102, so that the user can obtain the output directory based on the user terminal. Specifically, the user enters the interactive interface from the user entrance based on the user terminal and uploads the data to be processed. The cloud service network system 100 automatically establishes an input directory containing the data to be processed and stores the input directory in the website service layer 101. After the user submits a request for task processing on the input directory based on the interactive interface, the database layer 103 responds to the request submitted by the user for task processing on the input directory, and adds a record bar associated with the request content, and the record bar at least includes the task status.

[0020] As the basis for data storage and management, the database layer 103 provides reliable data consistency and transaction processing capabilities, and supports efficient query operations. As an example, the database layer 103 can use a relational database management system, and the relational database management system architecture may include: a user information table for storing relevant information of registered users, such as user names, passwords, and email addresses; a user registration table for recording user registration information and timestamps, managing user registration processes, and counting the number of registered users; a task information table for saving the original data of each interpolation task, such as task name, task status, submission time, priority, etc.; a historical task table for storing the historical records of completed interpolation tasks, including the execution time, results, and logs of tasks; a prompt information table for storing prompts and notification information sent to users, such as task completion notifications, error prompts, etc.; and a task status table for recording the current status of each interpolation task, such as waiting status, running status, success status, and failure status. In this way, the database layer 103 can efficiently realize the effective storage and management of task-related data by using a relational database management system.

[0021] After the user submits a request to process the input directory, the database layer 103 automatically adds a record associated with the request content, which may include a user information item, a task information item, and a task status item to store the user information, the task information of the input directory to be processed, and the processing status of the current task. The information of the user information item, the task information item, the task status item, and other related items may be automatically obtained based on the user information table, the task information table, the task status table, and the like in the database layer 103. This is only used as an example and does not constitute a specific limitation on the database layer 103.

[0022] In this embodiment, the scheduling service layer 102 is in communication with the database layer 103, checks each record bar in the database layer 103 at a preset time interval, and schedules the website service layer 101 and the corresponding party in the cluster 104 to perform the task to the next task state based on the task state of each record bar. Specifically, the scheduling service layer 102 can use structured query language (SQL) to perform query, insert, update and delete operations on the database layer 103, and realize read and write operations on the record bars and other data information in the database layer 103, including querying task state, updating task state, inserting new record bars, etc.

[0023] The preset time interval may be 1s, 2s, or 3s, and may be the default value of the cloud service network system 100, and is not specifically limited thereto. For example, the scheduling service layer 102 queries each record strip in the database layer 103 at a time interval of 1s. If the target task status is read as the task has started to be established and has not been processed based on the target record strip, the input directory corresponding to the target record strip will be retrieved from the website service layer 101 to the scheduling service layer 102, and the task status of the target record strip in the database layer 103 will be updated to the task has been established and has not been processed. At this point, the task status of the target record strip is updated from the task has started to be established and has not been processed to the task has been established and has been completed and has not been processed, which also means that the processing progress of the input directory corresponding to the target record strip has been further advanced.

[0024] The scheduling service layer 102 can use a timed task (such as a Cron Job) to set a preset time interval and frequency to ensure timely acquisition of task status updates. The scheduling service layer 102 continues to query each record strip in the database layer 103 at a time interval of 1 second. If the task status of the target record strip is read as the task is established and not processed, it will communicate with the cluster 104. The cluster 104 will retrieve the input directory corresponding to the target record strip from the scheduling service layer 102 to the cluster 104 for processing. At this time, the scheduling service layer 102 further modifies the task status of the target record strip in the database layer 103 to the target task being processed. In other words, the scheduling service layer 102 modifies the task status in each record strip based on the progress of the task. At this time, the task status of the target record strip is updated from the task is established and not processed to the target task being processed, which also means that the processing progress of the input directory corresponding to the target record strip is further advanced.

[0025] In this way, by scheduling the query of the database layer 103 by the scheduling service layer 102, the website service layer 101 and the counterparts in the cluster 104 are scheduled to perform the task advancement process to the next task state based on the task state displayed in the queried record bar.

[0026] In this embodiment, the cluster 104 is connected to the scheduling service layer 102 for performing task processing on the input directory, and in response to the instruction of the scheduling service layer 102, the cluster 104 feeds back the task processing status or transmits the output directory to the scheduling service layer 102. That is, after the cluster 104 obtains the input directory from the scheduling service layer 102, it immediately processes and analyzes the corresponding data to be processed in the input directory, and feeds back the task processing status in the task processing to the scheduling service layer 102, and the scheduling service layer 102 updates the task status of the record in the database layer 103 accordingly after receiving the task processing status fed back by the cluster 104, so as to ensure the data consistency between the database layer 103 and the cluster 104, and provide accurate task status information to the user.

[0027] If the scheduling service layer 102 finds in the database layer 103 that the task status in the record bar is ended or other prompts indicating that the task processing has ended, it will send a request to obtain the output directory of the ended task to the cluster 104. In response to the instruction of the scheduling service layer 102 to obtain the output directory, the cluster 104 returns the corresponding output directory to the scheduling service layer 102, and the scheduling service layer 102 returns the output directory to the website service layer 101, so that the user can obtain the output directory based on the user terminal.

[0028] In this way, it is only necessary to provide the cloud service network system 100 described in the embodiment of the present application to the user, and the user only needs to upload the data to be processed to the cloud service network system 100 and submit a request for task processing, thereby improving the convenience and efficiency of the user's data analysis and processing based on the cloud service network system 100. At the same time, in the cloud service network system 100 provided in the embodiment of the present application, the cluster 104 does not communicate with the database layer 103 and the website service layer 101, and the communication with the cluster 104 is realized through the scheduling service layer 102, which can ensure reliable communication and data exchange with the cluster 104 and improve the security of data processing.

[0029] In some embodiments of the present application, in response to the request submitted by the user, the task status in the record strip corresponding to the request is updated to a first state indicating that the task has started to be established and has not been processed. That is, the user submits a request for task processing on the input directory based on the user terminal, and the cloud service network system 100 that is communicatively connected to the user terminal responds to the request, automatically adds a record strip associated with the task processing request in the database layer 103, and updates the task status in the record strip to the first state. Among them, the first state is used to indicate that the task has started to be established and has not been processed. For example, applicable text prompts include at least initial establishment, establishment, start establishment, or waiting for processing, etc., and there is no specific limitation on the text content used to indicate that the task has started to be established and has not been processed. Of course, the task has started to be established and has not been processed, which means that the task processing has just been established, but the task processing has not been performed on the data to be processed.

[0030] Furthermore, if Figure 2 As shown, in step S201, the scheduling service layer 102 regularly inspects the record strips in the database layer 103. In the process of querying each record strip in the database layer 103, if it is found that the task state of the concerned record strip is the first state (such as step S202), it means that the task corresponding to the concerned record strip has just been established but has not been processed. The scheduling service layer 102 is responsible for advancing the task to the next task state. The scheduling service layer 102 executes step S203, and the scheduling service layer 102 obtains the concerned input directory from the website service layer 101. The concerned input directory is the input directory corresponding to the concerned record strip, which is stored in the website service layer 101 and contains at least the data to be processed.

[0031] When the scheduling service layer 102 finds that the attention record strip is in the first state, the scheduling service layer 102 retrieves the attention input directory from the website service layer 101 to the scheduling service layer 102, and at the same time, executes step S204 to update the task state of the attention record strip to the second state, which is used to indicate that the attention task has been established and has not been processed, so as to represent that the attention task has been successfully established but the data to be processed is still being processed. That is to say, when the scheduling service layer 102 finds that the task state of the attention record strip is in the first state, the scheduling service layer 102 obtains the stored attention input directory from the website service layer 101, and updates the task state of the attention record strip to the second state indicating that the attention task has been established and has not been processed.

[0032] In step S205, if the task status of the concerned record strip is the second status, step S206 is executed, and the scheduling service layer 102 transmits the first instruction to the cluster 104. Specifically, if the scheduling service layer 102 inquires that the task status of the concerned record strip in the database layer 103 is the second status, it can be identified that the concerned task has been established but not processed. At this time, the scheduling service layer 102 transmits the first instruction of the establishment of the concerned task to the cluster to notify the cluster 104 that the concerned task is about to be processed, so that the cluster 104 obtains the concerned input directory from the scheduling service layer 102 and feeds back the concerned task number to the scheduling service layer 102 (such as step S207).

[0033] After receiving the first instruction from the scheduling service layer 102, the cluster 104 immediately obtains the concerned input directory from the scheduling service layer 102, and immediately places the obtained concerned input directory in the computing server for queueing for task processing. The concerned task number can be understood as the sequence number representing the task processing of the concerned task on the cluster 104. For example, if the sequence number of the concerned task in the cluster 104 is 30, then the concerned task number can be considered to be 30. That is to say, when the scheduling service layer 102 inquires that the task status of the concerned record strip is the second state, the scheduling service layer 102 transmits the first instruction of the establishment of the concerned task to the cluster 104; the cluster 104 responds to the first instruction, obtains the concerned input directory from the scheduling service layer 102, performs task processing on the concerned input directory, and feeds back the concerned task number of the concerned task to the scheduling service layer 102.

[0034] In some embodiments of the present application, the record strip also includes a task number item, and before the cluster 104 feeds back the concerned task number to the scheduling service layer 102, the task number item is empty. That is to say, in response to the request for submission, when the record strip associated with the request task processing is added in the database layer 103, the task number item in the record strip is empty at this time, and the task number item can be filled or edited. After receiving the concerned task number fed back by the cluster 104, the scheduling service layer 102 executes step S208, updates the concerned task number to the concerned record strip, and updates the task status to the third state. At this time, the task number item in the concerned record strip is updated to the concerned task number, and is no longer in an empty state. The third state is used to indicate that the concerned task is being processed. For example, the task status of the concerned record strip is updated to running, calculating, processing, etc., which can be used to indicate that the concerned task is being processed.

[0035] Further, in step S209, the task state of the concerned record strip is the third state. At this time, step S210 is executed, and the scheduling service layer 102 transmits a second instruction to the cluster 104. That is, when the scheduling service layer 102 inquires that the task state of the concerned record strip is the third state, the scheduling service layer 102 transmits a second instruction to the cluster 104 to view the processing progress of the concerned task, and the second instruction includes the concerned task number. The scheduling service layer 102 monitors each record strip in the database layer 103, and performs the advancement of the task to the next task according to the task state of each record strip. When the scheduling service layer 102 reads that the task state of the concerned record strip is in the third state, it can be known that the concerned task is being processed, and therefore, it is necessary to further monitor the processing progress of the concerned task. The scheduling service layer 102 transmits a second instruction to view the processing progress of the concerned task corresponding to the concerned task number to the cluster 104, and monitors and advances the processing progress of the concerned task.

[0036] In response to the second instruction, the cluster 104 feeds back the processing progress of the concerned task to the scheduling service layer 102 (such as step S211). In step S212, the scheduling service layer 102 updates the task status in the concerned record strip in the database layer 103 to match the processing progress. For example, the processing progress may include queuing and calculating, and the task status in the concerned record strip is updated to queuing and calculating accordingly. In other words, based on the processing progress of the concerned task fed back by the cluster 104, the scheduling service layer 102 updates the task status in the concerned record strip in the database layer 103 to match the processing progress. This is only used as an exemplary description and does not constitute a specific limitation.

[0037] In some embodiments of the present application, the task status in the concerned record strip in the database layer 103 is updated to match the processing progress, specifically including: when the processing progress indicates that the concerned task is being operated, the task status in the concerned record strip is updated to indicate the fourth state of being operated, for example, the cluster 104 is operating the data to be processed in the concerned input directory, at this time, the task processing progress can be indicated as being operated on the concerned task. The text used to represent the fourth state can be operating, analyzing, etc., which is not limited.

[0038] When the processing progress indicates that the processing of the concerned task is wrong, the task status in the concerned record bar is updated to the fifth status indicating that the concerned task is wrong. For example, if the data to be processed uploaded in the concerned input directory is wrong or other errors occur, it can be considered that the concerned task is wrong. The cluster 104 will also feed back the processing progress of the concerned task error to the scheduling service layer 102.

[0039] When the processing progress indicates that the processing of the concerned task is completed, the task status in the concerned record bar is updated to the sixth status indicating that the processing of the concerned task is completed. For example, the cluster 104 completes the calculation of all the data to be processed in the concerned input directory. At this time, it can be determined that the processing of the concerned task is completed.

[0040] Back to Figure 2 , in step S213, if the task status of the concerned record bar is the fourth status, step S214 is executed, and the scheduling service layer 102 continues to transmit the second instruction to the cluster 104 until the received processing progress indicates that the concerned task processing has an error or the concerned task processing has ended, and the second instruction is no longer transmitted to the cluster 104. That is to say, when the scheduling service layer 102 inquires that the task status of the concerned record bar is the fourth status, it can be known that the to-be-processed data in the concerned input directory corresponding to the concerned record bar is being calculated by the cluster 104. At this time, the scheduling service layer 102 will continue to monitor the processing progress of the concerned task, that is, continue to transmit the second instruction to the cluster 104 to notify the cluster 104 to inquire about the processing progress of the concerned task. In the case where the scheduling service layer 102 receives the task processing progress feedback from the cluster 104 indicating that the concerned task processing has an error or the concerned task processing has ended, the second instruction is no longer transmitted to the cluster 104. In this way, the processing progress of the concerned tasks is monitored in real time by the scheduling service layer 102, the scheduling service layer 102 obtains the real-time processing status of the concerned tasks, and updates the task status of the concerned record bar in the database layer 103 based on the processing progress, so that users can understand the execution status of each concerned task at any time.

[0041] In step S215, the task status of the concerned record bar is the fifth status, that is, when the scheduling service layer 102 inquires that the task status of the concerned record bar is the fifth status, step S216 is executed to end the entire task. In other words, when the processing progress of the concerned task fed back by the cluster 104 to the scheduling service layer 102 indicates that the concerned task has an error, the entire task is immediately ended and the program is exited, so that the user can review the cloud service network system 100, the user terminal or the uploaded data to be processed and other related information.

[0042] In step S217, the task state of the concerned record strip is the sixth state, that is, when the scheduling service layer 102 finds that the task state of the concerned record strip is the sixth state, step S218 is executed, and the scheduling service layer 102 transmits the third instruction to the cluster 104, so that the cluster 104 responds to the third instruction and transmits the concerned output directory to the scheduling service layer 102 (such as step S219). Wherein, the third instruction includes the concerned task number, which is used to indicate the acquisition of the concerned output directory. Specifically, when the scheduling service layer 102 finds that the task state of the concerned record strip is the sixth state, it means that the data to be processed in the concerned input directory has been calculated by the cluster 104, that is, the cluster 104 has completed the processing task of the concerned input directory, and automatically generated the concerned output directory, which at least includes the processing result of the concerned output directory. At this time, the scheduling service layer 102 transmits the third instruction to the cluster 104 to notify the cluster 104 that it will acquire the concerned output directory corresponding to the concerned task number. The third instruction can be, for example, "get output" and other related prompts that can be used to indicate the acquisition of the output directory. That is, the scheduling service layer 102 transmits a third instruction for obtaining the output directory of interest to the cluster 104 , where the third instruction includes the task number of interest; the cluster 104 transmits the output directory of interest to the scheduling service layer 102 in response to the third instruction.

[0043] The cluster 104 transmits the output directory of interest to the scheduling service layer 102 instead of directly transmitting it to the website service layer 101 , thereby improving the security of the output directory of interest during the data transmission process.

[0044] In some embodiments of the present application, the cluster 104 copies the output directory of interest to the scheduling service layer 102, so that both the output directory of interest and the input directory of interest are stored on the cluster 104. This is beneficial in that when an error occurs in the data in the output directory of interest, error correction can be performed based on the output directory of interest and the input directory of interest stored on the cluster 104, making it easier for users to find the cause of the error.

[0045] After receiving the output directory of interest returned by the cluster 104, the scheduling service layer 102 updates the task status of the record item of interest in the database layer 103 to the seventh status indicating that the task of interest has been returned to the scheduling service layer 102. That is, after the scheduling service layer 102 receives the output directory of interest returned by the cluster 104, the task status of the record item of interest in the database layer 103 is updated to the seventh status, and the text prompt used to represent the seventh status may be returned, fed back, outputted, etc., which is not limited thereto.

[0046] In step S220, the task status of the concerned record strip is the seventh status, that is, when the scheduling service layer 102 finds that the task status of the concerned record strip is the seventh status, step S221 is executed, and the scheduling service layer 102 cuts the received concerned output directory to the website service layer 101, that is, the scheduling service layer 102 cuts the received concerned output directory to the website service layer 101, so that the user can obtain the concerned output directory based on the website service layer 101. Specifically, in the process of the scheduling service layer 102 querying each record strip in the database layer 103, if the task status of the concerned record strip is found to be the seventh status, it means that the concerned task corresponding to the concerned record strip is completed, and the concerned output directory has been returned to the scheduling service layer 102. At this time, the scheduling service layer 102 cuts the concerned output directory to the website service layer 101, and the concerned output directory can be transmitted to the website service layer 101. At this time, the user can easily obtain the concerned output directory from the website service layer 101.

[0047] In some embodiments of the present application, in response to a user deleting a focused task based on an interactive interface, the task status of the focused record bar in the database layer 103 is updated to an eighth status indicating that the focused task is deleted. For example, if a user performs a deletion operation on a focused task based on an interactive interface provided by a user terminal, then correspondingly, the task status of the focused record bar corresponding to the deleted focused task in the database layer 103 is updated to the eighth status.

[0048] In step S222, the task state of the concerned record strip is the eighth state, that is, when the scheduling service layer 102 inquires that the task state of the concerned record strip is the eighth state, step S223 is executed, and the scheduling service layer 102 transmits the fourth instruction to the cluster 104 to delete the data related to the concerned task number, and the fourth instruction contains the concerned task number. In other words, the fourth instruction is still transmitted to the cluster 104 by the scheduling service layer 102 to notify the cluster 104 to delete the data related to the concerned task number, including the concerned input directory and the concerned output directory. In step S224, the cluster 104 responds to the fourth instruction, deletes the data associated with the concerned task number, and feeds back the processing progress of the deleted concerned task-related data to the scheduling service layer 102. When the scheduling service layer 102 receives the processing progress of the deleted concerned task-related data, it transfers the deleted concerned task-related data to the historical task, and deletes the concerned task-related data in the website service layer 101 at the same time. In this way, the safe deletion of relevant data in the cluster 104, the database layer 103 and the website service layer 101 is achieved.

[0049] In various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0050] In addition, it should be noted that the scheduling service layer 102 monitors and queries multiple records or other related task information in the database layer 103 at the same time, and the scheduling service layer 102 can perform corresponding advancement processing according to the task status of different records. For example, if the scheduling service layer 102 finds that the task status of the first record is the third status and the task status of the second record is the fifth status, the concerned tasks corresponding to the first record and the second record can be advanced respectively.

[0051] In some embodiments of the present application, the cloud service network system 100 further includes a network exchange layer 105, which is used to send instructions from the scheduling service layer 102 to the cluster 104. In other words, the scheduling service layer 102 communicates with the cluster 104 through the network exchange layer 105, so that the security of the cloud service network system 100 can be ensured. In addition, the cloud service network system 100 further includes an API service node 106, which has the function of delivering tasks to the computing server 107 in the cluster 104 and monitoring tasks.

[0052] This application describes various operations or functions that may be implemented as software codes or instructions or defined as software codes or instructions. Such content may be source code or differential code ("incremental" or "patch" code) that can be directly executed ("object" or "executable" form). Software codes or instructions may be stored in a computer-readable storage medium, and when executed, may cause a machine to perform the described functions or operations, and include any mechanism for storing information in a form accessible to a machine (e.g., a computing device, an electronic system, etc.), such as recordable or non-recordable media (e.g., read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.).

[0053] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims will be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the implementation of the application, and the examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0054] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. For example, those of ordinary skill in the art can use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features can be grouped together to simplify the application. This should not be interpreted as an intention that a disclosed feature that does not require protection is necessary for any claim. On the contrary, the subject matter of the present application may be less than all the features of a specific disclosed embodiment. Thus, the claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim is independently used as a separate embodiment, and it is considered that these embodiments can be combined with each other in various combinations or arrangements. The scope of the present application should be determined with reference to the full scope of the equivalent forms of the attached claims and these claims.

[0055] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A cloud service network system based on a high performance computing cluster, characterized in that: The cloud service network system is connected to the user terminal for communication, and is used at least to receive the data to be processed uploaded by the user and to establish an input directory including at least the data to be processed, as well as the submitted request for task processing on the input directory and obtaining the output directory, and includes a website service layer, a scheduling service layer, a database layer and a cluster, wherein: The website service layer is in communication with the scheduling service layer, and is used to store the input directory and receive the output directory returned by the scheduling service layer, so that the user can obtain the output directory based on the user terminal; The database layer, in response to a request submitted by a user to perform task processing on the input directory, adds a record item associated with the request content, wherein the record item at least includes a task status; The scheduling service layer is connected to the database layer for communication, checks each record in the database layer at a preset time interval, and schedules the website service layer and the counterpart in the cluster to perform the task to the next task state based on the task state of each record, including: When the task status is that the task has started to be created and has not been processed, the input directory will be transferred from the website service layer to the scheduling service layer; When the task status is task creation completed but not processed, it will communicate with the cluster, and the cluster will call the input directory from the scheduling service layer to the cluster for processing; Based on the progress of the task, the task status in each record bar is modified; The cluster is connected to the scheduling service layer in communication, but not to the database layer and the website service layer, and is used to process tasks on the input directory, and in response to instructions from the scheduling service layer, to feed back task processing status or transmit output directories to the scheduling service layer.

2. The cloud service network system according to claim 1, characterized in that: In response to the request submitted by the user, the task status in the record bar corresponding to the request is updated to a first status indicating that the task has started to be created and has not been processed.

3. The cloud service network system according to claim 2, characterized in that: When the scheduling service layer finds that the task status of the concerned record item is the first status, The scheduling service layer obtains the stored attention input directory from the website service layer, and updates the task status of the attention record bar to a second status indicating that the attention task has been established and has not been processed.

4. The cloud service network system according to claim 3, characterized in that: When the scheduling service layer finds that the task status of the concerned record item is the second status, The scheduling service layer transmits a first instruction to the cluster that the establishment of the concerned task is completed; In response to the first instruction, the cluster obtains the attention input directory from the scheduling service layer, performs task processing on the attention input directory, and feeds back the attention task number of the attention task to the scheduling service layer.

5. The cloud service network system according to claim 4, characterized in that: The record also includes a task number item, which is empty before the cluster feeds back the concerned task number to the scheduling service layer; After receiving the concerned task number fed back by the cluster, the scheduling service layer updates the concerned task number to the task number item in the concerned record bar, and at the same time, updates the task state of the concerned record bar to the third state indicating that the concerned task is being processed.

6. The cloud service network system according to claim 5, characterized in that: When the scheduling service layer finds that the task status of the concerned record item is the third status, The scheduling service layer transmits a second instruction to the cluster to check the processing progress of the concerned task, wherein the second instruction includes the concerned task number; In response to the second instruction, the cluster feeds back the processing progress of the concerned task to the scheduling service layer; The scheduling service layer updates the task status in the concerned record in the database layer to match the processing progress based on the processing progress of the concerned task fed back by the cluster.

7. The cloud service network system according to claim 6, characterized in that: The task status in the concerned record in the database layer is updated to match the processing progress, specifically including: When the processing progresses to indicate that the concerned task is being operated, the task status in the concerned record bar is updated to a fourth status indicating that the concerned task is being operated; In the case where the processing progress indicates that an error has occurred in the processing of the concerned task, the task status in the concerned record bar is updated to a fifth status indicating that an error has occurred in the concerned task; When the processing progresses to indicate that the processing of the task of interest is completed, the task status in the attention record bar is updated to a sixth status indicating that the processing of the task of interest is completed.

8. The cloud service network system according to claim 7, characterized in that: When the scheduling service layer finds that the task status of the concerned record item is the fourth status, The scheduling service layer continues to transmit the second instruction to the cluster until the received processing progress indicates that an error has occurred in the processing of the concerned task or the processing of the concerned task has ended, and then the second instruction is no longer transmitted to the cluster.

9. The cloud service network system according to claim 7, characterized in that: When the scheduling service layer finds that the task status of the concerned record bar is the fifth status, the entire task is terminated.

10. The cloud service network system according to claim 7, characterized in that: When the scheduling service layer finds that the task status of the concerned record item is the sixth status, The scheduling service layer transmits a third instruction for obtaining a concerned output directory to the cluster, wherein the third instruction includes a concerned task number; In response to the third instruction, the cluster transmits the output directory of interest to the scheduling service layer.

11. The cloud service network system according to claim 10, characterized in that: The cluster copies the output directory of interest to the scheduling service layer, so that the output directory of interest and the input directory of interest are both stored on the cluster; After receiving the concerned output directory returned by the cluster, the scheduling service layer updates the task status of the concerned record item in the database layer to the seventh status indicating that the concerned task has been returned to the scheduling service layer.

12. The cloud service network system according to claim 11, characterized in that: When the scheduling service layer finds that the task status of the concerned record item is the seventh status, The scheduling service layer cuts the received output directory of interest to the website service layer, so that the user can obtain the output directory of interest based on the website service layer.

13. The cloud service network system according to claim 11, characterized in that: In response to the user performing a focused task deletion operation based on the interactive interface, the task state of the focused record bar in the database layer is updated to an eighth state indicating that the focused task is deleted; When the scheduling service layer finds that the task status of the concerned record item is the eighth status, The scheduling service layer transmits a fourth instruction to the cluster to delete data related to the concerned task number, wherein the fourth instruction includes the concerned task number; The cluster deletes the data associated with the concerned task number in response to the fourth instruction, and feeds back to the scheduling service layer the progress of processing of deleting the concerned task associated data; When receiving the processing progress of the deleted task-related data, the scheduling service layer transfers the deleted task-related data to the historical tasks, and deletes the task-related data in the website service layer at the same time.

14. The cloud service network system according to claim 1, characterized in that: The cloud service network system further includes a network switching layer, which is used to send instructions from the scheduling service layer to the cluster.