A pipeline interface arrangement aggregation method and device

Through the pipelined interface orchestration aggregation method, the configuration files are parsed and the orchestration process is executed, which solves the problem that gateway services cannot process data from multiple software systems, and implements interface aggregation and data processing, which is suitable for data integration and processing of multiple software systems.

CN117978873BActive Publication Date: 2024-12-03CHINA ELECTRONICS INVESTMENT HLDG LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410130067.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-30
Publication Date
2024-12-03
Estimated Expiration
2044-01-30

AI Technical Summary

Technical Problem

The interface orchestration function of existing gateway services cannot effectively process and integrate data from multiple software systems, and cannot dynamically orchestrate it, and is suitable for data integration of multiple software systems.

Method used

The interface orchestration aggregation method is adopted in the form of pipelines. By obtaining and parsing configuration files, executing orchestration processes, requesting multiple services, receiving response data, performing calculation processing, and returning processing results. The method includes parsing configuration files, judging configuration formats, converting orchestration steps and task data, executing parallel or serial tasks, processing interface requests, data processing, and calling tasks.

Benefits of technology

It realizes the integration and processing of data of multiple software systems, supports interface aggregation and data processing, can reuse pipeline processes, flexibly orchestrate, and solves the problem of operating and aggregating interface data of different software services.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117978873B_ABST
    Figure CN117978873B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides a method and system for interface orchestration aggregation in the form of a pipeline, the method comprising: obtaining a configuration file submitted for interface orchestration, parsing the configuration file into orchestration steps and task data; when a console requests data, finding the corresponding orchestration steps and task data according to the request of the console, executing the orchestration process according to the orchestration steps and task data, and requesting corresponding multiple services according to the orchestration process; receiving response data from the multiple services, and performing calculation processing according to the content of the task data and the response data to obtain a processing result, and returning the processing result to the console. The embodiment of the present invention solves the problem of needing to operate and collect data of multiple software systems, and various interface aggregation and data processing can be completed through the form of orchestration pipeline.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of interface orchestration of gateway services, and in particular to a pipeline-style interface orchestration aggregation method and device. Background Art

[0002] When using different software systems, especially software systems purchased from other manufacturers to handle different businesses, it is often necessary to operate or integrate the businesses or data of multiple systems. Different software systems often only provide interfaces for external calls. If data from different systems needs to be processed centrally in a certain service, a method is needed to call the interfaces of different software systems and collect data for processing to form new data for return.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art: currently, the interface orchestration function of the gateway service is generally used for processing, but the gateway service generally only provides interface aggregation, but does not perform data processing, and generally cannot be dynamically orchestrated, and is not suitable for data integration of multiple software systems. Summary of the invention

[0004] In view of this, an object of an embodiment of the present invention is to provide a pipelined interface orchestration aggregation method and device to solve the above-mentioned problem.

[0005] To achieve the above-mentioned purpose, in a first aspect, an embodiment of the present invention provides a pipelined interface orchestration aggregation method, comprising:

[0006] Obtaining a configuration file submitted for interface orchestration, parsing the configuration file into orchestration steps and task data;

[0007] When the console requests data, find the corresponding orchestration steps and task data according to the request of the console, execute the orchestration process according to the orchestration steps and task data, and request corresponding multiple services according to the orchestration process;

[0008] Response data is received from the plurality of services, and calculation processing is performed according to the content of the task data and the response data to obtain processing results, and the processing results are returned to the console.

[0009] In some possible implementations, the step of obtaining a submitted configuration file for interface orchestration and parsing the configuration file into orchestration steps and tasks specifically includes:

[0010] Receive configuration files and parse them;

[0011] Determine whether the configuration format of the configuration file is correct; the configuration format of the configuration file is correct means that the configuration file conforms to a preset format specification, and the configuration content of the configuration file conforms to a specification that can be recognized by the aggregation service;

[0012] If the configuration format is correct, the orchestration steps and task data in the configuration file are obtained by parsing, and the orchestration steps and task data are converted into a program data structure and stored in a cache in the aggregation controller;

[0013] If the configuration format is incorrect, the configuration file submission will be abandoned directly.

[0014] In some possible implementations, when the console requests data, the corresponding orchestration steps and task data are found according to the request of the console, the orchestration process is executed according to the orchestration steps and task data, and corresponding multiple services are requested according to the orchestration process; response data is received from the multiple services, and calculation processing is performed according to the content of the task data and the response data to obtain a processing result, and the processing result is returned to the console; specifically including:

[0015] receiving a pipeline execution request, wherein the pipeline execution request carries an orchestrated pipeline number, and acquiring orchestration data corresponding to the pipeline according to the pipeline number;

[0016] Acquire the arrangement steps to be executed one by one according to the content of the arrangement data;

[0017] Determine whether the type of the current orchestration step is a parallel step;

[0018] If the type of the current orchestration step is a parallel step, all tasks in the current orchestration step are taken out and executed in parallel. After all tasks are executed, the next orchestration step is executed. If the current orchestration step is the last orchestration step, the processing result is returned to the console.

[0019] If the type of the current orchestration step is not a parallel step, the tasks in the current orchestration step are taken out in sequence and executed serially in sequence. After the current task is executed, it is determined whether the current task is the last task in the current orchestration step. If it is not the last task in the current orchestration step, the next task is executed. If it is the last task in the current orchestration step, the current orchestration step is ended. If the current orchestration step is the last orchestration step, the processing result is returned to the console.

[0020] In some possible implementations, the task execution process includes the following steps:

[0021] After starting the task, obtain the task type and determine whether the task type is an interface request task;

[0022] If the task type is an interface request task, determine whether input parameters are required. If input parameters are required, obtain the input parameters from the parameter output and pass them to the current task, and send a network request; if input parameters are not required, directly send the network request according to the request content of the configuration file; after sending the network request, receive the response data of the network request, and save the response data according to the configuration file;

[0023] If the task type is a data processing task, the previously saved response data is obtained, the specified response data is extracted according to the configuration file for corresponding calculation processing, new data is generated as the processing result and the processing result is saved;

[0024] If the task type is a call task, other scheduled and executable pipelines in the aggregation service are called to obtain the processing results and save the data.

[0025] In a second aspect, a pipeline-style interface orchestration aggregation device is provided, which is applied to an aggregation service, and the device includes:

[0026] The orchestration engine is used to obtain the configuration file submitted for interface orchestration, parse the configuration file, and parse it into orchestration steps and task data;

[0027] The aggregation controller is used to find the corresponding orchestration steps and task data according to the request of the console when the console requests data, execute the orchestration process according to the orchestration steps and task data, and request corresponding multiple services according to the orchestration process; receive response data from the multiple services, and perform calculations and processing based on the content of the task data and the response data to obtain processing results, and return the processing results to the console.

[0028] In some possible implementations, the orchestration engine specifically includes:

[0029] A receiving module, used for receiving a configuration file;

[0030] Configuration file parsing module, used to parse the configuration file;

[0031] The format determination module is used to determine whether the configuration format of the configuration file is correct; the configuration format of the configuration file is correct means that the configuration file conforms to a preset format specification, and the configuration content of the configuration file conforms to a specification that can be recognized by the aggregation service;

[0032] The orchestration information parsing and saving module is used to parse the orchestration steps and task data in the configuration file if the configuration format is correct, and convert the orchestration steps and task data into the program data structure and save it in the cache of the aggregation controller; if the configuration format is incorrect, the configuration file submission is directly abandoned.

[0033] In some possible implementations, the aggregation controller further specifically includes:

[0034] A request receiving module, used for receiving a pipeline execution request, wherein the pipeline execution request carries a scheduled pipeline number;

[0035] An arrangement data acquisition module, used to acquire arrangement data corresponding to the pipeline according to the pipeline number;

[0036] An arrangement step acquisition module, used to acquire the arrangement steps to be executed one by one according to the content of the arrangement data;

[0037] An arrangement step type determination module is used to determine whether the type of the current arrangement step is a parallel step;

[0038] A parallel task execution module, for taking out all the tasks in the current orchestration step and executing them in parallel if the type of the current orchestration step is a parallel step, and entering the execution of the next orchestration step after all the tasks are executed, and returning the processing result to the console if the current orchestration step is the last orchestration step;

[0039] The serial task execution module is used to, if the type of the current orchestration step is not a parallel step, sequentially take out the tasks in the current orchestration step and execute these tasks in sequence. After the current task is executed, it is determined whether the current task is the last task in the current orchestration step. If it is not the last task in the current orchestration step, the next task is executed. If it is the last task in the current orchestration step, the current orchestration step is ended. If the current orchestration step is the last orchestration step, the processing result is returned to the console.

[0040] In some possible implementations, the parallel task execution module or the serial task execution module specifically includes:

[0041] The task type acquisition submodule is used to acquire the task type after starting the task and determine whether the task type is an interface request task;

[0042] The interface request task processing submodule is used to determine whether input parameters are required if the task type is an interface request task. If input parameters are required, the input parameters are obtained from the parameter output and passed to the current task, and a network request is sent; if input parameters are not required, the network request is directly sent according to the request content of the configuration file; after sending the network request, the response data of the network request is received, and the response data is saved according to the configuration file;

[0043] The data processing task processing submodule is used to obtain previously saved response data if the task type is a data processing task, extract the specified response data according to the configuration file, perform corresponding calculation processing, generate new data as the processing result and save the processing result;

[0044] The calling task processing submodule is used to call other already scheduled and executable pipelines in the aggregation service if the task type is a calling task, obtain the processing results and save the data.

[0045] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the pipeline interface orchestration aggregation method described in the first aspect is implemented.

[0046] According to a fourth aspect, a computer device is provided, comprising:

[0047] one or more processors;

[0048] A storage device for storing one or more programs;

[0049] When the one or more programs are executed by the one or more processors, the one or more processors implement the pipeline interface orchestration aggregation method as described in the first aspect.

[0050] The above technical solution has the following beneficial effects:

[0051] The embodiment of the present invention solves the problem of needing to operate and collect data of multiple software systems, and can complete various interface aggregation and data processing by arranging pipelines.

[0052] The embodiment of the present invention can arrange the interface processing flow through pipeline arrangement.

[0053] The embodiment of the present invention can perform aggregate data processing of interfaces.

[0054] The embodiments of the present invention can reuse the processes of other pipelines and arrange them flexibly.

[0055] The embodiments of the present invention can solve the problem of operating and aggregating different software service interface data. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 is a general architecture diagram of an embodiment of the present invention;

[0058] Figure 2 is a specific functional block diagram of the orchestration engine 110 in an embodiment of the present invention;

[0059] Figure 3 is a specific functional block diagram of the aggregation controller 120 in an embodiment of the present invention;

[0060] Figure 4 It is a specific functional block diagram of the parallel task execution module 125 or the serial task execution module 126 in an embodiment of the present invention;

[0061] Figure 5 This is a flow chart of a pipeline interface orchestration aggregation method according to an embodiment of the present invention.

[0062] Figure 6 is a flow chart of configuration submission performed by the orchestration engine 110 according to an embodiment of the present invention;

[0063] Figure 7 is a flow chart of pipeline execution executed by the aggregation controller 120 according to an embodiment of the present invention;

[0064] Figure 8 is a flowchart of the task execution performed by the aggregation controller 120 according to an embodiment of the present invention;

[0065] Fig. 9 It is a functional block diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] The interface arrangement of current gateway services can solve the problem of partial data collection, but it is not applicable to data processing and operation of different software systems. The embodiment of the present invention solves the problem of needing to operate and collect data of multiple software systems, and can complete various interface aggregation and data processing by arranging pipeline forms.

[0068] Figure 1 is a general architecture diagram of an embodiment of the present invention, such as Figure 1 As shown, an embodiment of the present invention provides an interface arrangement aggregation device in the form of a pipeline, which is applied to an aggregation service 100. The aggregation service 100 connects a console 200 and multiple services. Figure 1 Service A, service B and service C are exemplarily depicted in the figure, and the device includes:

[0069] The orchestration engine 110 is used to obtain the submitted configuration file for interface orchestration, parse the configuration file, and parse it into orchestration steps and task data;

[0070] The aggregation controller 120 is used to find the corresponding orchestration steps and task data according to the request of the console 200 when the console 200 requests data, execute the orchestration process according to the orchestration steps and task data, and request corresponding multiple services according to the orchestration process; receive response data from multiple services, and perform calculation processing according to the content of the task data (the content represents the information carried by the task data) and the response data to obtain the processing result, and return the processing result to the console 200. The request of the console may include: pipeline execution request, interface request or program call request.

[0071] like Figure 1 As shown, the overall process includes the following steps: the user submits the interface orchestration configuration file to the aggregation service 100, and the orchestration engine 110 in the aggregation service 100 parses the orchestration configuration file after receiving it, and parses it into orchestration steps and task data, and passes the parsed orchestration steps and task data to the aggregation controller 120 for standby. When the console 200 requests data, the aggregation controller 120 finds the orchestration data corresponding to the request, and the orchestration data includes orchestration steps and task data, executes the orchestration process according to the orchestration data, and requests different services according to the orchestration process. After receiving the response data from different services, the aggregation controller 120 performs calculation processing according to the actual content of the task. For example, if the actual content of the task is to calculate the sum of a field of the data returned by two interfaces, the field is extracted and calculated, and the new data is returned to the console 200 to complete the request. The calculation processing can also include merging the data of multiple responses into one data, converting the data results into a unified data format, etc.

[0072] Figure 2 is a specific functional block diagram of the orchestration engine 110 in an embodiment of the present invention, such as Figure 2As shown, in some embodiments, the orchestration engine 110 specifically includes: a receiving module 111 for receiving a configuration file; a configuration file parsing module 112 for parsing the configuration file; a format judgment module 113 for judging whether the configuration format of the configuration file is correct; the correct configuration format of the configuration file means that the configuration file conforms to the preset format specification, and the configuration content of the configuration file conforms to the specification that can be recognized by the aggregation service; the orchestration information parsing and saving module 114 is used to parse and obtain the orchestration steps and task data in the configuration file if the configuration format is correct, and convert the orchestration steps and task data into the data structure of the program and save it in the cache of the aggregation controller; if the configuration format is incorrect, the configuration file submission is directly abandoned. Among them, the configuration file must first conform to the standard yaml or json format specification, and the configuration content needs to conform to the specification that can be recognized by the aggregation service. For example: if the configuration file contains characters that do not conform to the standard yaml or json format, it is directly considered incorrect; if the configuration file conforms to the standard yaml or json format, but the content does not conform to the format specification that can be recognized by the aggregation service, it is also considered incorrect.

[0073] exist Figure 2 The configuration process is followed by Figure 3 The execution process can only be executed after configuration. Figure 3 is a specific functional block diagram of the aggregation controller 120 in an embodiment of the present invention, such as Figure 3As shown, in some embodiments, the aggregation controller 120 further specifically includes: a request receiving module 121, which is used to receive a pipeline execution request, and the pipeline execution request carries the orchestration pipeline number; an orchestration data acquisition module 122, which is used to acquire the orchestration data corresponding to the pipeline (number) according to the pipeline number, and the orchestration data includes orchestration steps and task data; an orchestration step acquisition module 123, which is used to acquire the orchestration steps to be executed one by one according to the content of the orchestration data; an orchestration step type judgment module 124, which is used to judge whether the type of the current orchestration step is a parallel step; a parallel task execution module 125, which is used to take out all the tasks in the current orchestration step and execute these tasks in parallel if the type of the current orchestration step is a parallel step. After all tasks are completed, the execution of the next orchestration step is entered. If the current orchestration step is the last orchestration step, the processing result is returned to the console; the serial task execution module 126 is used to, if the type of the current orchestration step is not a parallel step, sequentially take out the tasks in the current orchestration step and execute these tasks in sequence. After the current task is executed, it is determined whether the current task is the last task in the current orchestration step. If it is not the last task in the current orchestration step, the next task in the current orchestration step is executed. If it is the last task in the current orchestration step, the current orchestration step is ended. If the current orchestration step is the last orchestration step, the processing result is returned to the console.

[0074] Figure 4 is a specific functional block diagram of the parallel task execution module 125 or the serial task execution module 126 in an embodiment of the present invention, such as Figure 4 As shown, in some embodiments, the parallel task execution module 125 or the serial task execution module 126 specifically includes: a task type acquisition submodule 1251, which is used to acquire the task type after starting the task and determine whether the task type is an interface request task; an interface request task processing submodule 1252, which is used to determine whether input parameters are required if the task type is an interface request task, and if input parameters are required, obtain the input parameters from the parameter output and pass them into the current task, and send a network request, which network request includes but is not limited to an HTTP request; if no input parameters are required, directly send the network request according to the request content of the configuration file; after sending the network request, receive the response data of the network request, and save the response data according to the configuration file; a data processing task processing submodule 1253, which is used to acquire the previously saved response data if the task type is a data processing task, extract the specified response data according to the configuration file for corresponding calculation processing, generate new data as the processing result and save the processing result; a calling task processing submodule 1254, which is used to call other already orchestrated and executable pipelines in the aggregation service if the task type is a calling task, acquire the processing result and save the data.

[0075] Figure 5 is a flow chart of a pipelined interface orchestration aggregation method according to an embodiment of the present invention, such as Figure 5 As shown, the method is executed by the aggregation service and includes the following steps:

[0076] S11: Obtain the submitted configuration file for interface orchestration, parse the configuration file into orchestration steps and task data;

[0077] S12: when the console requests data, find the corresponding orchestration steps and task data according to the request of the console, execute the orchestration process according to the orchestration steps and task data, and request corresponding multiple services according to the orchestration process;

[0078] S13: receiving response data from multiple services, performing calculations and processing based on the content of the task data and the response data, obtaining processing results, and returning the processing results to the console.

[0079] In some embodiments, step S11 obtains a submitted configuration file for interface orchestration, parses the configuration file, and parses it into orchestration steps and tasks, specifically including:

[0080] Receive configuration files and parse them;

[0081] Determine whether the configuration format of the configuration file is correct; the configuration format of the configuration file is correct means that the configuration file conforms to the preset format specification, and the configuration content of the configuration file conforms to the specification that can be recognized by the aggregation service;

[0082] If the configuration format is correct, the orchestration steps and task data in the configuration file are parsed and converted into the data structure of the program and saved in the cache;

[0083] If the configuration format is incorrect, the configuration file submission will be abandoned directly.

[0084] In some embodiments, step S12 and step S13 specifically include the following steps:

[0085] Receive a pipeline execution request, the pipeline execution request carries an orchestrated pipeline number, and obtain orchestration data corresponding to the pipeline according to the pipeline number;

[0086] Obtain the orchestration steps that need to be executed one by one according to the content of the orchestration data;

[0087] Determine whether the type of the current orchestration step is a parallel step;

[0088] If the type of the current orchestration step is a parallel step, all tasks in the current orchestration step are taken out and executed in parallel. After all tasks are executed, the next orchestration step is executed. If the current orchestration step is the last orchestration step, the processing result is returned.

[0089] If the type of the current orchestration step is not a parallel step, the tasks in the current orchestration step are taken out in sequence and executed serially in sequence. After the current task is executed, it is determined whether the current task is the last task in the current orchestration step. If it is not the last task in the current orchestration step, the next task is executed. If it is the last task in the current orchestration step, the current orchestration step is ended. If the current orchestration step is the last orchestration step, the processing result is returned.

[0090] In some embodiments, the task execution process may include the following steps:

[0091] After starting the task, obtain the task type and determine whether the task type is an interface request task;

[0092] If the task type is an interface request task, determine whether input parameters are required. If input parameters are required, obtain the input parameters from the parameter output and pass them to the current task, and send a network request; the network request includes but is not limited to HTTP requests; if no input parameters are required, send the network request directly according to the request content of the configuration file; after sending the network request, receive the response data of the network request, and save the response data according to the configuration file;

[0093] If the task type is a data processing task, the previously saved response data is obtained, the specified response data is extracted according to the configuration file, and the corresponding calculation processing is performed, and new data is generated as the processing result and the processing result is saved;

[0094] If the task type is a call task, other scheduled and executable pipelines in the aggregation service are called to obtain the processing results and save the data.

[0095] Figure 6 is a configuration submission flow chart of an embodiment of the present invention, such as Figure 6 As shown in the figure, the configuration submission process includes the following steps:

[0096] S21: The aggregation service receives the configuration file. The configuration file supports YAML and JSON formats.

[0097] S22: After receiving, parse (the result of the parsing is the data structure of the program) the configuration file content.

[0098] S23: Compare the configuration file format specifications (the configuration file must first comply with the standard yaml or json format specifications, and the configuration content must comply with the specifications that can be recognized by the aggregation service) to determine whether the configuration format is correct. For example: If the configuration file contains characters that do not comply with the standard yaml or json format, it is directly considered incorrect. If it complies with the standard yaml or json format, but the content does not comply with the format specifications that can be recognized by the aggregation service, it is also considered incorrect. The general format specifications of the configuration file are as follows (taking json as an example):

[0099]

[0100] S24: If the format is correct, the process information in the configuration file is parsed, which is mainly the orchestration information, including the orchestration steps and task data.

[0101] S25: Convert the arrangement information into a data structure of the program and save it in a cache in the aggregation controller for future use.

[0102] S26: If the format is incorrect, the configuration file submission will be abandoned.

[0103] Figure 7 is a flow chart of the pipeline execution of an embodiment of the present invention, such as Figure 7 As shown, the pipeline execution process executed by the aggregation controller includes the following steps:

[0104] S31: receiving a pipeline execution request from the service console 200, wherein the pipeline execution request carries a scheduled pipeline number, which is used to identify a specific pipeline.

[0105] S32: Obtain the orchestration data corresponding to the pipeline through the pipeline number. For example, a user has orchestrated a pipeline that aggregates service A and service B data. When the user needs to obtain the aggregated data, he sends a pipeline execution request. The aggregated service will find the orchestration data of the pipeline and start execution, and return the result after execution.

[0106] S33: Obtaining the orchestration steps to be executed one by one according to the content of the orchestration data, wherein the orchestration step is a collection of tasks, and the tasks in the orchestration step can be executed sequentially or in parallel according to the settings.

[0107] S34: Determine whether the type of the choreography step is a parallel step. If it is a parallel step, it means that the tasks in the choreography step can be executed in parallel. If it is a serial step, the tasks must be executed sequentially.

[0108] S35: If it is executed in parallel, then in step S351, all tasks in the orchestration step are acquired or taken out, and these tasks are executed in step S352. In step S353, all tasks are waited for to be completed. In step S354, it is determined whether all tasks are completed. If not, it returns to step S353. After all tasks are completed, it enters the execution of the next orchestration step. In step S355, it is determined whether all orchestration steps are completed. If it is the last orchestration step, it returns the result in step S356 (first to the aggregation controller, then to the console), and the process ends. If the current step is not the last step, it returns to step S33.

[0109] S36: If it is a serial execution, then in step S361, the tasks in the orchestration step are taken out in order, and these tasks are executed in step S362. After the current task is executed in step S363, it is determined whether the current task is the last task in the orchestration step. If it is not the last task in this orchestration step, it returns to step S361 to execute the next task. If it is the last task, the current orchestration step is terminated. In step S355, it is determined whether all orchestration steps are executed. If the current orchestration step is the last orchestration step, the result is returned in step S356 (first returned to the aggregation controller, then returned to the console), and the process ends.

[0110] Figure 8 is a task execution flow chart of an embodiment of the present invention, such as Figure 8 As shown, the task execution process performed by the aggregation controller includes the following steps:

[0111] S41: After starting the task, obtain the task type.

[0112] S42: First determine whether the task type is an interface request task.

[0113] S43: If the task type is an interface request task, determine whether it is necessary to enter parameters. If it is necessary to enter parameters, take the parameters from the parameter output (input from the outside or the output of the previous step and task) (get the parameters from the parameter output of the previous task) and pass them to the current task, and send a network request, including but not limited to HTTP requests. If no parameters are required, send the request directly according to the request content configured in the task. After sending the network request, receive the response data of the network request (HTTP and other requests are response-based protocols, and a response will be returned after sending the request, so you can directly receive it), and save the response data that needs to be saved according to the task configuration (the task configuration in the configuration), and the execution ends.

[0114] S44: If the task type is a data processing task, obtain the previously saved response data, extract the specified data according to the task configuration, perform corresponding calculations and processing, and generate new data. For example, the response data of service A contains three types of data: Aa, Ab, and Ac; the response data of service B contains three types of data: Ba, Bb, and Bc. The configuration file needs to calculate the sum of Aa data and Ba data. Then, take Aa data from the response data of service A, take Ba data from the response data of service B, and then add them together to get the result. After generating new data, save the new data and end the execution.

[0115] S45: If the task type is a calling task (other pipelines can be called), other scheduled and executable pipelines in the aggregation service 100 are called, the results of the called pipelines are obtained, the data is saved, and the process ends.

[0116] The embodiments of the present invention are of great significance for flexibly configuring and extending services that require operation and aggregation of multiple independent software service data.

[0117] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0118] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, each step of the above-mentioned pipeline interface orchestration aggregation method is implemented.

[0119] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. Of course, there are other ways of readable storage media, such as quantum memory, graphene memory, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0120] The embodiment of the present invention also provides a computer device, such as Fig. 9 As shown, it includes one or more processors 301 , a communication interface 302 , a memory 303 and a communication bus 304 , wherein the processor 301 , the communication interface 302 , and the memory 303 communicate with each other via the communication bus 304 .

[0121] Memory 303, used for storing computer programs;

[0122] The processor 301 is used to implement the steps of the above-mentioned pipeline interface orchestration aggregation method when executing the program stored in the memory 303.

[0123] The processor 301 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0124] The memory 303 may include a large capacity memory for data or instructions. For example, but not limitation, the memory 303 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 303 may include a removable or non-removable (or fixed) medium. In a specific embodiment, the memory 303 is a non-volatile solid-state memory. In a specific embodiment, the memory 303 includes a read-only memory (ROM). In appropriate cases, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0125] Communication bus 304 includes hardware, software or both, for coupling the above-mentioned components to each other.For example, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations.In suitable cases, bus may include one or more buses.Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.

[0126] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0127] Although the present application provides method operation steps such as embodiments or flow charts, more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the actual device or client product is executed, it can be executed in the order of the method shown in the embodiments or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0128] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0129] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0130] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0131] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A pipeline-based interface orchestration aggregation method, characterized in that: The method is performed by an aggregation service, and the method includes: Obtaining a configuration file submitted for interface orchestration, parsing the configuration file into orchestration steps and task data; When the console requests data, find the corresponding orchestration steps and task data according to the request of the console, execute the orchestration process according to the orchestration steps and task data, and request corresponding multiple services according to the orchestration process; receiving response data from the plurality of services, performing calculation processing according to the content of the task data and the response data, obtaining processing results, and returning the processing results to the console; Among them, when the console requests data, the corresponding orchestration steps and task data are found according to the request of the console, the orchestration process is executed according to the orchestration steps and task data, and the corresponding multiple services are requested according to the orchestration process; response data is received from the multiple services, and calculation processing is performed according to the content of the task data and the response data to obtain the processing result, and the processing result is returned to the console; specifically including: receiving a pipeline execution request, wherein the pipeline execution request carries an orchestrated pipeline number, and acquiring orchestration data corresponding to the pipeline according to the pipeline number; Acquire the arrangement steps to be executed one by one according to the content of the arrangement data; Determine whether the type of the current orchestration step is a parallel step; If the type of the current orchestration step is a parallel step, all tasks in the current orchestration step are taken out and executed in parallel. After all tasks are executed, the next orchestration step is executed. If the current orchestration step is the last orchestration step, the processing result is returned to the console. If the type of the current orchestration step is not a parallel step, the tasks in the current orchestration step are taken out in order and the tasks are executed serially in order. After the current task is executed, it is determined whether the current task is the last task in the current orchestration step. If it is not the last task in the current orchestration step, the next task is executed. If it is the last task in the current orchestration step, the current orchestration step is terminated. If the current orchestration step is the last orchestration step, the processing result is returned to the console. The task execution process includes the following steps: After starting the task, obtain the task type and determine whether the task type is an interface request task; If the task type is an interface request task, determine whether input parameters are required. If input parameters are required, obtain the input parameters from the parameter output and pass them to the current task, and send a network request; if input parameters are not required, directly send the network request according to the request content of the configuration file; after sending the network request, receive the response data of the network request, and save the response data according to the configuration file; If the task type is a data processing task, the previously saved response data is obtained, the specified response data is extracted according to the configuration file for corresponding calculation processing, new data is generated as the processing result and the processing result is saved; wherein the calculation processing includes: calculating the sum of specific fields in the response data of two interfaces, or merging multiple response data into one data; If the task type is a call task, call other scheduled and executable pipelines in the aggregation service to obtain the processing results and save the data; The obtaining of the submitted configuration file for interface orchestration and parsing of the configuration file into orchestration steps and tasks specifically includes: Receive configuration files and parse them; Determine whether the configuration format of the configuration file is correct; the correct configuration format of the configuration file means that the configuration file conforms to the standard yaml or json format specification, and the configuration content of the configuration file conforms to the specification that can be recognized by the aggregation service; if the configuration file contains characters that do not conform to the standard yaml or json format, the configuration format is directly considered to be incorrect; if the configuration file conforms to the standard yaml or json format, but the content does not conform to the format specification that can be recognized by the aggregation service, the configuration format is also considered to be incorrect; If the configuration format is correct, the orchestration steps and task data in the configuration file are parsed and converted into a program data structure and stored in a cache in the aggregation controller; If the configuration format is incorrect, the configuration file submission will be abandoned directly.

2. A pipeline-style interface arrangement and aggregation device, characterized in that: Applied in aggregation services, the device comprises: The orchestration engine is used to obtain the configuration file submitted for interface orchestration, parse the configuration file, and parse it into orchestration steps and task data; The aggregation controller is used to find the corresponding orchestration steps and task data according to the request of the console when the console requests data, execute the orchestration process according to the orchestration steps and task data, and request corresponding multiple services according to the orchestration process; receive response data from the multiple services, and perform calculation processing according to the content of the task data and the response data to obtain a processing result, and return the processing result to the console; The aggregation controller specifically includes: A request receiving module, used for receiving a pipeline execution request, wherein the pipeline execution request carries a scheduled pipeline number; An arrangement data acquisition module, used to acquire arrangement data corresponding to the pipeline according to the pipeline number; An arrangement step acquisition module, used to acquire the arrangement steps to be executed one by one according to the content of the arrangement data; An arrangement step type determination module is used to determine whether the type of the current arrangement step is a parallel step; A parallel task execution module, for taking out all the tasks in the current orchestration step and executing them in parallel if the type of the current orchestration step is a parallel step, and entering the execution of the next orchestration step after all the tasks are executed, and returning the processing result to the console if the current orchestration step is the last orchestration step; A serial task execution module, for, if the type of the current orchestration step is not a parallel step, taking out the tasks in the current orchestration step in order and executing these tasks in order in series, after the current task is executed, determining whether the current task is the last task in the current orchestration step, if it is not the last task, executing the next task, if it is the last task in the current orchestration step, ending the current orchestration step, and if the current orchestration step is the last orchestration step, returning the processing result to the console; The parallel task execution module or the serial task execution module specifically includes: The task type acquisition submodule is used to acquire the task type after starting the task and determine whether the task type is an interface request task; The interface request task processing submodule is used to determine whether input parameters are required if the task type is an interface request task. If input parameters are required, the input parameters are obtained from the parameter output and passed to the current task, and a network request is sent; if input parameters are not required, the network request is directly sent according to the request content of the configuration file; after sending the network request, the response data of the network request is received, and the response data is saved according to the configuration file; The data processing task processing submodule is used for obtaining previously saved response data if the task type is a data processing task, extracting the specified response data according to the configuration file for corresponding calculation processing, generating new data as the processing result and saving the processing result; wherein the calculation processing includes: calculating the sum of specific fields in the response data of two interfaces, or merging multiple response data into one data; The calling task processing submodule is used to call other already scheduled and executable pipelines in the aggregation service if the task type is a calling task, obtain the processing results and save the data; The orchestration engine specifically includes: A receiving module, used for receiving a configuration file; Configuration file parsing module, used to parse the configuration file; The format judgment module is used to judge whether the configuration format of the configuration file is correct; the configuration format of the configuration file is correct means that the configuration file conforms to the standard yaml or json format specification, and the configuration content of the configuration file conforms to the specification that can be recognized by the aggregation service; wherein, if the configuration file contains characters that do not conform to the standard yaml or json format, the configuration format is directly considered to be incorrect; if the configuration file conforms to the standard yaml or json format, but the content does not conform to the format specification that can be recognized by the aggregation service, the configuration format is also considered to be incorrect; The orchestration information parsing and saving module is used to parse the orchestration steps and task data in the configuration file if the configuration format is correct, and convert the orchestration steps and task data into the program data structure and save it in the cache of the aggregation controller; if the configuration format is incorrect, the configuration file submission is directly abandoned.

3. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the pipeline interface orchestration aggregation method as claimed in claim 1 is implemented.

4. A computer device, characterized in that: It includes: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the pipeline interface orchestration aggregation method according to claim 1.