Method, device and storage medium for generating a computing job

Abstract

This application provides a method, apparatus, and storage medium for generating computational jobs. The method includes: obtaining a first structured query language to create a draft entity containing the first structured query language; creating a first deployment entity in streaming mode based on the draft entity, obtaining first configuration information of the first deployment entity, and starting the first deployment entity to create a first instance in streaming mode and a first computational job corresponding to the first instance and running according to the first configuration information; creating a second deployment entity in batch mode based on the draft entity, obtaining second configuration information of the second deployment entity, and starting the second deployment entity to create a second instance in batch mode and a second computational job corresponding to the second instance and running according to the second configuration information, ensuring consistency between the results of streaming jobs and batch jobs.

Description

Technical Field

[0001] This application relates to the field of cloud computing, and more specifically, to methods, devices, and storage media for generating computing jobs. Background Technology

[0002] In big data processing, batch computing and stream computing are very important concepts. Generally speaking, in batch computing, data is collected first and then processed, while stream computing processes data in real time. The computing engine supports running computing jobs in either streaming or batch mode.

[0003] The current mainstream architecture is to use a stream computing scheme and a batch computing scheme in parallel. The two schemes are developed and maintained independently. However, the inventors discovered that for the same computing needs and the same data sources, the computing results obtained by using stream jobs and batch jobs are often different. Summary of the Invention

[0004] This application provides a method, apparatus, and storage medium for generating computational jobs in order to ensure the consistency of results between streaming jobs and batch jobs.

[0005] In a first aspect, this application provides a method for generating computational jobs, comprising:

[0006] Obtain a first structured query language to create a draft entity containing the first structured query language;

[0007] Based on the draft entity, a first deployment entity in streaming mode is created, first configuration information of the first deployment entity is obtained, and the first deployment entity is started to create a first instance in streaming mode and a first computing job corresponding to the first instance and running according to the first configuration information.

[0008] Based on the draft entity, a second deployment entity in batch mode is created, the second configuration information of the second deployment entity is obtained, and the second deployment entity is started to create a second instance in batch mode and a second computing job corresponding to the second instance and running according to the second configuration information.

[0009] Optional, also includes:

[0010] Obtain the second structured query language, and update the first structured query language in the draft entity to the second structured query language;

[0011] The target deployment entity is updated based on the updated draft entity, and the target deployment entity includes the first deployment entity and / or the second deployment entity.

[0012] Optionally, the target deployment entity includes the first deployment entity, and the method further includes:

[0013] Stop the first instance entity;

[0014] The updated first deployment entity is initiated to create a third instance in streaming mode and a third compute job corresponding to the third instance, running according to the first configuration information.

[0015] Optionally, the target deployment entity includes the second deployment entity, and the method further includes:

[0016] The updated second deployment entity is initiated to create a fourth instance in batch mode and a fourth compute job corresponding to the fourth instance, which runs according to the second configuration information.

[0017] Optional, also includes:

[0018] Modify the configuration information of the target deployment entity to the target configuration information, wherein the target deployment entity is the first deployment entity, the configuration information of the target deployment entity is the first configuration information, and the target configuration information is the third configuration information; or, the target deployment entity is the second deployment entity, the configuration information of the target deployment entity is the second configuration information, and the target configuration information is the fourth configuration information.

[0019] Optionally, the target deployment entity is the first deployment entity, and the method further includes:

[0020] Stop the first instance entity;

[0021] The updated first deployment entity is initiated to create a fifth instance in streaming mode and a fifth compute job corresponding to the fifth instance, running according to the third configuration information.

[0022] Optionally, the target deployment entity is the second deployment entity, and the method further includes:

[0023] The updated second deployment entity is initiated to create a sixth instance in batch mode and a sixth compute job corresponding to the sixth instance, which runs according to the fourth configuration information.

[0024] Secondly, this application provides a method for generating computational jobs, including:

[0025] In response to a draft entity creation request, a draft entity containing a first structured query language is created, wherein the draft creation request includes the first structured query language;

[0026] In response to a request to create a streaming mode deployment entity, a first deployment entity in streaming mode is created based on the draft entity; a first configuration instruction for the first deployment entity is received, the first configuration instruction including first configuration information; in response to a first start instruction, the first deployment entity is started to create a first instance in streaming mode and a first computing job corresponding to the first instance and running according to the first configuration information;

[0027] In response to a batch mode deployment entity creation request, a second batch mode deployment entity is created based on the draft entity; a second configuration instruction for the second deployment entity is received, the second configuration instruction including second configuration information; in response to a second start instruction, the second deployment entity is started to create a second instance of batch mode and a second computing job corresponding to the second instance, running according to the second configuration information.

[0028] Thirdly, this application provides a method for generating computational jobs, including:

[0029] Send a draft entity creation request to the computing platform. The draft entity creation request includes a first structured query language input by the user, which is used to create draft entities.

[0030] Send a streaming mode deployment entity creation request to the computing platform to enable the computing platform to create a first streaming mode deployment entity based on the draft entity; send a first configuration instruction for the first deployment entity to the computing platform, the first configuration instruction including first configuration information; send a first start instruction to the computing platform to enable the computing platform to start the first deployment entity to create a first instance of streaming mode and a first computing job corresponding to the first instance and running according to the first configuration information;

[0031] A batch mode deployment entity creation request is sent to the computing platform to enable the computing platform to create a second batch mode deployment entity based on the draft entity; a second configuration instruction for the second deployment entity is sent to the computing platform, the second configuration instruction including second configuration information; a second start instruction is sent to the computing platform to enable the computing platform to start the second deployment entity to create a second instance of batch mode and a second computing job corresponding to the second instance and running according to the second configuration information.

[0032] Fourthly, this application provides a computing platform, including: a memory and a processor;

[0033] The memory is used to store computer programs;

[0034] The processor is configured to execute a computer program stored in the memory, wherein the computer program, when executed, causes the processor to perform the method described in the first aspect or the second aspect.

[0035] Fifthly, this application provides a client, including: a memory and a processor;

[0036] The memory is used to store computer programs;

[0037] The processor is configured to execute a computer program stored in the memory, wherein the computer program, when executed, causes the processor to perform the method described in the third aspect.

[0038] In a sixth aspect, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the methods described in the first, second, or third aspects.

[0039] In the computational job generation method, device, and storage medium provided in this application, the draft entity includes a first structured query language. Based on this draft entity, both streaming jobs and batch jobs can be created. Since streaming jobs and batch jobs are created based on the same draft entity, both streaming jobs and batch jobs perform data processing based on the first structured query language. The data processing steps performed in streaming jobs and batch jobs, such as data query and data calculation, are exactly the same, thereby ensuring that for the same computational needs and the same data source, the results of streaming jobs and batch jobs are consistent. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram of a computational job generation chain provided in an embodiment of this application. Figure 1 ;

[0042] Figure 2 This is a schematic diagram of a computational job generation chain provided in an embodiment of this application. Figure 2 ;

[0043] Figure 3 This is a schematic diagram illustrating the correspondence between entities provided in an embodiment of this application;

[0044] Figure 4This is a flowchart illustrating another method for generating computational jobs provided in an embodiment of this application;

[0045] Figure 5 This is a schematic diagram illustrating the process of creating a computational job provided in an embodiment of this application;

[0046] Figure 6 This is a schematic diagram illustrating the process of updating SQL code provided in an embodiment of this application;

[0047] Figure 7 This is a schematic diagram illustrating the process of updating configuration information provided in an embodiment of this application;

[0048] Figure 8 This is a schematic block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0050] First, the terminology used in the embodiments of this application will be introduced.

[0051] 1. Stream computing: Processes the data stream immediately when it is generated, where the data size is unknown and infinitely continuous.

[0052] 2. Batch computing: Processing and analyzing a set of data that has been stored for a period of time, when the data size is known and finite.

[0053] 3. Stream and Batch Processing Integration: Using the same development paradigm to achieve both stream and batch processing of big data, thereby ensuring consistency between the processing and the results and reducing development and maintenance costs.

[0054] 4. Configuration information: Calculates the runtime parameters of the job, including but not limited to memory, CPU, and state information.

[0055] 5. Draft: A draft entity containing Structured Query Language (SQL) code, providing a development and debugging environment and a deployment checkpoint.

[0056] 6. Deployment: An entity generated by Draft that contains SQL code, batch attributes, and configuration information, used to create an instance.

[0057] 7. Instance: An entity generated by the Deployment that contains SQL code, batch attributes, and configuration information, and corresponds one-to-one with a job.

[0058] 8. Job: The actual execution entity of a streaming job or batch job.

[0059] Currently, the mainstream approach of building separate stream computing and batch computing solutions mainly suffers from the following two problems:

[0060] 1. Inconsistent Results: Since stream jobs and batch jobs are developed independently, the SQL code in stream jobs and batch jobs will differ for the same computing needs. These differences in SQL code may lead to differences in the data query process and data calculation process during the execution of stream jobs and batch jobs, resulting in inconsistent final calculation results.

[0061] 2. High development and maintenance costs: Since stream computing and batch computing solutions are independent, with different model designs, different code, and are developed or maintained by different personnel, they require twice the manpower compared to integrated stream and batch computing. Furthermore, when the data of stream jobs and batch jobs are inconsistent and statistical changes are required, multiple people need to collaborate.

[0062] To address the aforementioned issues, this application provides a unified batch and stream processing solution. Through a unified model design and a single set of SQL code, it eliminates inconsistencies between batch and stream results and reduces operation and maintenance costs. Figure 1 As shown in the embodiments of this application, the integrated batch processing solution includes a four-layer model: Draft, Deployment, Instance, and Job. The computational job generation chain is as follows: Draft generates Deployment, Deployment generates Instance, and then Instance generates Job. Wherein, as... Figure 2 As shown, the Draft includes SQL code, which is the computation logic; the Deployment includes SQL code, Configuration information, and the Deployment type, i.e., whether the Deployment is in streaming or batch mode; the Instance includes SQL code, Configuration information, and the Instance type, i.e., whether the Instance is in streaming or batch mode. The process of generating a computation job is as follows:

[0063] 1: Store SQL code on Draft.

[0064] 2: Create or update Deployments from Draft. When creating or updating a Deployment, select the type of Deployment to be created or updated, which can be either streaming mode or batch mode.

[0065] 3: Edit the Configuration information on the Deployment to determine the running parameters for the corresponding streaming or batch job.

[0066] 4: Start the Deployment to create an Instance, and finally create the actual running Job, i.e., the corresponding streaming job or batch job.

[0067] like Figure 3 As shown in this embodiment, two Deployments can be created based on each Draft: one streaming Deployment (stream Deployment) and one batch Deployment (batch Deployment). An unlimited number of Instances can be created based on each Deployment, and the Configuration information for each Instance can be different. That is, an unlimited number of streaming Instances can be created based on a streaming Deployment, and an unlimited number of batch Instances can be created based on a batch Deployment. Each Instance can also create a Job; specifically, a streaming Job is created based on each streaming Instance, and a batch Job is created based on each batch Instance.

[0068] Figure 4 This is a flowchart illustrating a method for generating computational jobs according to an embodiment of this application. The method of this embodiment is applied to a computing platform. Figure 4 As shown, the method includes:

[0069] S401. Obtain the first structured query language to create a draft entity containing the first structured query language.

[0070] The first structured query language, or first SQL code, is used to implement specific computational logic. In this embodiment, a streaming computational job (i.e., a streaming job) or a batch computational job (i.e., a batch job) can be created based on the first structured query language. Referring to the foregoing description, in this embodiment, a draft entity including the first structured query language is first created. Subsequently, a streaming deployment entity and a batch deployment entity can be created based on the draft entity.

[0071] S402. Create a first deployment entity in streaming mode based on the draft entity, obtain the first configuration information of the first deployment entity, and start the first deployment entity to create a first instance in streaming mode and a first computing job corresponding to the first instance and running according to the first configuration information.

[0072] The first deployment entity includes the first structured query language and the type stream mode of the first deployment entity found in the draft entity, and may also include first configuration information. The first configuration information includes, but is not limited to, information on memory, CPU, and status, and may also include start time information or start condition information for the streaming job. The first deployment entity is started to create a first instance. The first instance includes the first structured query language, the type stream mode of the first instance, and the first configuration information. The first instance then generates a first computation job, which is a streaming job and runs according to the first configuration information.

[0073] S403. Create a second deployment entity in batch mode based on the draft entity, obtain the second configuration information of the second deployment entity, and start the second deployment entity to create a second instance in batch mode and a second computing job corresponding to the second instance and running according to the second configuration information.

[0074] The second deployment entity includes the first Structured Query Language (SCL) from the draft entity and the batch mode of the second deployment entity's type, and may also include second configuration information. The second configuration information includes, but is not limited to, information on memory, CPU, and status, and may also include start and end time information or start and end condition information for the batch job. The second deployment entity is started to generate a second instance. The second instance includes the first SCL, the batch mode of the second instance's type, and the second configuration information. The second instance then generates a second computation job, which is a batch job and runs according to the second configuration information.

[0075] In the method provided in this application embodiment, the draft entity includes a first structured query language. Based on the draft entity, streaming jobs and batch jobs can be created, that is, both streaming jobs and batch jobs are run based on the first structured query language. In this way, the data processing steps performed during the execution of streaming jobs and batch jobs, such as data query and data calculation, are exactly the same, thereby ensuring that for the same calculation requirements and the same data source, the results of streaming jobs and batch jobs are consistent.

[0076] In the above embodiments, after creating the first and second computation jobs based on the draft entity, new computation jobs can be created by updating the structured query language in the draft entity.

[0077] Optionally, a second structured query language is obtained, and the first structured query language in the draft entity is updated to the second structured query language. Based on the updated draft entity, the target deployment entity is updated, and the target deployment entity includes the first deployment entity and / or the second deployment entity. The updated target deployment entity includes the second structured query language. It should be noted that after updating the target deployment entity, the updated target deployment entity may not be started immediately; both the first and second computation jobs will continue to run.

[0078] Since streaming jobs process data in real time without time limits, when it is necessary to start the updated first deployment entity to create a new streaming computing job, the first computing job must be stopped first. Optionally, if the target deployment entity includes the first deployment entity, the first instance entity is stopped; the updated first deployment entity is started to create a third instance in streaming mode and a third computing job corresponding to the third instance that runs according to the first configuration information.

[0079] Since batch jobs process data before processing and are limited in time, when it is necessary to start the updated second deployment entity to generate a new batch computing job, it is not necessary to stop the second computing job. Optionally, if the target deployment entity includes the second deployment entity, the updated second deployment entity is started to create a fourth instance in batch mode and a fourth computing job corresponding to the fourth instance that runs according to the second configuration information.

[0080] In the above embodiments, after creating the first and second computing jobs based on the draft entity, new computing jobs can also be created by updating the configuration information of the deployment entity.

[0081] Optionally, the configuration information of the target deployment entity can be modified to target configuration information, wherein the target deployment entity is the first deployment entity, the configuration information of the target deployment entity is the first configuration information, and the target configuration information is the third configuration information; or, the target deployment entity is the second deployment entity, the configuration information of the target deployment entity is the second configuration information, and the target configuration information is the fourth configuration information.

[0082] Since streaming jobs process data in real time without time limits, when it is necessary to start the updated first deployment entity to create a new streaming computing job, the first computing job must be stopped first. Optionally, if the target deployment entity is the first deployment entity, the first instance entity is stopped; the updated first deployment entity is started to create the fifth instance in streaming mode and the fifth computing job corresponding to the fifth instance, which runs according to the third configuration information.

[0083] Since batch jobs process data before processing and are limited in time, when it is necessary to start the updated second deployment entity to create a new batch computing job, it is not necessary to stop the second computing job. Optionally, if the target deployment entity is the second deployment entity, the updated second deployment entity is started to create a sixth instance in batch mode and a sixth computing job corresponding to the sixth instance, which runs according to the fourth configuration information.

[0084] The following section describes the three processes from the user's perspective: creating computing jobs, updating SQL code, and configuring information through the client.

[0085] Reference Figure 5 As shown, the process of creating a computation job includes:

[0086] 1: Users create a Draft and fill in the SQL code on the Draft.

[0087] 2: Click Create Deployment, then select Create Stream Deployment or Batch Deployment.

[0088] 3: Configure the Configuration information on the Deployment, click Start to create the Instance and finally create the Job.

[0089] Reference Figure 6 As shown, the process of updating the SQL code includes:

[0090] 1. The user determines the draft to be modified and then modifies the SQL code on the draft.

[0091] 2: Click Update Deployment, then select Update Stream Deployment or Batch Deployment.

[0092] For update stream Deployments, optional,

[0093] 3: Click Stop to stop the flow of the Deployment corresponding to the Instance.

[0094] 4: Click Start Stream Deployment to create a new stream instance corresponding to the new SQL code and eventually create a new stream Job.

[0095] For updating batch Deployments, optional.

[0096] 5: Click Start Batch Deployment to create a batch instance corresponding to the new SQL code and eventually create a new batch Job.

[0097] Reference Figure 7 As shown, the process of updating configuration information includes:

[0098] 1. The user finds the stream Deployment or batch Deployment to be modified and modifies the Configuration information on the Deployment.

[0099] For update stream Deployments, optional,

[0100] 2: Click Stop to stop the flow of the instance corresponding to the Deployment.

[0101] 3: Click to start the Deployment, which will create a new Stream Instance corresponding to the new Configuration information and finally create a new Stream Job.

[0102] For updating batch Deployments, optional.

[0103] 4: Click to start batch Deployment, create a batch instance corresponding to the new Configuration information, and finally create a new batch Job.

[0104] This application's integrated stream and batch computing solution enables the simultaneous generation and maintenance of both stream computing and batch computing jobs based on a single set of SQL code. Users only need to write the SQL code once during development to quickly create the corresponding stream and batch jobs. Users no longer need to manually manage multiple development processes. All stream and batch logic code is handled by the Draft entity. Each Draft can generate both stream and batch Deployments, ensuring complete consistency in the computation process of both jobs while requiring only one development effort. Different parameters during stream and batch runtime are determined by modifying the Configuration of the two Deployments. Runtime parameters are separated from the SQL code; users modifying runtime parameters do not change the SQL code. Through a four-layer model structure of Draft, Deployment, Instance, and Job, the inconsistency between stream computing and batch computing jobs from development to maintenance is shielded, ensuring data consistency and reducing development and maintenance costs.

[0105] The following section describes the methods executed on both the client and computing platform sides, taking into account the interaction between the client and the computing platform.

[0106] The client executes the following method:

[0107] Send a draft entity creation request to the computing platform. The draft entity creation request includes a first structured query language input by the user. The first structured query language is used to create draft entities.

[0108] Send a streaming mode deployment entity creation request to the computing platform to enable the computing platform to create a first streaming mode deployment entity based on the draft entity; send a first configuration instruction to the computing platform for the first deployment entity, the first configuration instruction including first configuration information; send a first start instruction to the computing platform to enable the computing platform to start the first deployment entity to create a first instance of streaming mode and a first computing job corresponding to the first instance and running according to the first configuration information;

[0109] A batch mode deployment entity creation request is sent to the computing platform to enable the computing platform to create a second batch mode deployment entity based on the draft entity; a second configuration instruction for the second deployment entity is sent to the computing platform, the second configuration instruction including second configuration information; a second start instruction is sent to the computing platform to enable the computing platform to start the second deployment entity to create a second instance of batch mode and a second computing job corresponding to the second instance and running according to the second configuration information.

[0110] Requests and instructions sent by the client to the computing platform can be triggered by user actions.

[0111] Optionally, a draft entity update request is sent to the computing platform to update the first structured query language in the draft entity to the second structured query language;

[0112] Send a target deployment entity update request to the computing platform to update the first structured query language in the target deployment entity to a second structured query language. The target deployment entity includes the first deployment entity and / or the second deployment entity.

[0113] Optionally, the target deployment entity includes the first deployment entity, and the methods executed by the client also include:

[0114] Send a first stop command to the computing platform to stop the first instance entity;

[0115] A third startup command is sent to the computing platform to start the updated first deployment entity, thereby creating a third instance in streaming mode and a third computing job corresponding to the third instance, which runs according to the first configuration information.

[0116] Optionally, the target deployment entity may include a second deployment entity, and the methods executed by the client may also include:

[0117] A fourth startup command is sent to the computing platform to initiate the updated second deployment entity, thereby creating a fourth instance in batch mode and a fourth computing job corresponding to the fourth instance, which runs according to the second configuration information.

[0118] Optionally, a configuration information update instruction for the target deployment entity is sent to the computing platform to modify the configuration information of the target deployment entity to the target configuration information, wherein the target deployment entity is a first deployment entity, the configuration information of the target deployment entity is the first configuration information, and the target configuration information is the third configuration information; or, the target deployment entity is a second deployment entity, the configuration information of the target deployment entity is the second configuration information, and the target configuration information is the fourth configuration information.

[0119] Optionally, the target deployment entity is the first deployment entity, and the methods executed by the client also include:

[0120] Send a second stop command to the computing platform to stop the first instance entity;

[0121] A fifth start command is sent to the computing platform to initiate the updated first deployment entity, thereby creating a fifth instance in streaming mode and a fifth computing job corresponding to the fifth instance, which runs according to the third configuration information.

[0122] Optionally, the target deployment entity is the second deployment entity, and the methods executed by the client also include:

[0123] A sixth start command is sent to the computing platform to initiate the updated second deployment entity, thereby creating a sixth instance in batch mode and a sixth computing job corresponding to the sixth instance, which runs according to the fourth configuration information.

[0124] The computing platform executes the following method:

[0125] In response to a draft entity creation request, a draft entity containing a first structured query language is created, wherein the draft creation request includes the first structured query language.

[0126] In response to a request to create a streaming mode deployment entity, a first deployment entity in streaming mode is created based on a draft entity; a first configuration instruction for the first deployment entity is received, the first configuration instruction including first configuration information; in response to a first start instruction, the first deployment entity is started to create a first instance in streaming mode and a first computing job corresponding to the first instance and running according to the first configuration information;

[0127] In response to a batch mode deployment entity creation request, a second batch mode deployment entity is created based on the draft entity; a second configuration instruction for the second deployment entity is received, the second configuration instruction including second configuration information; in response to a second start instruction, the second deployment entity is started to create a second instance of batch mode and a second computing job corresponding to the second instance, which runs according to the second configuration information.

[0128] Optionally, in response to a draft entity update request, the first structured query language in the draft entity is updated to the second structured query language;

[0129] In response to a target deployment entity update request, the first structured query language in the target deployment entity is updated to the second structured query language. The target deployment entity includes the first deployment entity and / or the second deployment entity.

[0130] Optionally, the target deployment entity includes the first deployment entity, and the method executed by the computing platform further includes:

[0131] In response to the first stop command, the first instance entity is stopped;

[0132] In response to the third startup command, the updated first deployment entity is started to create a third instance in streaming mode and a third compute job corresponding to the third instance, running according to the first configuration information.

[0133] Optionally, the target deployment entity includes a second deployment entity, and the method executed by the computing platform further includes:

[0134] In response to the fourth startup command, the updated second deployment entity is started to create a fourth instance in batch mode and a fourth compute job corresponding to the fourth instance, which runs according to the second configuration information.

[0135] Optionally, in response to the configuration information update command, the configuration information of the target deployment entity is modified to the target configuration information, wherein the target deployment entity is the first deployment entity, the configuration information of the target deployment entity is the first configuration information, and the target configuration information is the third configuration information; or, the target deployment entity is the second deployment entity, the configuration information of the target deployment entity is the second configuration information, and the target configuration information is the fourth configuration information.

[0136] Optionally, the target deployment entity is the first deployment entity, and the method executed by the computing platform further includes:

[0137] In response to the second stop command, the first instance entity is stopped;

[0138] In response to the fifth startup command, the updated first deployment entity is started to create a fifth instance in streaming mode and a fifth compute job corresponding to the fifth instance, running according to the third configuration information.

[0139] Optionally, the target deployment entity is a second deployment entity, and the method executed by the computing platform further includes:

[0140] In response to the sixth startup command, the updated second deployment entity is launched to create a sixth instance in batch mode and a sixth compute job corresponding to the sixth instance, which runs according to the fourth configuration information.

[0141] Figure 8 This is a schematic block diagram of the electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device 800 may include at least one processor 801 for implementing the method for generating computing jobs provided in the embodiments of this application. The electronic device 800 may be the aforementioned computing platform or client.

[0142] Optionally, the electronic device 800 further includes at least one memory 802 for storing program instructions and / or data. The memory 802 is coupled to the processor 801. The coupling in this embodiment is an indirect coupling or communication connection between devices, units, or modules, and can be electrical, mechanical, or other forms, used for information exchange between devices, units, or modules. The processor 801 may operate in conjunction with the memory 802. The processor 801 may execute program instructions stored in the memory 802. At least one of the at least one memory may be included in the processor.

[0143] Optionally, the electronic device 800 further includes a communication interface 803 for communicating with other devices via a transmission medium, thereby enabling the electronic device 800 to communicate with other devices. The communication interface 803 may be, for example, a transceiver, interface, bus, circuit, or a device capable of transmitting and receiving functions. The processor 801 can utilize the communication interface 803 to transmit and receive data and / or information, and to implement the methods provided in the embodiments of this application. For details, please refer to the detailed descriptions in the preceding embodiments; further elaboration is not repeated here.

[0144] This application embodiment does not limit the specific connection medium between the processor 801, memory 802, and communication interface 803. This application embodiment... Figure 8 The processor 801, memory 802, and communication interface 803 are connected via bus 804. Bus 804 is... Figure 8 The connections between other components are shown in bold lines only and are not intended to be limiting. This bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0145] It should be understood that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiments can be completed by the integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0146] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0147] This application also provides a computer-readable storage medium storing a computer program (also referred to as code or instructions). When the computer program is run, it causes the computer to perform the methods as described in any of the foregoing embodiments.

[0148] The terms “unit”, “module”, etc., used in this specification may be used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution.

[0149] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. In the several embodiments provided in this application, it should be understood that the disclosed apparatus, devices, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0150] The unit described as a separate component may or may not be physically separate. The component shown as a unit may or may not be a physical unit; that is, it may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0151] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0152] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. This computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disk, hard disk, magnetic tape), optical media (e.g., digital video disc (DVD)), or semiconductor media (e.g., solid-state disk (SSD)).

[0153] If this function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0154] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0155] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for generating computational jobs, characterized in that, include: Obtain a first structured query language to create a draft entity containing the first structured query language; Based on the draft entity, a first deployment entity in streaming mode is created, first configuration information of the first deployment entity is obtained, and the first deployment entity is started to create a first instance in streaming mode and a first computing job corresponding to the first instance and running according to the first configuration information. Based on the draft entity, a second deployment entity in batch mode is created, the second configuration information of the second deployment entity is obtained, and the second deployment entity is started to create a second instance in batch mode and a second computing job corresponding to the second instance and running according to the second configuration information.

2. The method according to claim 1, characterized in that, Also includes: Obtain the second structured query language, and update the first structured query language in the draft entity to the second structured query language; The target deployment entity is updated based on the updated draft entity, and the target deployment entity includes the first deployment entity and / or the second deployment entity.

3. The method according to claim 2, characterized in that, The target deployment entity includes the first deployment entity, and the method further includes: Stop the first instance entity; The updated first deployment entity is initiated to create a third instance in streaming mode and a third compute job corresponding to the third instance, running according to the first configuration information.

4. The method according to claim 2, characterized in that, The target deployment entity includes the second deployment entity, and the method further includes: The updated second deployment entity is initiated to create a fourth instance in batch mode and a fourth compute job corresponding to the fourth instance, which runs according to the second configuration information.

5. The method according to claim 1, characterized in that, Also includes: Modify the configuration information of the target deployment entity to the target configuration information, wherein the target deployment entity is the first deployment entity, the configuration information of the target deployment entity is the first configuration information, and the target configuration information is the third configuration information; or, the target deployment entity is the second deployment entity, the configuration information of the target deployment entity is the second configuration information, and the target configuration information is the fourth configuration information.

6. The method according to claim 5, characterized in that, The target deployment entity is the first deployment entity, and the method further includes: Stop the first instance entity; The updated first deployment entity is initiated to create a fifth instance in streaming mode and a fifth compute job corresponding to the fifth instance, running according to the third configuration information.

7. The method according to claim 5, characterized in that, The target deployment entity is the second deployment entity, and the method further includes: The updated second deployment entity is initiated to create a sixth instance in batch mode and a sixth compute job corresponding to the sixth instance, which runs according to the fourth configuration information.

8. A method for generating computational jobs, characterized in that, include: In response to a draft entity creation request, a draft entity containing a first structured query language is created, wherein the draft creation request includes the first structured query language; In response to a request to create a streaming mode deployment entity, a first deployment entity in streaming mode is created based on the draft entity; a first configuration instruction for the first deployment entity is received, the first configuration instruction including first configuration information; In response to the first startup command, the first deployment entity is started to create a first instance in streaming mode and a first computing job corresponding to the first instance, running according to the first configuration information; In response to a batch mode deployment entity creation request, a second batch mode deployment entity is created based on the draft entity; a second configuration instruction for the second deployment entity is received, the second configuration instruction including second configuration information; in response to a second start instruction, the second deployment entity is started to create a second instance of batch mode and a second computing job corresponding to the second instance, running according to the second configuration information.

9. A method for generating computational jobs, characterized in that, include: Send a draft entity creation request to the computing platform. The draft entity creation request includes a first structured query language input by the user, which is used to create draft entities. Send a streaming mode deployment entity creation request to the computing platform to enable the computing platform to create a first deployment entity in streaming mode based on the draft entity; send a first configuration instruction for the first deployment entity to the computing platform, the first configuration instruction including first configuration information; Send a first start command to the computing platform to start the first deployment entity to create a first instance in streaming mode and a first computing job corresponding to the first instance and running according to the first configuration information; Send a batch mode deployment entity creation request to the computing platform so that the computing platform creates a second deployment entity in batch mode based on the draft entity; Send a second configuration instruction for the second deployment entity to the computing platform, the second configuration instruction including second configuration information; A second startup command is sent to the computing platform to cause the computing platform to start the second deployment entity to create a second instance in batch mode and a second computing job corresponding to the second instance and running according to the second configuration information.

10. A computing platform, characterized in that, include: Memory and processor; The memory is used to store computer programs; The processor is configured to execute a computer program stored in the memory, wherein when the computer program is executed, the processor performs the method described in any one of claims 1-8.

11. A client, characterized in that, include: Memory and processor; The memory is used to store computer programs; The processor is configured to execute a computer program stored in the memory, wherein the computer program, when executed, causes the processor to perform the method of claim 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method as described in any one of claims 1-9.

Images