Job generation method, apparatus and device, and storage medium
By determining the target job steps and target programs of the target main job in the sub-job generation and adding external and internal file definitions to it, the problems of poor sub-job queue queue queuing and flexibility in the existing technology are solved, and flexible queue inheritance and rapid changes of sub-jobs are realized.
Patent Information
- Application Number
- CN202411858825.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-17
- Publication Date
- 2025-04-11
AI Technical Summary
In the prior art, when sub-job generation, it is necessary to allocate exclusive input queues, but this has queue queuing problems and waste of resources. At the same time, the program associated with the main job writes a sub-job exclusive queue, which has poor flexibility and cannot quickly change the queue.
By determining the target job step and target program of the target main job, adding external file definitions to the target job step, adding internal file definitions to the target program, and adding job queue names to the external file definition, the target sub-job of the target main job is generated based on the linkage mechanism between the external file definition and the internal file definition.
It realizes that sub-jobs flexibly inherit the queue of the main job, and does not need to re-modify, compile, test and deploy programs when changing the queue, which improves the reliability and flexibility of sub-job generation.
Smart Images
Figure CN119806686A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of computer technology, in particular to the field of cloud computing technology, and specifically to a job generation method, apparatus, device and storage medium. Background Art
[0002] If a sub-job does not specify an input queue, the default input queue will be assigned, which may easily cause queues. In the actual operation of submitting a job, it is often necessary to specify an available input queue so that the job can obtain host resources and run faster. The current practice is generally to assign a dedicated input queue to a sub-job. When the program associated with the main job generates a sub-job, it specifies the dedicated input queue name in the job header of the sub-job.
[0003] Although the method of allocating exclusive input queues to sub-jobs and specifying exclusive input queue names in the job headers of sub-jobs when generating sub-jobs by the program associated with the main job can alleviate the queuing problem to a certain extent, there are two fundamental problems that are not completely solved: first, an exclusive input queue needs to be allocated, and when there are many sub-jobs, the exclusive input queue still needs to be queued, and when there are fewer sub-jobs, queue resources will be wasted; second, the method of hard-coding the exclusive input queue of the sub-job through the program associated with the main job is very inflexible. For example, if the input queue needs to be replaced, the program must be modified, compiled, tested and deployed, which not only increases extra workload and is prone to error correction, but more importantly, the input queue cannot be changed quickly and each change requires the program to be modified, compiled, tested and deployed again. Summary of the invention
[0004] The present application provides a job generation method, apparatus, device and storage medium to improve the reliability and flexibility of sub-job generation.
[0005] According to one aspect of the present application, a method for generating a job is provided, the method comprising:
[0006] Determine a target job step and a target program of a target main job; the target job step refers to a task for generating a sub-job; the target program refers to a program for generating a sub-job;
[0007] Adding an external file definition to the target job step and adding an internal file definition to the target program; the external file definition refers to the external file data that needs to be accessed or processed when the job is executed; the internal file definition refers to the internal file data generated when the job is executed;
[0008] A target sub-job of the target main job is generated according to the external file definition, the internal file definition and the target program.
[0009] According to another aspect of the present application, a job generation device is provided, the device comprising:
[0010] A data determination module, used to determine a target job step and a target program of a target main job; the target job step refers to a task for generating a sub-job; the target program refers to a program for generating a sub-job;
[0011] A data adding module, used to add external file definitions for the target job step, and to add internal file definitions for the target program; external file definitions refer to external file data that needs to be accessed or processed when the job is executed; internal file definitions refer to internal file data generated when the job is executed;
[0012] The job generation module is used to generate a target sub-job of the target main job according to the external file definition, the internal file definition and the target program.
[0013] According to another aspect of the present application, an electronic device is provided, the electronic device comprising:
[0014] one or more processors;
[0015] A memory for storing one or more programs;
[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement any one of the job generation methods provided in the embodiments of the present application.
[0017] According to another aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, any one of the job generation methods provided in the embodiments of the present application is implemented.
[0018] According to another aspect of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the computer program implements any one of the job generation methods provided in the embodiments of the present application.
[0019] The present application determines the target job step and target program of the target main job; the target job step refers to the task used to generate the sub-job; the target program refers to the program used to generate the sub-job; an external file definition is added to the target job step, and an internal file definition is added to the target program; the external file definition refers to the external file data that needs to be accessed or processed when the job is executed; the internal file definition refers to the internal file data generated when the job is executed; according to the external file definition, the internal file definition and the target program, the target sub-job of the target main job is generated. The above technical solution, by adding external file definitions and internal file definitions, can enable the sub-job to flexibly inherit the queue of the main job, and there is no need to re-modify, compile, test and deploy the program when changing the queue. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 is a flowchart of a method for generating a job according to Embodiment 1 of the present application;
[0021] Figure 2 is a flowchart of a method for generating a job according to Embodiment 2 of the present application;
[0022] Figure 3 It is a structural schematic diagram of a job generation device provided according to Embodiment 3 of the present application;
[0023] Figure 4 It is a structural schematic diagram of an electronic device that implements the job generation method of an embodiment of the present application. DETAILED DESCRIPTION
[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0026] In addition, it should be noted that in the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of relevant data such as target operation steps and target procedures are in compliance with the relevant laws and regulations and do not violate public order and good morals.
[0027] Embodiment 1
[0028] Figure 1This is a flowchart of a job generation method provided according to the first embodiment of the present application. This embodiment is applicable to the case where the job queue is inherited from the main job to the sub-job, and can be executed by a job generation device. The job generation device can be implemented in the form of hardware and / or software. The job generation device can be configured in a computer device, such as a server. Figure 1 As shown, the method includes:
[0029] S110, determine the target job step and target program of the target main job; the target job step refers to the task used to generate the sub-job; the target program refers to the program used to generate the sub-job.
[0030] In this embodiment, the target main job refers to the main job currently running on the job login system. The target job step refers to the task used to generate a sub-job, which is a step in the job execution process; it should be noted that a job may include at least a job step, each job step implements one or more functions, and one or more job steps are connected in series to form a non-empty job; specifically, in the context of the target main job, the target job step is a part of building or executing smaller and more specific tasks, which will eventually constitute the entire main job. The target program refers to the program or code used to generate sub-jobs; the target program can be a software application, a script, or a set of instructions for automating or executing tasks defined in the target job step. Sub-jobs refer to the decomposition of a larger job or task into smaller and more specific parts or components; each sub-job has its own determined set of inputs and outputs, and can be independently executed by different processors or computing nodes.
[0031] S120. Add external file definitions for the target job step, and add internal file definitions for the target program; external file definitions refer to external file data that needs to be accessed or processed when the job is executed; internal file definitions refer to internal file data generated when the job is executed.
[0032] In this embodiment, external file definitions refer to external file data that needs to be accessed or processed when the job is executed; these file data may include input data, configuration files, resource files, etc., which are necessary for the job execution. Internal file definitions refer to internal file data generated when the job is executed; these file data may be intermediate results, log files, output data, etc., which are created or modified during the job execution process.
[0033] Specifically, an external file definition is added to the sub-job generation job step of the main job; and an internal file definition is added to the program associated with the job step.
[0034] Optionally, adding an external file definition to a target job step may be to determine an external file associated with a target main job and a storage location of the external file; the external file includes an input file and an output file; and according to the external file and the storage location of the external file, adding an external file definition to the target job step.
[0035] In this embodiment, external files refer to files that are stored on a computer's external storage device (such as a hard disk, USB flash drive, optical disk, network storage, etc.) and are relatively independent of the currently executing program or process; these files are not part of the program itself, but may be read, written, or modified while the program is running. Input files refer to files that need to be provided to a job step for processing before executing the job step of the target main job; input files usually contain the original data or information required for the job. Output files refer to files containing processing results or data generated after the job step of the target main job is executed; output files are usually used for subsequent processing, analysis, or storage. The storage location of an external file refers to the specific location of the external file in the file system or storage medium, including the directory path, file name, etc.; the storage location is key information to ensure that the job step can correctly access the external file.
[0036] Optionally, adding an internal file definition to the target program can be to define the internal file in the target program and dynamically generate the storage location of the internal file based on the job environment of the target main job; an internal file refers to a temporary or intermediate file during the job execution process; based on the internal file and the storage location, add the internal file definition to the target job step.
[0037] In this embodiment, internal files refer to temporary or intermediate files generated during the execution of the target program; internal files are usually used to store temporary data, process results, or serve as input for subsequent operations. The operating environment refers to the environment in which the target main job and the target program are executed, including the operating system, hardware configuration, software dependencies, etc. The operating environment has an important impact on the generation and management of internal files. The storage location of the internal file refers to the specific location of the internal file in the file system or storage medium, including the directory path, file name, etc.; the storage location is key information to ensure that the job step can correctly access the internal file.
[0038] S130 , generating a target sub-job of the target main job according to the external file definition, the internal file definition and the target program.
[0039] In this embodiment, the target sub-job refers to a smaller and more specific job generated based on the target job steps and the target program; the target sub-job is a component of the target main job, each of which performs a specific task and together completes the goal of the entire main job.
[0040] Specifically, through the linkage between the external file definition and the internal file definition, the internal file definition obtains the required data from the external file definition, and uses the target program to generate the target sub-job of the target main job.
[0041] The embodiment of the present application determines the target job step and target program of the target main job; the target job step refers to the task used to generate the sub-job; the target program refers to the program used to generate the sub-job; an external file definition is added to the target job step, and an internal file definition is added to the target program; the external file definition refers to the external file data that needs to be accessed or processed when the job is executed; the internal file definition refers to the internal file data generated when the job is executed; according to the external file definition, the internal file definition and the target program, the target sub-job of the target main job is generated. The above technical solution, by adding external file definitions and internal file definitions, can enable the sub-job to flexibly inherit the queue of the main job, and there is no need to re-modify, compile, test and deploy the program when changing the queue.
[0042] Embodiment 2
[0043] Figure 2 This is a flow chart of a job generation method provided in accordance with the second embodiment of the present application. Based on the technical solutions of the above embodiments, this embodiment refines "generating a target sub-job of a target main job according to external file definition, internal file definition and target program" into "adding a job queue name in the external file definition; the job queue name refers to the queue name used to specify the job entering the job login system; based on the linkage mechanism between the external file definition and the internal file definition, generating a target sub-job of the target main job according to the target program". It should be noted that for the parts not described in detail in the embodiments of the present application, please refer to the relevant statements of other embodiments. Figure 2 As shown, the method includes:
[0044] S210, determine the target job step and target program of the target main job; the target job step refers to the task used to generate the sub-job; the target program refers to the program used to generate the sub-job.
[0045] S220. Add external file definitions for the target job step, and add internal file definitions for the target program; external file definitions refer to external file data that needs to be accessed or processed when the job is executed; internal file definitions refer to internal file data generated when the job is executed.
[0046] S230. Add a job queue name in the external file definition; the job queue name refers to the queue name used to specify the job entering the job login system.
[0047] In this embodiment, the job queue name is an identifier used to specify the queue of jobs entering the job login system; it is used to identify and manage different job queues in the job login system; each job queue corresponds to a group of jobs with similar characteristics or processing requirements; by assigning a common queue name to these jobs, the system can organize, schedule and execute these jobs more efficiently.
[0048] In an optional implementation, after adding an external file definition for the target job step, the job queue name may be directly added to the external file definition.
[0049] S240. Based on the linkage mechanism between the external file definition and the internal file definition, generate a target sub-job of the target main job according to the target program.
[0050] Optionally, based on the linkage mechanism between the external file definition and the internal file definition, the job queue name is written into the target sub-job; when the target sub-job submission is identified, the target program is used to generate the target sub-job of the target main job.
[0051] In this embodiment, the linkage mechanism refers to an interactive relationship established between the external file definition and the internal file definition, so that they can cooperate with each other to jointly complete a task or achieve a certain goal.
[0052] It is understandable that the child job is generated only after the main job is loaded into the memory and runs. The child job obtains the queue name from the SPOOL (Simultaneous Peripheral Operation Online, spooling technology) buffer pool, which is more direct and efficient.
[0053] Furthermore, after generating the target sub-job of the target main job, the target sub-job can also be submitted to the job login system and queued using the job queue name to wait for allocation of host resources; when it is recognized that the target sub-job has obtained the host resources, the target sub-job is executed.
[0054] In this embodiment, the job login system refers to a computer system for receiving, managing and scheduling jobs (including target sub-jobs) submitted by users; it generally includes functional modules such as job submission, job queuing, job scheduling and resource allocation; users can submit jobs through the system and view the status, progress and results of jobs. Host resources refer to the hardware and software resources required for executing jobs in a computer system; these resources include processors, memory, storage devices, input and output devices, operating systems and applications, etc.
[0055] Specifically, after the target sub-job obtains the required host resources, the computer system processes, calculates or analyzes the data according to predetermined steps and algorithms; during the job execution, the system monitors the execution status, progress and results of the job, and adjusts and optimizes resources as needed.
[0056] Exemplarily, the technical solution of the present application can be implemented by the following steps:
[0057] Step 1, add an external file definition in the sub-job generation job step of the main job.
[0058] Step 2: Add the queue name in the external file definition.
[0059] Step 3, add an internal file definition in the program associated with this job step.
[0060] Step 4: Get the queue name defined in the external file through the linkage between internal and external file definitions.
[0061] Step 5: Write the queue name obtained from the internal file definition into the child job.
[0062] Step 6: Submit the main job, trigger the program to generate and submit the sub-job.
[0063] Step 7: The sub-job enters the JES (Job Entry Subsystem) and queues for execution using the queue name defined in the external file.
[0064] Step 8: The sub-job obtains host resources and starts running, and the process ends.
[0065] The embodiment of the present application determines the target job step and target program of the target main job; the target job step refers to the task used to generate the sub-job; the target program refers to the program used to generate the sub-job; an external file definition is added to the target job step, and an internal file definition is added to the target program; the external file definition refers to the external file data that needs to be accessed or processed when the job is executed; the internal file definition refers to the internal file data generated when the job is executed; the job queue name is added to the external file definition; the job queue name refers to the queue name used to specify the job entering the job login system; based on the linkage mechanism between the external file definition and the internal file definition, the target sub-job of the target main job is generated according to the target program. The above technical solution, by adding external file definitions and internal file definitions, can enable the sub-job to flexibly inherit the queue of the main job, and there is no need to re-modify, compile, test and deploy the program when changing the queue.
[0066] Embodiment 3
[0067] Figure 3It is a structural diagram of a job generation device provided according to Example 3 of the present application, which can be applicable to the situation where the job queue is inherited from the main job to the sub-job. The job generation device can be implemented in the form of hardware and / or software. The job generation device can be configured in a computer device, such as a server.
[0068] like Figure 3 As shown, the device comprises:
[0069] The data determination module 310 is used to determine the target job step and target program of the target main job; the target job step refers to the task used to generate the sub-job; the target program refers to the program used to generate the sub-job;
[0070] The data adding module 320 is used to add external file definitions for the target job step and internal file definitions for the target program; external file definitions refer to external file data that needs to be accessed or processed when the job is executed; internal file definitions refer to internal file data generated when the job is executed;
[0071] The job generation module 330 is used to generate a target sub-job of a target main job according to an external file definition, an internal file definition and a target program.
[0072] The embodiment of the present application determines the target job step and target program of the target main job; the target job step refers to the task used to generate the sub-job; the target program refers to the program used to generate the sub-job; an external file definition is added to the target job step, and an internal file definition is added to the target program; the external file definition refers to the external file data that needs to be accessed or processed when the job is executed; the internal file definition refers to the internal file data generated when the job is executed; according to the external file definition, the internal file definition and the target program, the target sub-job of the target main job is generated. The above technical solution, by adding external file definitions and internal file definitions, can enable the sub-job to flexibly inherit the queue of the main job, and there is no need to re-modify, compile, test and deploy the program when changing the queue.
[0073] Optionally, the job generation module 330 includes:
[0074] The queue name adding unit is used to add the job queue name in the external file definition; the job queue name refers to the queue name used to specify the job entering the job login system;
[0075] The job generation unit is used to generate a target sub-job of a target main job according to a target program based on a linkage mechanism between an external file definition and an internal file definition.
[0076] Optionally, a job generation unit is used to:
[0077] Based on the linkage mechanism between external file definition and internal file definition, the job queue name is written into the target sub-job;
[0078] When the target sub-job submission is identified, the target program is used to generate the target sub-job of the target main job.
[0079] Optionally, the device further includes a job execution module; the job execution module is used to:
[0080] Submit the target sub-job to the job login system and queue it using the job queue name to wait for the allocation of host resources;
[0081] When it is identified that the target sub-job has acquired the host resources, the target sub-job is executed.
[0082] Optionally, the data adding module 320 is specifically used for:
[0083] Determine the external files associated with the target master job and the storage location of the external files; the external files include input files and output files;
[0084] Add external file definitions to the target job step based on the external file and its storage location.
[0085] Optionally, the data adding module 320 is further specifically used for:
[0086] Define internal files in the target program and dynamically generate storage locations for internal files based on the target main job's job environment; internal files refer to temporary or intermediate files during job execution;
[0087] Add internal file definitions to the target job step based on the internal files and storage locations.
[0088] The job generation device provided in the embodiments of the present application can execute the job generation method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects for executing each job generation method.
[0089] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.
[0090] Embodiment 4
[0091] Figure 44 is a schematic diagram of the structure of the electronic device 410 that implements the job generation method of the embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present application described and / or required herein.
[0092] like Figure 4 As shown, the electronic device 410 includes at least one processor 411, and a memory connected to the at least one processor 411 in communication, such as a read-only memory (ROM) 412, a random access memory (RAM) 413, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 411 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 412 or the computer program loaded from the storage unit 418 to the random access memory (RAM) 413. In RAM413, various programs and data required for the operation of the electronic device 410 can also be stored. The processor 411, ROM412 and RAM413 are connected to each other via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.
[0093] Multiple components in the electronic device 410 are connected to the I / O interface 415, including: an input unit 416, such as a keyboard, a mouse, etc.; an output unit 417, such as various types of displays, speakers, etc.; a storage unit 418, such as a disk, an optical disk, etc.; and a communication unit 419, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 419 allows the electronic device 410 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0094] The processor 411 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 411 executes the various methods and processes described above, such as a job generation method.
[0095] In some embodiments, the job generation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 410 via the ROM 412 and / or the communication unit 419. When the computer program is loaded into the RAM 413 and executed by the processor 411, one or more steps of the job generation method described above may be performed. Alternatively, in other embodiments, the processor 411 may be configured as the job generation method in any other appropriate manner (e.g., by means of firmware).
[0096] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0097] Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable job generating device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0098] In the context of the present application, a computer readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device or equipment. A computer readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer readable storage medium may be a machine readable signal medium. A more specific example of a machine readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0099] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0100] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0101] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0102] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution of this application can be achieved, and this document is not limited here.
[0103] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.
Claims
1. A method for generating a job, characterized in that: include: Determine a target job step and a target program of a target main job; the target job step refers to a task for generating a sub-job; the target program refers to a program for generating a sub-job; Adding an external file definition to the target job step and adding an internal file definition to the target program; the external file definition refers to the external file data that needs to be accessed or processed when the job is executed; the internal file definition refers to the internal file data generated when the job is executed; A target sub-job of the target main job is generated according to the external file definition, the internal file definition and the target program.
2. The method according to claim 1, characterized in that The step of generating a target sub-job of the target main job according to the external file definition, the internal file definition and the target program comprises: Add a job queue name in the external file definition; the job queue name is a queue name used to specify the job entering the job login system; Based on the linkage mechanism between the external file definition and the internal file definition, a target sub-job of the target main job is generated according to the target program.
3. The method according to claim 2, characterized in that The step of generating a target sub-job of the target main job based on the linkage mechanism between the external file definition and the internal file definition and according to the target program includes: Based on the linkage mechanism between the external file definition and the internal file definition, writing the job queue name into the target sub-job; When the target sub-job submission is identified, the target program is used to generate the target sub-job of the target main job.
4. The method according to claim 3, characterized in that After generating the target sub-job of the target main job, the method further includes: Submit the target sub-job to the job login system and queue it using the job queue name to wait for allocation of host resources; In the case of identifying that the target sub-job acquires the host resources, the target sub-job is executed.
5. The method according to claim 1, characterized in that Add external file definition for the target job step, including: Determine the external files associated with the target main job and the storage location of the external files; the external files include input files and output files; According to the external file and the storage location of the external file, an external file definition is added to the target job step.
6. The method according to claim 1, characterized in that Add internal file definitions for the target program, including: An internal file is defined in the target program, and a storage location of the internal file is dynamically generated according to the operating environment of the target main job; the internal file refers to a temporary or intermediate file during the execution of the job; An internal file definition is added to the target job step according to the internal file and the storage location.
7. A job generating device, characterized in that: include: A data determination module, used to determine a target job step and a target program of a target main job; the target job step refers to a task for generating a sub-job; the target program refers to a program for generating a sub-job; A data adding module, used to add external file definitions for the target job step, and to add internal file definitions for the target program; external file definitions refer to external file data that needs to be accessed or processed when the job is executed; internal file definitions refer to internal file data generated when the job is executed; The job generation module is used to generate a target sub-job of the target main job according to the external file definition, the internal file definition and the target program.
8. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the job generation method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the job generation method as described in any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the job generation method according to any one of claims 1 to 6.