A method, apparatus and device for regulating code development

Abstract

The application relates to a method, device and equipment for standardizing code development, and belongs to the technical field of software development. The application creates a to-be-developed report in response to a target report creation instruction by calling a target report creation interface; creates a job based on a task index in response to a job creation instruction by calling a job creation interface, and determines a corresponding execution engine; then creates a corresponding temporary table in response to a temporary table creation instruction by calling a temporary table creation interface; develops a code segment of the temporary table according to the task index to obtain a corresponding developed temporary table; and based on a preset logical rule, associates and summarizes a plurality of developed temporary tables to obtain a created target report, and verifies the correctness of the target report through the execution engine. In the code development process, the application creates a job with a plurality of temporary tables to constrain and standardize the development of the code, thereby reducing the code development cost and making the code maintenance and management more convenient.

Description

Technical Field

[0001] This invention belongs to the field of software development technology, and specifically relates to a method, apparatus and equipment for developing standardized code. Background Technology

[0002] With the continuous development of Internet technology, the demand for data processing is increasing, requiring data analysis technicians to develop a large amount of SQL (Structured Query Language) code to meet these data needs.

[0003] Currently, data analysis technicians mainly develop SQL code by hand. However, different data analysts have different coding methods and code development habits, such as the implementation of functions, the order of code steps, and the diversity of comments. This leads to higher development costs and higher costs for code adjustment and maintenance later on, making code management more complicated. Summary of the Invention

[0004] Therefore, the present invention provides a method, apparatus and equipment for standardizing code development, which helps to solve the problem that the code development cost and the subsequent adjustment and maintenance cost of the code are increased due to the different coding methods and habits of technical personnel during the code development process.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] Firstly, the present invention provides a method for standardizing code development, comprising:

[0007] In response to the target report creation command, the target report creation interface is invoked to create a report task to be developed; wherein, the report task to be developed includes at least one task indicator;

[0008] In response to the job creation instruction, based on the task metrics, the job creation interface is invoked to create the job and determine the corresponding execution engine;

[0009] In response to a temporary table creation command, a temporary table creation interface is invoked to create a corresponding temporary table; wherein, the job includes at least one of the temporary tables;

[0010] Based on the task metrics, code segments are developed for the temporary table to obtain the corresponding developed temporary table;

[0011] Based on preset logical rules, multiple developed temporary tables are linked and summarized to obtain the created target report, and the correctness of the target report is verified by the execution engine.

[0012] Furthermore, the step of responding to the job creation instruction, based on the task metrics, calling the job creation interface, creating a job, and determining the corresponding execution engine includes:

[0013] Receive the job creation instruction;

[0014] Based on the aforementioned task indicators, the task type is determined;

[0015] Based on the job type, the job creation interface is called to obtain the corresponding job and determine the corresponding execution engine.

[0016] Furthermore, the step of responding to a temporary table creation command by calling the temporary table creation interface to create the corresponding temporary table includes:

[0017] Receive the temporary table creation instruction;

[0018] Based on the task metrics, the temporary table creation interface is called to create the temporary table in the job, thus obtaining the corresponding temporary table.

[0019] Further, the step of developing code segments for the temporary table based on the task metrics to obtain the corresponding developed temporary table includes:

[0020] Based on the task metrics, name the temporary table and add code segment comments, and write the corresponding fields and logic into the temporary table;

[0021] Based on the task indicators, determine the corresponding data source table;

[0022] By using preset association methods and preset association fields, the temporary table is associated with at least one of the data source tables to obtain the corresponding developed temporary table.

[0023] Furthermore, the type of execution engine includes at least one of the following: Spark execution engine, Hive execution engine, and Presto execution engine.

[0024] Secondly, the present invention provides an apparatus for standardizing code development, comprising:

[0025] The report creation module is used to respond to the target report creation instruction, call the target report creation interface, and create a report task to be developed; wherein, the report task to be developed includes at least one task indicator;

[0026] The job creation module is used to respond to the job creation instruction, call the job creation interface based on the task indicators, create a job, and determine the corresponding execution engine.

[0027] A temporary table creation module is used to respond to a temporary table creation command by calling a temporary table creation interface to create a corresponding temporary table; wherein, the job includes at least one of the temporary tables;

[0028] The temporary table development module is used to develop code segments for the temporary table according to the task indicators, so as to obtain the corresponding developed temporary table;

[0029] The target report generation module is used to associate and summarize multiple developed temporary tables based on preset logical rules to obtain the created target report, and to verify the correctness of the target report through the execution engine.

[0030] Furthermore, the job creation module is specifically used for:

[0031] Receive the job creation instruction;

[0032] Based on the aforementioned task indicators, the task type is determined;

[0033] Based on the job type, the job creation interface is called to obtain the corresponding job and determine the corresponding execution engine.

[0034] Furthermore, the temporary table creation module is specifically used for:

[0035] Receive the temporary table creation instruction;

[0036] Based on the task metrics, the temporary table creation interface is called to create the temporary table in the job, thus obtaining the corresponding temporary table.

[0037] Furthermore, the temporary table development module is specifically used for:

[0038] Based on the task metrics, name the temporary table and add code segment comments, and write the corresponding fields and logic into the temporary table;

[0039] Based on the task indicators, determine the corresponding data source table;

[0040] By using preset association methods and preset association fields, the temporary table is associated with at least one of the data source tables to obtain the corresponding developed temporary table.

[0041] Thirdly, the present invention provides a device for standardizing code development, comprising:

[0042] One or more memories on which executable programs are stored;

[0043] One or more processors are configured to execute the executable program in the memory to implement the steps of any of the methods described above.

[0044] The present invention, by adopting the above technical solution, has at least the following beneficial effects:

[0045] This invention creates a report to be developed by responding to a target report creation command and calling the target report creation interface. The target report includes at least one task indicator. Then, responding to a job creation command, a job creation interface is called based on the task indicator to create a job and determine the corresponding execution engine. Next, responding to a temporary table creation command, a temporary table creation interface is called to create the corresponding temporary table. Each job includes at least one temporary table. Based on the task indicator, code segments are developed for the temporary tables to obtain the corresponding developed temporary tables. Based on preset logical rules, multiple developed temporary tables are linked and summarized to obtain the created target report, and the correctness of the target report is verified by the execution engine. During code development, creating jobs with multiple temporary tables constrains and standardizes code development, making code maintenance and management more convenient. This solves the problem of increased code development costs and subsequent code adjustment and maintenance costs due to different coding methods and habits of technical personnel.

[0046] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating a method for developing standard code according to an embodiment of the present invention;

[0049] Figure 2 This is a block diagram illustrating a device for developing standard code according to an embodiment of the present invention;

[0050] Figure 3 This is a block diagram illustrating a device for developing standard code, as shown in an embodiment of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0052] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for developing standard code according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0053] Step S11: In response to the target report creation instruction, call the target report creation interface to create a report task to be developed; wherein, the report task to be developed includes at least one task indicator.

[0054] Step S12: In response to the job creation instruction, based on the task indicators, call the job creation interface to create a job and determine the corresponding execution engine.

[0055] Step S13: In response to the temporary table creation instruction, call the temporary table creation interface to create the corresponding temporary table; wherein, the job includes at least one of the temporary tables.

[0056] Step S14: Based on the task indicators, develop code segments for the temporary table to obtain the corresponding developed temporary table.

[0057] Step S15: Based on preset logical rules, the multiple developed temporary tables are associated and summarized to obtain the created target report, and the correctness of the target report is verified by the execution engine.

[0058] It should be noted that, in this embodiment, the executing entity is an electronic device such as a terminal device or a server.

[0059] It should be noted that different data analysts have different coding methods and coding habits when developing user reporting tasks. In view of this, this solution is proposed to standardize the code development of different data analysts, making the later maintenance and management of the code more convenient.

[0060] Specifically, based on the user's task metrics, developers trigger a creation command by clicking the "Create Target Report" button to create the report task to be developed. Clicking the "Create Job" button triggers a job creation command, creating a job based on multiple task metrics in the report task to be developed. Simultaneously, an execution engine suitable for this report development is selected for later batch data processing. Clicking the "Create Temporary Table" button triggers a creation command to begin creating the temporary tables needed for development. Simultaneously, code segments are developed within these temporary tables based on the task metrics, resulting in developed temporary tables. Multiple task metrics can generate one or more corresponding developed temporary tables. Finally, based on preset logical rules, these multiple developed temporary tables are logically linked and summarized to obtain the desired target report. During the process of linking and summarizing multiple developed temporary tables, the execution engine continuously performs batch data processing to verify the correctness of the developed code.

[0061] Understandably, when developing a report, which involves multiple business functions and different task metrics, multiple temporary tables need to be created to temporarily store the relevant data in order to avoid overly complex code and the limitations of batch processing resources. Finally, the temporary tables are gradually merged and summarized to form the final report.

[0062] It should be noted that the report task to be developed includes multiple task indicators with different meanings, such as express delivery volume, weight, road transport costs, load weight, return trip revenue, etc. in the logistics industry. In this embodiment, the number and type of task indicators are not specifically limited.

[0063] A job is an operation performed on all report tasks to be developed. It includes temporary tables and code segments, and the scheduling system on the server executes the daily batch command to perform this job.

[0064] Temporary tables are used to temporarily store relevant data involved in the development process; each piece of code can be written into a temporary table.

[0065] The preset logical rules are pre-set logical rules based on keywords, etc., which are used to logically associate the developed temporary tables.

[0066] The aforementioned target report creation interface, job creation interface, and temporary table creation interface are all encapsulated program packages used to execute the instructions for creating the report task, job, and temporary table, respectively. By clicking the relevant buttons, users can call the interfaces to create jobs and temporary tables, and then develop code snippets for the temporary tables to complete the development of the target report, making code development more standardized and maintenance more efficient.

[0067] Understandably, this invention creates a report to be developed by responding to a target report creation instruction and calling the target report creation interface; wherein the report to be developed includes at least one task indicator; then, responding to a job creation instruction, a job creation interface is called based on the task indicator to create a job and determine the corresponding execution engine; then, responding to a temporary table creation instruction, a temporary table creation interface is called to create the corresponding temporary table; wherein the job includes at least one temporary table; code segments are developed for the temporary table according to the task indicator to obtain the corresponding developed temporary table; based on preset logical rules, multiple developed temporary tables are associated and summarized to obtain the created target report, and the correctness of the target report is verified by the execution engine. During code development, creating jobs with multiple temporary tables constrains and standardizes code development, making code maintenance and management more convenient, thereby solving the problem of increased code development costs and subsequent code adjustment and maintenance costs due to different coding methods and habits of technical personnel.

[0068] Furthermore, the step of responding to the job creation instruction, based on the task metrics, calling the job creation interface, creating a job, and determining the corresponding execution engine includes:

[0069] Receive the job creation instruction;

[0070] Based on the aforementioned task indicators, the task type is determined;

[0071] Based on the job type, the job creation interface is called to obtain the corresponding job and determine the corresponding execution engine.

[0072] It's important to note that when creating a job, the job type is first determined based on multiple task metrics in the report to be developed. Then, the job creation interface is called to create the corresponding job. Different task metrics result in different job types, representing different data and scenarios involved in the job, and therefore requiring different execution engines during batch processing. Thus, it's necessary to select the appropriate execution engine based on the job type to run the batch task.

[0073] Furthermore, the step of responding to a temporary table creation command by calling the temporary table creation interface to create the corresponding temporary table includes:

[0074] Receive the temporary table creation instruction;

[0075] Based on the task metrics, the temporary table creation interface is called to create the temporary table in the job, thus obtaining the corresponding temporary table.

[0076] It should be noted that when the developer clicks the "Create Temporary Table" button, the temporary table creation interface is called according to the task metrics, and the required temporary table is created in the job created above.

[0077] Further, the step of developing code segments for the temporary table based on the task metrics to obtain the corresponding developed temporary table includes:

[0078] Based on the task metrics, name the temporary table and add code segment comments, and write the corresponding fields and logic into the temporary table;

[0079] Based on the task indicators, determine the corresponding data source table;

[0080] By using preset association methods and preset association fields, the temporary table is associated with at least one of the data source tables to obtain the corresponding developed temporary table.

[0081] It should be noted that after creating the temporary table, code segments for the temporary table are developed according to the task metrics. First, the created temporary table is named, the code segments within the temporary table are commented, and the corresponding fields and their corresponding logic methods are written. Then, the data source table related to the task metrics is imported. Finally, the data source table and the temporary table are linked through preset association methods and preset association fields to obtain the final developed temporary table.

[0082] It's important to note that the data source table is the upstream table required by the developer's final target report. The created temporary table can store data from the data source table, and then, based on preset association fields, it can be associated with one or more other data source tables to obtain additional data. Alternatively, it can be associated with a separate temporary table generated from other data source tables, without being associated with the original data source table; the choice is entirely up to the developer.

[0083] It should be noted that in this embodiment, the specific content of the preset association method is not specifically limited, nor is the number of temporary tables specifically limited.

[0084] Furthermore, the type of execution engine includes at least one of the following: Spark execution engine, Hive execution engine, and Presto execution engine.

[0085] It's important to note that SQL is the most general-purpose data processing language, capable of performing CRUD operations on databases. Combined with batch processing by an execution engine, it enables high-performance processing of various query types. Currently, the distributed nature of big data has led to a greater diversity of computing engines within the Hadoop (distributed computing and storage framework) ecosystem, which separates storage and computation. In this embodiment, commonly used execution engine types include: Spark, Hive, and Presto. Each execution engine has different application scenarios. Spark requires more memory; Hive is suitable for a wide range of scenarios but is slower; Presto is suitable for smaller data volumes, has higher timeliness, and is faster than Spark.

[0086] It should be noted that this embodiment does not specifically limit the type of execution engine.

[0087] Please see Figure 2 , Figure 2 This is a block diagram illustrating a specification code development apparatus 2 according to an embodiment of the present invention. The specification code development apparatus 2 includes:

[0088] The report task creation module 21 is used to respond to the target report creation instruction, call the target report creation interface, and create a report task to be developed; wherein, the report task to be developed includes at least one task indicator;

[0089] The job creation module 22 is used to respond to the job creation instruction, call the job creation interface based on the task indicators, create a job, and determine the corresponding execution engine;

[0090] The temporary table creation module 23 is used to call the temporary table creation interface in response to the temporary table creation command and create the corresponding temporary table; wherein, the job includes at least one of the temporary tables;

[0091] Temporary table development module 24 is used to develop code segments for the temporary table according to the task indicators, so as to obtain the corresponding developed temporary table;

[0092] The target report generation module 25 is used to associate and summarize multiple developed temporary tables based on preset logical rules to obtain the created target report, and to verify the correctness of the target report through the execution engine.

[0093] Furthermore, the job creation module 22 is specifically used for:

[0094] Receive the job creation instruction;

[0095] Based on the aforementioned task indicators, the task type is determined;

[0096] Based on the job type, the job creation interface is called to obtain the corresponding job and determine the corresponding execution engine.

[0097] Furthermore, the temporary table creation module 23 is specifically used for:

[0098] Receive the temporary table creation instruction;

[0099] Based on the task metrics, the temporary table creation interface is called to create the temporary table in the job, thus obtaining the corresponding temporary table.

[0100] Furthermore, the temporary table development module 24 is specifically used for:

[0101] Based on the task metrics, name the temporary table and add code segment comments, and write the corresponding fields and logic into the temporary table;

[0102] Based on the task indicators, determine the corresponding data source table;

[0103] By using preset association methods and preset association fields, the temporary table is associated with at least one of the data source tables to obtain the corresponding developed temporary table.

[0104] Regarding the apparatus 2 for developing standard code in the above embodiments, the specific methods by which each module performs operations have been described in detail in the embodiments of the related methods described above, and will not be elaborated here.

[0105] Please see Figure 3 , Figure 3 This is a block diagram illustrating a device for developing standard code according to an embodiment of the present invention. The device 3 for developing standard code includes:

[0106] One or more memories 31, on which executable programs are stored;

[0107] One or more processors 32 are configured to execute the executable program in the memory 31 to implement the steps of any of the methods described above.

[0108] Regarding the device 3 for developing standard code in the above embodiments, the specific method by which its processor 32 executes the program in the memory 31 has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0109] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0110] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" or "more" means at least two.

[0111] It should be understood that when an element is referred to as "fixed to" or "set on" another element, it may be directly on the other element or may have an intervening element present at the same time; when an element is referred to as "connected to" another element, it may be directly connected to the other element or may have an intervening element present at the same time. In addition, the term "connected" as used herein may include wireless connections; the word "and / or" as used includes any unit and all combinations of one or more of the associated listed items.

[0112] Any process or method description in the flowchart or otherwise herein can be understood as: representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0113] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0114] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0115] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0116] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0117] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0118] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for standardizing code development, characterized in that, include: In response to the target report creation command, the target report creation interface is invoked to create a report task to be developed; wherein, the report task to be developed includes at least one task indicator; In response to the job creation instruction, based on the task metrics, the job creation interface is invoked to create the job and determine the corresponding execution engine; In response to a temporary table creation command, a temporary table creation interface is invoked to create a corresponding temporary table; wherein, the job includes at least one of the temporary tables; Based on the task indicators, code segment development is performed on the temporary table to obtain the corresponding developed temporary table, including: naming the temporary table and commenting on the code segment according to the task indicators, and writing the corresponding fields and logic methods into the temporary table; determining the corresponding data source table according to the task indicators; and associating the temporary table with at least one of the data source tables through preset association methods and preset association fields to obtain the corresponding developed temporary table. Based on preset logical rules, multiple developed temporary tables are linked and summarized to obtain the created target report, and the correctness of the target report is verified by the execution engine.

2. The method according to claim 1, characterized in that, In response to the job creation instruction, based on the task metrics, the job creation interface is invoked to create a job and determine the corresponding execution engine, including: Receive the job creation instruction; Based on the aforementioned task indicators, the task type is determined; Based on the job type, the job creation interface is called to obtain the corresponding job and determine the corresponding execution engine.

3. The method according to claim 1, characterized in that, The step of responding to a temporary table creation command by calling the temporary table creation interface to create the corresponding temporary table includes: Receive the temporary table creation instruction; Based on the task metrics, the temporary table creation interface is called to create the temporary table in the job, thus obtaining the corresponding temporary table.

4. The method according to claim 1, characterized in that, The type of execution engine includes at least one of the following: Spark execution engine, Hive execution engine, and Presto execution engine.

5. An apparatus for developing standardized code, characterized in that, The method for developing specification code as described in any one of claims 1 to 4, the apparatus for developing specification code, comprises: The report creation module is used to respond to the target report creation instruction, call the target report creation interface, and create a report task to be developed; wherein, the report task to be developed includes at least one task indicator; The job creation module is used to respond to the job creation instruction, call the job creation interface based on the task indicators, create a job, and determine the corresponding execution engine. A temporary table creation module is used to respond to a temporary table creation command by calling a temporary table creation interface to create a corresponding temporary table; wherein, the job includes at least one of the temporary tables; The temporary table development module is used to develop code segments for the temporary table according to the task indicators, so as to obtain the corresponding developed temporary table; The target report generation module is used to associate and summarize multiple developed temporary tables based on preset logical rules to obtain the created target report, and to verify the correctness of the target report through the execution engine.

6. The apparatus according to claim 5, characterized in that, The job creation module is specifically used for: Receive the job creation instruction; Based on the aforementioned task indicators, the task type is determined; Based on the job type, the job creation interface is called to obtain the corresponding job and determine the corresponding execution engine.

7. The apparatus according to claim 5, characterized in that, The temporary table creation module is specifically used for: Receive the temporary table creation instruction; Based on the task metrics, the temporary table creation interface is called to create the temporary table in the job, thus obtaining the corresponding temporary table.

8. The apparatus according to claim 5, characterized in that, The temporary table development module is specifically used for: Based on the task metrics, name the temporary table and add code segment comments, and write the corresponding fields and logic into the temporary table; Based on the task indicators, determine the corresponding data source table; By using preset association methods and preset association fields, the temporary table is associated with at least one of the data source tables to obtain the corresponding developed temporary table.

9. A device for developing standardized code, characterized in that, include: One or more memories on which executable programs are stored; One or more processors are configured to execute the executable program in the memory to implement the steps of the method according to any one of claims 1-4.

Images