Database stored function generation methods, apparatus, electronic devices and program products

Abstract

This disclosure relates to a method, apparatus, electronic device, and program product for generating database stored functions. The method includes: receiving raw code to be processed and obtaining the corresponding syntax structure information of the raw code; obtaining user-preset attribute information and a database stored function generation template; inputting the database stored function generation template, attribute information, and raw code into a database stored function generation module to obtain multiple database stored function sub-results; and merging the multiple database stored function sub-results based on the syntax structure information to obtain a target database stored function. This disclosure enables corresponding database stored function conversion based on code type and attribute information, allowing the above method to automatically generate database stored functions corresponding to various code types, thereby reducing the manual cost in the database stored function conversion process and improving the conversion efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of big data technology, and in particular to database storage function generation methods, apparatus, electronic devices, and program products. Background Technology

[0002] With the rapid development of big data technology, data engineers often need to convert data analysis programs written in Python (the Python programming language) and the pandas library (the pandas data analysis library) into database stored functions suitable for databases when analyzing and processing datasets. In this case, data engineers need to convert Python data analysis scripts into database stored functions.

[0003] In related technologies, Python data analysis scripts are typically converted into database stored functions (Structured Query Language, SQL) through manual coding. Data engineers need to rewrite the corresponding SQL after understanding the code logic of the Python data analysis script. However, manual coding not only requires developers to be proficient in Python and pandas libraries, but also to understand the operation methods and SQL characteristics of specific databases. This consumes a lot of time and energy for data engineers, resulting in low efficiency in the conversion of database stored functions. Summary of the Invention

[0004] In view of the above, exemplary embodiments of this disclosure provide a database storage function generation method, apparatus, electronic device, and program product to solve the problems existing in the related art.

[0005] A first aspect of the exemplary embodiments of this disclosure provides a database stored function generation method, the method comprising:

[0006] Receive the raw code to be processed and obtain the syntax structure information corresponding to the raw code;

[0007] Obtain user-preset attribute information and database stored function generation template; the attribute information is used to define the parameter attribute values ​​of the database stored functions;

[0008] The database storage function generation template, the attribute information, and the original code are input into the database storage function generation module to obtain multiple database storage function sub-results. The database storage function generation module includes a code classification sub-module and a database storage function conversion sub-module. The code classification sub-module splits the original code into multiple code blocks based on the code type. The database storage function conversion sub-module adjusts the database storage function generation template based on the code type and the attribute information, and converts the code blocks corresponding to each code type into corresponding database storage function sub-results based on the adjusted database storage function generation template.

[0009] Based on the grammatical structure information, the multiple database storage function sub-results are combined to obtain the target database storage function.

[0010] A second aspect of the exemplary embodiments of this disclosure provides a database stored function generation apparatus, the apparatus comprising:

[0011] The data acquisition module is used to receive the raw code to be processed and to acquire the syntax structure information corresponding to the raw code;

[0012] The data processing module is used to obtain user-preset attribute information and database storage function generation templates; the attribute information is used to define the parameter attribute values ​​of the database storage functions.

[0013] The data processing module is further configured to input the database storage function generation template, the attribute information, and the original code into the database storage function generation module to obtain multiple database storage function sub-results; the database storage function generation module includes a code classification sub-module and a database storage function conversion sub-module, wherein the code classification sub-module splits the original code into multiple code blocks based on the code type; the database storage function conversion sub-module adjusts the database storage function generation template based on the code type and the attribute information, and converts the code blocks corresponding to each code type into corresponding database storage function sub-results based on the adjusted database storage function generation template;

[0014] The data processing module is also used to combine the multiple database storage function sub-results based on the syntax structure information to obtain the target database storage function.

[0015] A third aspect of the exemplary embodiments of this disclosure provides an electronic device, comprising:

[0016] At least one processor;

[0017] A memory for storing at least one processor-executable instruction; wherein at least one processor executes the instruction to implement the steps of the method described above.

[0018] According to a fourth aspect of this disclosure, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the method described in the exemplary embodiments of this disclosure.

[0019] The above-described at least one technical solution adopted in the exemplary embodiments of this disclosure can achieve the following beneficial effects: receiving the original code to be processed and obtaining the syntax structure information corresponding to the original code. Then, obtaining the user-preset attribute information and database stored function generation template, wherein the attribute information is used to define the parameter attribute values ​​of the database stored function. Furthermore, the database stored function generation template, attribute information, and original code are input into the database stored function generation module to obtain multiple database stored function sub-results. The database stored function generation module includes a code classification sub-module and a database stored function conversion sub-module. The code classification sub-module splits the original code into multiple code blocks based on the code type. The database stored function conversion sub-module adjusts the database stored function generation template based on the code type and attribute information, and converts the code blocks corresponding to each code type into corresponding database stored function sub-results based on the adjusted database stored function generation template. Finally, multiple database stored function sub-results are combined based on the syntax structure information to obtain the target database stored function. The database stored function generation template is adjusted according to the code type and attribute information so that the adjusted database stored function generation template can adapt to different business needs and database types. The code classification submodule is used to split the original code into multiple code blocks, and corresponding database storage functions are converted according to the code type and attribute information. This allows the above method to automatically generate database storage functions corresponding to various code types, thereby reducing the manual cost in the database storage function conversion process and improving the conversion efficiency. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the exemplary embodiments of this disclosure, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a database stored function generation method provided as an example of this disclosure;

[0022] Figure 2 This is a flowchart illustrating an exemplary database storage function generation method provided in this disclosure.

[0023] Figure 3 This is a schematic block diagram of the functional modules of a database stored function generation apparatus provided as an example in this disclosure;

[0024] Figure 4 A schematic block diagram of a chip provided as an example of this disclosure;

[0025] Figure 5 This is a structural block diagram of an exemplary electronic device provided in this disclosure. Detailed Implementation

[0026] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0027] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0028] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0029] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0030] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0031] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0032] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0033] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device. It is understood that the above notification and user authorization process is merely illustrative and does not constitute a limitation on the implementation of this disclosure; other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0034] With the rapid development of big data technology, data engineers often need to convert data analysis programs written in Python and pandas libraries into database stored functions when analyzing and processing datasets. This requires data engineers to transform Python data analysis scripts into database stored functions.

[0035] In related technologies, Python data analysis scripts are typically converted into database stored functions through manual coding. Data engineers need to rewrite the corresponding SQL after understanding the code logic of the Python data analysis script. However, manual coding not only requires developers to be proficient in Python and pandas libraries, but also to understand the operation methods and SQL characteristics of specific databases. This consumes a lot of time and energy for data engineers, resulting in low efficiency in the conversion of database stored functions.

[0036] Therefore, to address the aforementioned problems, this exemplary embodiment provides a method for generating database stored functions. First, an XML (Extensible Markup Language) definition transformation configuration file is created. Then, based on the configuration file, the main framework code of the stored function is created. Next, the framework hierarchy of the Python code is extracted, and type identification is performed on the Python code. Different types of code are converted into SQL statements, and the SQL statements are concatenated according to the framework hierarchy to generate the target stored function.

[0037] For example, Figure 1 This is a flowchart illustrating a database stored function generation method provided as an example of this disclosure. Figure 1 As shown, the database stored function generation method may include the following steps:

[0038] Step S101: Define the transformation configuration file based on XML.

[0039] For example, a transformation configuration file can be defined based on XML. The configuration file defines the relevant information necessary for the storage function generation process, such as database type, database connection information, code generation mode, stored function format, external table format, data type mapping relationship, variable type prediction mode, and SQL structure optimization mode. It also pre-sets configuration templates for two mainstream open-source databases, MySQL and PostgreSQL. If support for other databases is required, the corresponding XML configuration file can be modified as needed according to the template.

[0040] The database type can be configured to specify the type of the target database, such as MySQL (Database Management System) or PostgreSQL (Database Management System). The subsequently generated stored function SQL will be adjusted and generated accordingly based on the syntax and characteristics of the target database.

[0041] Database connection information refers to the connection details of the target database, which may include the host address, port number, username, and password. This information is used to establish a connection with the database during the generation process, enabling the execution of SQL statements and the creation of stored functions. Different database environments can be quickly connected by modifying the connection information through configuration files.

[0042] The stored function format can include the stored function's name, parameters, return value, and other relevant information. By configuring the stored function format, you can ensure that the generated stored functions match the user's requirements and are generated according to the defined format.

[0043] The external table format is used to specify information such as the path, delimiter, and column types of external files. This allows the generation process to create corresponding database tables based on the external table format defined in the configuration file, and then import data from the external files into the database tables using appropriate SQL statements.

[0044] Data type mappings represent the mapping relationship between Python data types and target database data types. By specifying data type mappings in the configuration file, mapping rules between different data types can be defined to ensure correct data type conversion during the generation process.

[0045] Variable type prediction mode: During the generation of stored functions, corresponding code can be generated based on the type inference of variables. The configuration file can define the variable type prediction mode, such as based on type hints or based on function call tracing. If type hints are selected, Python's type hints can be used to directly map the data types of the input parameters and return values ​​of the generated database functions. If function call tracing is selected, the data types of the function's input parameters and return values ​​can be inferred by analyzing the function call statements.

[0046] SQL structure optimization modes can include whether to enable the query optimizer and whether to perform index optimization. Appropriate optimization methods can be selected based on requirements to generate more efficient stored function SQL.

[0047] Based on this, by creating an XML definition transformation configuration file, we can flexibly configure the process of generating database stored functions to adapt to different databases and requirements.

[0048] Step S102: Create the main framework code for the stored function.

[0049] Based on the XML configuration file created in step S101, the main framework code of the stored functions can be created. The database type in the configuration file can then be read. <dbtype>, Stored function name <functionname>and stored function framework template content <content>The system retrieves user-configured attribute values ​​from XML and replaces them with placeholders in the framework template code, outputting the framework code for the stored functions.

[0050] Specifically, the XML configuration file is first parsed. This file contains all the necessary attributes and parameters, such as the database type. <dbtype>, Stored function name <functionname>and stored function framework template content <content>These attribute information are defined in the XML file according to different database types and user requirements.

[0051] Then, the attribute values ​​from the XML file are used to populate a pre-designed stored function code template. This template is a code framework containing placeholders that defines the basic structure of the stored function, including the function declaration, input parameters, return type, and any initial logic framework it may contain.

[0052] In templates, the database type affects the syntax used to declare variables and structure statements in function code, because different database systems have different SQL syntax and features. Store function name. <functionname>It's user-defined, used to identify functions and call them from the database. The framework template content... <content>This defines the internal structure of the function, which may include the declaration of local variables, control flow statements (such as conditional statements and loops), and placeholder SQL queries or commands.

[0053] Step S103: Determine the code type.

[0054] This tool performs code pattern matching on Python data analysis code, automatically categorizing lines or blocks of code into three types: data loading code, data manipulation code, and user-defined function code.

[0055] Specifically, the code is first analyzed to create an Abstract Syntax Tree (AST). An AST is a representation of the syntactic structure of a programming language. It breaks down the code into nodes, each representing a construct in the code, such as an expression, statement, or declaration. This representation makes the structure and control flow of the programming language clearer, facilitating subsequent processing and analysis.

[0056] Next, we define the recognition rules for the three types of code. These rules can be generated based on keywords or regular expressions. Table 1 shows the keywords / regular expressions corresponding to the three code types:

[0057] Table 1. Keywords / Regular Expressions for the Three Code Types

[0058]

[0059] As shown in Table 1, data loading code typically involves reading external data sources; therefore, keywords may include pandas library functions used for loading data, such as "read_csv" and "read_excel". Data manipulation code involves processing and transforming the dataset; keywords may include "groupby", "merge", "join", "sort", and "filter". User-defined function code refers to user-created functions that may perform specific data processing tasks; identifying this type of code may involve matching the "def" keyword with the function name.

[0060] Finally, for each node in the AST, the code type corresponding to the node is identified using keyword / regular expression matching. Codes with the same code type are merged to form a code block of that code type, and the matching result is output.

[0061] In an Abstract Syntax Tree (AST), connected nodes that are identified as being of the same type are merged into a single code block. The purpose of this is to group logically connected and functionally identical code together, making it easier to translate them into corresponding SQL statements later. The merged code block serves as the output matching result, providing a clear logical unit for the next step of the transformation process.

[0062] Based on this, identifying and distinguishing different code types in Python data analysis scripts not only helps to transform complex data processing logic into a form that the database layer can understand and execute, but also facilitates code maintenance and subsequent performance optimization.

[0063] Step S104: Determine the data loading mode.

[0064] It can be set by the user in the XML configuration file. <dataloadtype>The attribute value determines whether the data loading mode is an external table or a regular table. If <dataloadtype>Setting it to "foreign" indicates that the data loading mode is an external table, using the database's external table mechanism to load data, and proceeds to step S105. If <dataloadtype>Setting it to "normal" indicates that the data loading mode is a normal table. The conversion process can be carried out by creating a normal table and using the COPY statement in SQL to import the external file data into the database, and then proceeding to step S106.

[0065] Step S105: Generate data loading SQL based on the external table.

[0066] For data loading code, it can be configured by the user in the XML configuration file. <dataloadtype>The attribute value determines whether the data loading mode is an external table or a regular table. If <dataloadtype>Setting it to "foreign" indicates that the data loading mode is external table, using the database's external table mechanism to load data.

[0067] Taking the PostgreSQL database as an example, this database supports creating external tables based on CSV files, and generating SQL based on data from external tables can specifically include:

[0068] First, a SQL specification template for an external table needs to be created for PostgreSQL and added to the XML configuration file. Unknown values ​​in the template, such as file paths and table names, can be represented using placeholders. Table 2 shows the SQL statement template for the external table schema:

[0069] Table 2 External Table Schema SQL Statement Template

[0070]

[0071]

[0072] As shown in Table 2, corresponding to the external table schema SQL statement template on the right, ${servername} reads the user configuration from the XML file; ${tablename} is the DataFrame name testdata defined in the Python program; ${columns} automatically identifies the fields and their types from the CSV file data; ${filepath} can be filled with the CSV file path " / usr / test.csv"; ${seperator} is the field read, which is , if omitted in the Python statement.

[0073] Then, the data loading code identified in step S103 can be read, and the relevant information in the code (such as DataFrame name, CSV file path, etc.) can be replaced and filled into the placeholders of the template code. The field names can be obtained by reading the header of the CSV file, and the corresponding SQL can be output.

[0074] Next, define the field type. In this embodiment, two methods for defining field types are supported: 1. Ignore the actual type of the field and define the field as "varchar". 2. Automatically identify the field type by reading sample data, and identify the numeric, datetime, and boolean types according to the data type. When creating the table, create the corresponding numeric, datetime, and boolean types, and define the remaining types as varchar.

[0075] For data files in other formats such as Excel or JSON, you can first use a third-party tool to convert them to CSV format before further processing.

[0076] Finally, generate the data loading SQL based on the external table.

[0077] Based on this, by defining SQL templates and field types, data loading code in Python can be converted into SQL, thereby enabling data processing and analysis at the database level.

[0078] Step S106: Generate data and load SQL based on a regular table.

[0079] For data loading code, it can be configured by the user in the XML configuration file. <dataloadtype>The attribute value determines whether the data loading mode is an external table or a regular table. If <dataloadtype>Setting it to "normal" indicates that the data loading mode is a normal table. The conversion process can be carried out by creating a normal table and using the COPY statement in SQL to import data from external files into the database.

[0080] Taking PostgreSQL database as an example, generating and loading SQL based on ordinary tables can specifically include:

[0081] First, based on the SQL statement specifications for creating a regular database table and loading data using the COPY statement, a corresponding SQL template needs to be created and added to the XML configuration file. Unknown values ​​in the template, such as file paths, table names, field names, and field types, are represented using placeholders. Table 3 shows the SQL statement template for a regular table schema:

[0082] Table 3 SQL Statement Template for Normal Table Schema

[0083]

[0084]

[0085] As shown in Table 2, corresponding to the SQL statement template of the normal table mode on the right, ${tablename} is the DataFrame name testdata defined in the Python program; ${columns} automatically identifies the fields and their types from the CSV file data; ${filepath} is the CSV file path " / usr / test.csv"; ${seperator} is the field separator ";" if it is omitted in the Python statement, it is ",".

[0086] Then, the data loading code identified in step S103 can be read, and the relevant information in the code (such as DataFrame name, CSV file path, etc.) can be replaced and filled into the placeholders of the template code. The field names can be obtained by reading the header of the CSV file, and the corresponding SQL can be output.

[0087] Next, define the field type. In this embodiment, two methods for defining field types are supported: 1. Ignore the actual type of the field and define the field as "varchar". 2. Automatically identify the field type by reading sample data, and identify the numeric, datetime, and boolean types according to the data type. When creating the table, create the corresponding numeric, datetime, and boolean types, and define the remaining types as varchar.

[0088] For data files in other formats such as Excel or JSON, you can first use a third-party tool to convert them to CSV format before further processing.

[0089] Finally, generate the data loading SQL based on the external table.

[0090] Based on this, by converting data loading operations in Python code into SQL statements, it is possible to create ordinary tables in the database and load data, thereby enabling data processing and analysis at the database level.

[0091] Step S107: Define the data operation operator mapping rules.

[0092] For data manipulation code, you can first define data manipulation operator mapping rules, and then, based on these rules, transform the Python data analysis script into database stored functions. Defining data manipulation operator mapping rules can specifically include:

[0093] First, we need to categorize and analyze the data manipulation operators provided by Python pandas. The types of data manipulation operators can include: data selection and indexing, data filtering, data sorting, data aggregation, data merging, data transformation, data cleaning, column operations, or statistical functions.

[0094] Next, data manipulation operator mapping rules are defined. For various types of operators supported by Python pandas, their corresponding encoding patterns in database SQL are defined as the basis for automatic generation of data manipulation SQL code. Table 4 shows some of the data manipulation operator mapping rules:

[0095] Table 4 Data Operation Operator Mapping Rules

[0096]

[0097] Predefined data manipulation operator mapping rules can be stored in an accessible location, typically an XML configuration file or a database. These rules can be templated strings containing placeholders to dynamically insert specific column names, table names, conditions, etc.

[0098] Based on this, we define data manipulation operator mapping rules. For various types of operator operations supported by Python pandas, we define their corresponding encoding patterns in database SQL, which serve as the basis for automatic generation of data manipulation SQL code.

[0099] Step S108: Generate data operation SQL.

[0100] For data manipulation code, the mapping rules defined in step S107 can be used to generate corresponding data manipulation SQL. In this embodiment, two data manipulation SQL generation methods are supported: 1. One-to-one generation with the original Python program: For each Python data manipulation statement's return result, a corresponding mapping SQL is used to create a table. 2. Generating nested query statements: For multiple Python statements operating on the same dataframe, nested subqueries are used to merge them into a single SQL statement for output. The choice of which data manipulation SQL generation method to use depends on the user's configuration in the XML configuration file. <sqltype>.

[0101] Extract the data operation code identified in step S103, and generate equivalent data operation SQL using the data operation operator mapping rules defined in step S107 above.

[0102] Based on this, according to the data operation operator mapping rules, the data analysis script written in Python is converted into the corresponding data operation SQL.

[0103] Step S109: Generate user-defined function mapping code blocks based on templates.

[0104] For User Defined Function (UDF) code, the plpython3u extension plugin can be used to generate the corresponding custom function code in the database. The plpython3u extension plugin creates SQL statement templates that contain the user-defined function logic, and automatically populates the data types of parameters and return values ​​during the code generation process.

[0105] Taking PostgreSQL database as an example, generating user-defined function mapping code blocks based on templates can specifically include:

[0106] First, create a database function definition statement template based on plpython3u and add it to the XML configuration file. Some unknowns that need to be loaded at runtime are represented by placeholders.

[0107] Then, the user-defined function (UDF) code identified in step S103 is read, and relevant information in the code (such as function name, parameter name, parameter type, return type, function body, etc.) is replaced and filled into the placeholders in the template code.

[0108] Furthermore, based on the settings configured by the user in the XML configuration file... <udftype>The attribute value determines the data type of the SQL input parameters and return values.

[0109] Specifically, when <udftype>When "type hint" is used, Python type hints are used to infer the parameter and return types. <udftype>When "autoinfer" is selected, the data type is inferred by analyzing the UDF call statements.

[0110] Step S110: Determine the data type acquisition method.

[0111] For user-defined function (UDF) code, it is possible to obtain the XML configuration file. <udftype>The attribute value, if <udftype>If the attribute value is typehint, then proceed to step S111; if <udftype>If the attribute value is autoinfer, then step S112 is executed.

[0112] Step S111: Fill in the input parameter or return value types of the user-defined function based on type hints.

[0113] For user-defined function (UDF) code, if the XML configuration file contains... <udftype>If the user sets it to `typehint`, the data types of the database function's input parameters and return values ​​will be directly mapped based on Python type hints. Mapping rules can be predefined to specify the mapping relationship between Python data types and database data types. Then, the information extracted from the type hints is used to replace the placeholders for parameters and return values ​​in the function definition template, resulting in a function definition that includes database-compatible data type declarations. Finally, the final database function definition code is generated based on the replaced template. It is crucial to ensure that every parameter and return value in the code uses the correct database data type extracted from the type hints. Table 5 shows an example of user-defined function code generation with type hints:

[0114] Table 5. Examples of user-defined function code generation with type hints

[0115]

[0116] This method of using type hints to populate the input and return types of user-defined functions requires Python coders to declare parameters according to the type hint specifications when writing code. For existing code, type hints need to be manually added to the Python user-defined function code before using this method.

[0117] Based on this, by filling in the input parameters or return value types of user-defined functions based on type hints, it is possible to ensure accurate type conversion of field types between different programming environments, enabling user-defined functions to be efficiently and accurately converted into database stored functions.

[0118] Step S112: Automatically infer the input parameter or return value type of the user-defined function based on the call statement.

[0119] For UDF code, if the XML configuration file contains <udftype>If the user sets it to autoinfer, the data types of the function's input parameters and return value will be automatically inferred based on the UDF call statement. Taking the UDF in Table 5 as an example, when the UDF does not contain type hints, the source is automatically traced back to the statement that calls the UDF based on the function name. When the input parameters 'a' and 'b' of the UDF are identified as corresponding to the "a" and "b" fields of the data set (as shown in the second column of Table 6 below), the data types of the input parameters of the UDF are automatically determined based on the data types of these two fields in the data table (for example, in the example in Table 6 below, the data types of the "a" and "b" fields of the data set are both int). The return value type is determined based on the return statement in the UDF (for example, in the example in Table 6 below, the variable type of the return is int, so the return value type is inferred to be int). If the return statement contains unknown variable types that cannot be directly determined, a bottom-up approach can be used to trace back within the UDF to complete the data type inference. Table 6 shows an example of automatically inferring user-defined functions by tracing the call statement:

[0120] Table 6 shows examples of automatically inferring user-defined functions based on call statement tracing.

[0121]

[0122] Table 6 shows the transformation results of the statement calling UDF. The generated result of the UDF itself is the same as the third column in Table 6 above.

[0123] Based on this, in the absence of explicit type hints, the input parameter or return value type of user-defined functions can be automatically inferred from the call statement. This can infer the appropriate data type, ensuring accurate type conversion of field types between different programming environments, and enabling user-defined functions to be efficiently and accurately converted into database stored functions.

[0124] Step S113: Assemble all SQL statements in sequence.

[0125] The generated data loading, data manipulation, and user-defined function code blocks are assembled and spliced ​​into a complete database storage function SQL code according to the abstract syntax tree structure identified in step S103.

[0126] Specifically, the generated code block and the abstract syntax tree structure identified in step S103 are first obtained.

[0127] Then, the execution order of each code block is determined based on the abstract syntax tree structure. In determining the execution order, it is necessary to ensure that data loading operations are performed before data operations and function calls, and also to ensure that all dependent functions or data structures have been defined and initialized before being referenced.

[0128] Next, begin assembling code blocks in logical order. Appropriate control flow statements (such as IF conditional statements and LOOP loop statements) can be used to organize the code blocks. Necessary declaration statements can also be inserted to allocate space and data types for variables, temporary tables, etc. Error handling and transaction management code can also be added where necessary.

[0129] Finally, generate the complete database stored functions.

[0130] Step S114: Output database storage functions based on the configuration file.

[0131] For example, it is necessary to generate schema attributes based on the XML configuration file. <genmode>If the configuration value is `sqlfile`, the generated database stored function will be saved as a local SQL file. If the configuration value is `dbfunction`, the target database connection information configured by the user will be read, and the SQL statement will be executed directly in the target database to create the corresponding stored function.

[0132] Specifically, targeting <genmode>The value is sqlfile, and the output database stored functions can include:

[0133] First, create a text file containing the generated SQL statements for database stored functions.

[0134] Then determine the local path and filename for saving the SQL file. The local path and filename can be specified in advance in the configuration file or generated dynamically by the program.

[0135] Next, write the SQL code into a file, ensuring the file format is correct, such as character encoding, so that it can be opened in any SQL editor or database management tool.

[0136] Finally, after saving the file, notify the user that the SQL file has been successfully created and provide the file location information.

[0137] Specifically, targeting <genmode>The value is dbfunction, and the output database stored functions can include:

[0138] First, read the database connection information from the XML configuration file, which may include the database type, server address, port, username, password, database name, etc.

[0139] Then, this information is used to establish a connection to the target database. After the connection is established, an SQL statement containing the stored function definition is executed to create the function in the database.

[0140] Next, check the result of the SQL statement execution to confirm whether the stored function was successfully created.

[0141] Finally, if the execution is successful, a message indicating successful creation should be sent to the user. If the execution fails, error information should be captured, and error details should be provided to the user so that they can troubleshoot and correct the problem.

[0142] Based on this, users can flexibly control the generation and deployment of stored functions according to the generation mode attribute value in the XML configuration file, thereby better integrating database stored functions into their development and deployment process.

[0143] One or more technical solutions provided in the exemplary embodiments of this disclosure automate the generation of Python data analysis scripts into database stored functions, significantly reducing the workload of manual writing and adjustment. Secondly, configurability is provided through XML configuration files and rule definitions, enabling adaptation to different databases and user needs. Furthermore, the use of syntax analysis and pattern matching technologies accurately identifies different types of code blocks, improving the accuracy and reliability of the generated code. Through data operation mapping rules, Python data operation code is transformed into corresponding SQL statements, reducing the complexity and errors of manually writing SQL. Finally, code blocks can be assembled and concatenated sequentially to generate complete database stored function SQL, and can be saved as a local file or directly executed in the target database according to the configuration, providing flexible code assembly and execution methods.

[0144] Therefore, the database stored function generation method provided in the exemplary embodiments of this disclosure can automatically generate database stored functions corresponding to various code types, thereby reducing the human cost in the database stored function conversion process and improving the conversion efficiency of database stored functions.

[0145] Based on the above embodiments, this disclosure also provides a method for generating database stored functions. Figure 2 This is a flowchart illustrating an exemplary database stored function generation method provided in this disclosure, such as... Figure 2 As shown, this method for generating database stored functions may include the following steps:

[0146] Step S210: Receive the original code to be processed and obtain the syntax structure information corresponding to the original code.

[0147] In the embodiments, the original code can be various types of code for analyzing and processing datasets, including but not limited to Python, R, Java, or C / C++.

[0148] For example, obtaining the syntax structure information corresponding to the original code may include the following steps:

[0149] First, perform a syntax analysis on the original code, breaking it down into lexical units such as identifiers, operators, and constants. This can be achieved using a lexical analyzer or regular expressions.

[0150] Then, a syntax tree is constructed based on the lexical units, representing the syntactic structure of the original code. The parser can parse the lexical units according to the grammar rules (usually defined in grammatical form) and generate a syntax tree. Commonly used parsing algorithms include recursive descent and logistic regression (LR) parsing.

[0151] Finally, the syntax tree is semantically checked and analyzed to ensure the legality and semantic correctness of the code. Semantic analysis can include type checking, scope analysis, and semantic rule verification.

[0152] Step S220: Obtain the user-preset attribute information and database stored function generation template; the attribute information is used to define the parameter attribute values ​​of the database stored function.

[0153] In this embodiment, the attribute information is equivalent to the XML configuration file in the above embodiments. The XML configuration file can be used to define and configure the relevant information and parameters required in the process of automatically generating database stored functions.

[0154] Specifically, the attribute information may include:

[0155] The database type (such as MySQL, PostgreSQL, etc.) and the corresponding connection information (such as hostname, port number, username, password, etc.) are required so that the program can establish a connection with the target database.

[0156] Code generation mode: The code generation mode can be specified via an XML configuration file, including stored function format, external table format, data type mapping, variable type prediction mode, and SQL structure optimization mode. This configuration information affects the structure and syntax of the generated stored function code.

[0157] Data loading method: XML configuration files can set attributes for the data loading code, such as the data loading type and external file path. Based on the settings in the configuration file, you can choose to load data into the database using regular tables and COPY statements, or use other methods for data loading.

[0158] User-defined functions: XML configuration files can define the attributes of user-defined functions, such as function type (typehint or autoinfer), function name, input parameter types, return value type, etc. This configuration information can be processed and mapped appropriately based on the user-defined function code when the stored function is generated.

[0159] By pre-setting attribute information, relevant configuration information can be separated from the code, making configuration modification and adjustment more convenient and flexible. Meanwhile, XML provides a universal data format that is easy to parse and process, enabling programs to generate and operate accordingly based on the information in the configuration file.

[0160] For example, by parsing user-preset attribute information, attribute information and database stored function generation templates can be obtained. Attribute information may include database type, connection information, code generation mode, data loading method, user-defined functions, and other information.

[0161] Furthermore, to parse the XML content, you can use an appropriate XML parsing library, such as ElementTree or BeautifulSoup, to parse the XML configuration file and extract the required attribute information.

[0162] The attribute information extracted from the XML configuration file is then stored in an appropriate data structure, such as a dictionary or object, for use when generating database stored functions later.

[0163] Finally, based on the user-defined attribute information, the appropriate database stored function generation template is selected. The database stored function generation template can be a user-defined template stored in the template library; different templates can adapt to different code generation needs and configuration options.

[0164] Step S230: Input the database stored function generation template, attribute information, and original code into the database stored function generation module to obtain multiple database stored function sub-results. The database stored function generation module includes a code classification sub-module and a database stored function conversion sub-module. The code classification sub-module splits the original code into multiple code blocks based on the code type. The database stored function conversion sub-module adjusts the database stored function generation template based on the code type and attribute information, and converts the code blocks corresponding to each code type into corresponding database stored function sub-results based on the adjusted database stored function generation template.

[0165] In this embodiment, the database stored function generation module may include a code classification submodule and a database stored function conversion submodule.

[0166] The code categorization submodule can split the original code into multiple code blocks based on the type of each line of code. Categorization can be based on different needs and rules, such as functions, classes, or statement blocks. Dividing the original code into multiple parts facilitates subsequent database storage function conversion.

[0167] Then, the database stored function conversion submodule is used to adjust the database stored function generation template based on code type and attribute information. An appropriate database stored function generation template is selected based on the code type and attribute information, and then adjusted and customized as necessary.

[0168] Finally, the adjusted database stored function generation template is applied to each code block, converting each code block into a corresponding database stored function sub-result. This process may involve template replacement, variable mapping, and code structure adjustments to ensure that the generated database stored function sub-results meet the expected format and syntax requirements.

[0169] Step S240: Combine multiple database storage function sub-results based on syntax structure information to obtain the target database storage function.

[0170] In this embodiment, the syntactic structure information of the previously acquired original code can be used to parse and analyze the sub-results of multiple database storage functions. The parsing process may involve operations such as syntax tree traversal, node matching, and structural analysis.

[0171] Then, based on the grammatical structure information, the results of multiple database storage functions are combined in the correct order and hierarchy by merging function calls, variable passing, and statement order of different code blocks.

[0172] Finally, the merged database storage function sub-results can be combined into the final target database storage function through operations such as code block concatenation, indentation, and formatting.

[0173] Based on this, the database stored function generation template is adjusted according to code type and attribute information, making it adaptable to different business needs and database types. The original code is split into multiple code blocks using a code classification submodule, and corresponding database stored function conversions are performed based on code type and attribute information. This allows the above method to automatically generate database stored functions corresponding to various code types, thereby reducing the manual cost of database stored function conversion and improving conversion efficiency.

[0174] Based on the above embodiments, in another embodiment provided in this disclosure, step S230 may specifically include:

[0175] Retrieve preset code classification rules; the code classification rules include key parameters corresponding to multiple code types, and / or regular expressions;

[0176] Iterate through the lines of code in the original code;

[0177] Determine the code type corresponding to the line of code when the line of code includes key parameters and / or regular expressions;

[0178] Merge lines of code of the same type into code blocks to obtain code blocks corresponding to each code type.

[0179] In this embodiment, the preset code classification rules can classify the original code into three types based on keywords and / or regular expressions: data loading code, data manipulation code, and user-defined function code. The code types and their corresponding keywords and / or regular expressions can be found in Table 1 above, along with the corresponding content in Table 1.

[0180] Syntactic analysis is performed on the original code to create an Abstract Syntax Tree (AST). An AST is a representation of the syntactic structure of a programming language. It breaks down the code into nodes, each node representing a construct in the code, such as an expression, statement, or declaration.

[0181] Traverse each node in the AST and identify the code type corresponding to the node using keyword / regular expression matching. Merge codes with the same code type to form a code block of that code type and output the matching result.

[0182] In an Abstract Syntax Tree (AST), connected nodes that are identified as being of the same type are merged into a single code block. The purpose of this is to group logically connected and functionally identical code together, making it easier to translate them into corresponding SQL statements later. The merged code block serves as the output matching result, providing a clear logical unit for the next step of the transformation process.

[0183] Based on this, identifying and distinguishing different code types in Python data analysis scripts not only helps to transform complex data processing logic into a form that the database layer can understand and execute, but also facilitates code maintenance and subsequent performance optimization.

[0184] Based on the above embodiments, in another embodiment provided in this disclosure, the code type includes data loading code, and the above method may specifically include:

[0185] When the code type is data loading code, the data loading mode is determined based on the attribute information;

[0186] Adjust the database stored function generation template based on the data loading mode to obtain the adjusted database stored function generation template;

[0187] Obtain the first identifier information in the code block corresponding to the data loading code, and fill the first identifier information into the adjusted database storage function generation template to obtain the database storage function sub-result corresponding to the data loading code.

[0188] In this embodiment, the data loading code can be configured by the user in the XML configuration file. <dataloadtype>The attribute value determines whether the data loading mode is an external table or a regular table. If <dataloadtype>Setting it to "foreign" indicates that the data loading mode is external table, using the database's external table mechanism to load data. If <dataloadtype>Setting it to "normal" indicates that the data loading mode is a normal table. The conversion process can be carried out by creating a normal table and using the COPY statement in SQL to import data from external files into the database.

[0189] The database stored function generation template can be adjusted accordingly based on the determined data loading mode. For example, if the data loading mode is an external table, the adjusted template needs to include the definition and loading statements of the external table; if the data loading mode is a regular table, the adjusted template needs to include the table creation and data import statements.

[0190] In this embodiment, the first identification information can be a variable in a code block, a file path, or other identifier used to uniquely identify the data loading operation. The first identification information is then filled into the corresponding position in the adjusted database stored function generation template. Based on the template's structure and syntax, the specific position where the first identification information will be filled is determined to ensure that the generated database stored function sub-result has the correct identification information.

[0191] For example, the database stored function generation template for the data loading mode of an external table can be found in Table 2 above, and the database stored function generation template for the data loading mode of a regular table can be found in Table 3 above.

[0192] Based on this, the database stored function generation template is adjusted according to the data loading mode, and the first identifier information in the code block corresponding to the data loading code is filled into the adjusted template to obtain the database stored function sub-result corresponding to the data loading code. Thus, based on the specific data loading requirements and configuration information, database stored function sub-results suitable for external tables or ordinary tables are generated.

[0193] Based on the above embodiments, in another embodiment provided in this disclosure, the above method for data loading code may specifically include:

[0194] Retrieve pre-defined field type rules;

[0195] The field type definition rules determine the field type corresponding to each field in the database stored function; the field type definition rules include defining the field type of each field as the first field type, and / or setting the corresponding field type based on different field categories;

[0196] Fill the field types into the adjusted database stored function generation template.

[0197] In this embodiment, the field type definition rules may include defining the field type of each field as a first field type, which may be "varchar". The field type definition rules may also include setting corresponding field types based on different field categories. For example, the field type may be automatically identified by reading sample data, and numeric, datetime, and boolean types may be identified based on the data type. These types may be created as numeric, datetime, and boolean respectively during table creation, while other types may be defined as varchar. For data files in other formats such as Excel or JSON, they can be converted to CSV format using third-party tools before further processing.

[0198] In this embodiment, the specific steps of filling field types into the adjusted database stored function generation template may include:

[0199] First, generate the template structure based on the adjusted database stored functions, and determine the specific locations where the field types need to be populated. This could be variables, placeholders, or other identifiers in the template.

[0200] Populate the determined field types into the corresponding positions in the adjusted database stored function generation template. Ensure that the field types are correctly populated into the corresponding field definitions to ensure that the generated database stored functions have the correct field types.

[0201] Based on this, pre-defined field type rules are obtained, and the corresponding field types for each field in the database stored function are determined according to these rules. Then, these field types are filled into the corresponding positions in the adjusted database stored function generation template to generate database stored function sub-results that conform to the field type definition rules.

[0202] Based on the above embodiments, in another embodiment provided in this disclosure, the code type includes data manipulation code, and the above method may specifically include:

[0203] When the code type is data manipulation code, obtain the data manipulation operator mapping rules; the data manipulation operator mapping rules include the mapping relationship between the original code and the database stored functions;

[0204] Based on the data operation operator mapping rules and attribute information, the database storage function generation template is adjusted to obtain the adjusted database storage function generation template.

[0205] Obtain the second identifier information from the code block corresponding to the data operation code, and fill the second identifier information into the adjusted database storage function generation template to obtain the database storage function sub-result corresponding to the data operation code.

[0206] In this embodiment, the data operation operator mapping rule is used to represent the mapping relationship between the original code and the database stored functions. The data operation operator mapping rule can be stored in configuration files, database tables or other forms.

[0207] For data manipulation code, you can first define data manipulation operator mapping rules, and then, based on these rules, transform the Python data analysis script into database stored functions. Defining data manipulation operator mapping rules can specifically include:

[0208] First, we need to categorize and analyze the data manipulation operators provided by Python pandas. These operators can include: data selection and indexing, data filtering, data sorting, data aggregation, data merging, data transformation, data cleaning, column operations, and statistical functions.

[0209] Next, we define the data manipulation operator mapping rules. For various types of operators supported by Python pandas, we define their corresponding encoding patterns in database SQL, which serve as the basis for the automatic generation of data manipulation SQL code. The data manipulation operator mapping rules can be found in Table 4 above.

[0210] Based on the attribute information, the database stored function generation template can be adjusted accordingly to generate suitable database stored functions.

[0211] The second identification information can also be extracted from the code block corresponding to the data operation code. The second identification information can be a variable, table name, conditional statement, or other identifier in the code block, used to uniquely identify the data operation.

[0212] The second identifier information is then filled into the corresponding position in the adjusted database stored function generation template. Based on the template's structure and syntax, the specific position for filling the second identifier information is determined to ensure that the generated database stored function sub-results have the correct identifier information.

[0213] Based on this, and using the data operation operator mapping rules and attribute information, the database stored function generation template is adjusted. The second identifier information from the code block corresponding to the data operation code is then filled into the adjusted template, thereby obtaining the database stored function sub-result corresponding to the data operation code. Thus, based on specific data operation requirements and mapping rules, a stored function sub-result suitable for the database is generated.

[0214] Based on the above embodiments, in another embodiment provided in this disclosure, the above method specifically may include, for data manipulation code:

[0215] Retrieve the database stored function generation pattern from the attribute information; the database stored function generation pattern includes a single statement generation pattern or a nested query statement generation pattern.

[0216] Based on the database stored function generation pattern and the adjusted database stored function generation template, the database stored function sub-results corresponding to the data operation code are obtained.

[0217] In this embodiment, an XML configuration file or database table containing attribute information is obtained. The attribute information includes settings related to the database stored function generation mode. The attribute information is parsed according to the storage format of the configuration file or database table. After parsing, the attribute values ​​of the database stored function generation mode are obtained. The database stored function generation mode includes a single-statement generation mode or a nested query statement generation mode. The single-statement generation mode means that each data operation code corresponds to an independently generated database stored function statement, while the nested query statement generation mode means that multiple data operation codes operating on the same data source will be merged into a single nested query statement.

[0218] For example, it can be based on the user profile. <sqltype>The settings determine whether to use a single-statement generation mode or a nested query statement generation mode. Then, based on the selected generation mode, the database stored function generation template is adjusted, and based on the template and specific attribute information, database stored function sub-results corresponding to the data operation codes are generated. This allows for the generation of stored function sub-results suitable for different database requirements.

[0219] Based on the above embodiments, in another embodiment provided in this disclosure, the code type includes user-defined function code, and the above method may specifically include:

[0220] When the code type is user-defined function code, obtain the third identifier information in the code block corresponding to the user-defined function code;

[0221] The third identifier information is filled into the database stored function generation template to obtain the adjusted database stored function generation template.

[0222] Determine the field type corresponding to each field in the user-defined function code based on the attribute information;

[0223] Generate templates based on field types and adjusted database stored functions to obtain database stored function sub-results corresponding to user-defined function code.

[0224] In this embodiment, the third identification information may include information such as function name, parameter name, parameter type, return type, and function body. The database storage function generation template can be created using plpython3u.

[0225] Specifically, first create a database function definition statement template based on plpython3u and add it to the XML configuration file. Some unknown quantities that need to be loaded at runtime are represented by placeholders.

[0226] Then, obtain the user-defined function code and replace the third identifier information in the code with the placeholders in the template code.

[0227] Furthermore, based on the settings configured by the user in the XML configuration file... <udftype>The attribute value determines the data type of the SQL input parameters and return values.

[0228] Specifically, when <udftype>When "type hint" is used, Python type hints are used to infer the parameter and return types. <udftype>When "autoinfer" is selected, the data type is inferred by analyzing the UDF call statements.

[0229] If in the XML configuration file <udftype>If the user sets it to `typehint`, the data types of the database function's input parameters and return values ​​will be directly mapped and generated based on Python type hints. Mapping rules can be predefined to specify the mapping relationship between Python data types and database data types. Then, the information extracted from the type hints is used to replace the placeholders for parameters and return values ​​in the function definition template, resulting in a function definition that includes database-compatible data type declarations. Finally, the final database function definition code is generated based on the replaced template. It is crucial to ensure that every parameter and return value in the code uses the correct database data type extracted from the type hints. See Table 5 above for an example of user-defined function code generation with type hints.

[0230] If in the XML configuration file <udftype>If the user sets it to autoinfer, the data types of the function's input parameters and return value will be automatically inferred based on the UDF call statement. Using the UDFs in Table 5 as an example, if the UDF does not contain type hints, the call will be automatically traced back to the statement that calls the UDF based on the function name. For automatic inference of user-defined functions based on call statement tracing, please refer to Table 6 above.

[0231] Based on this, by defining the input parameter or return value types of user-defined function code in different ways, it is possible to ensure accurate type conversion of field types between different programming environments, enabling user-defined functions to be efficiently and accurately converted into database storage functions.

[0232] One or more technical solutions provided in the exemplary embodiments of this disclosure automate the generation of Python data analysis scripts into database stored functions, significantly reducing the workload of manual writing and adjustment. Secondly, configurability is provided through XML configuration files and rule definitions, enabling adaptation to different databases and user needs. Furthermore, the use of syntax analysis and pattern matching technologies accurately identifies different types of code blocks, improving the accuracy and reliability of the generated code. Through data operation mapping rules, Python data operation code is transformed into corresponding SQL statements, reducing the complexity and errors of manually writing SQL. Finally, code blocks can be assembled and concatenated sequentially to generate complete database stored function SQL, and can be saved as a local file or directly executed in the target database according to the configuration, providing flexible code assembly and execution methods.

[0233] Therefore, the database stored function generation method provided in the exemplary embodiments of this disclosure can automatically generate database stored functions corresponding to various code types, thereby reducing the human cost in the database stored function conversion process and improving the conversion efficiency of database stored functions.

[0234] The foregoing primarily describes the solutions provided by exemplary embodiments of this disclosure. It is understood that, in order to achieve the above functions, the electronic device includes corresponding hardware structures and / or software modules for performing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0235] The exemplary embodiments of this disclosure can divide the electronic device into functional units according to the above method examples. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into a single processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in the exemplary embodiments of this disclosure is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0236] In the case of dividing each functional module according to its corresponding function, an exemplary embodiment of this disclosure provides a database storage function generation apparatus, which can be a server or a chip applied to a server. Figure 3 This is a schematic block diagram of the functional modules of a database stored function generation apparatus provided as an example of this disclosure. Figure 3 As shown, the database stored function generation device 300 includes:

[0237] The data acquisition module 310 is used to receive the raw code to be processed and acquire the syntax structure information corresponding to the raw code;

[0238] The data processing module 320 is used to obtain user-preset attribute information and database storage function generation templates; the attribute information is used to define the parameter attribute values ​​of the database storage functions.

[0239] The data processing module 320 is further configured to input the database storage function generation template, the attribute information, and the original code into the database storage function generation module to obtain multiple database storage function sub-results; the database storage function generation module includes a code classification sub-module and a database storage function conversion sub-module, wherein the code classification sub-module splits the original code into multiple code blocks based on the code type; the database storage function conversion sub-module adjusts the database storage function generation template based on the code type and the attribute information, and converts the code blocks corresponding to each code type into corresponding database storage function sub-results based on the adjusted database storage function generation template;

[0240] The data processing module 320 is also used to combine the multiple database storage function sub-results based on the syntax structure information to obtain the target database storage function.

[0241] In another embodiment provided in this disclosure, the data processing module 320 is further configured to obtain preset code classification rules; the code classification rules include key parameters and / or regular expressions corresponding to multiple code types respectively; traverse the code lines in the original code; when a code line includes the key parameters and / or the regular expressions, determine the code type corresponding to the code line; merge code lines with the same code type into code blocks to obtain code blocks corresponding to each code type respectively.

[0242] In another embodiment provided in this disclosure, the code type includes data loading code. The data processing module 320 is further configured to: determine a data loading mode based on the attribute information when the code type is data loading code; adjust the database storage function generation template based on the data loading mode to obtain an adjusted database storage function generation template; obtain the first identification information in the code block corresponding to the data loading code, and fill the first identification information into the adjusted database storage function generation template to obtain the database storage function sub-result corresponding to the data loading code.

[0243] In another embodiment provided in this disclosure, the data processing module 320 is further configured to obtain pre-set field type definition rules; determine the field type corresponding to each field in the database stored function based on the field type definition rules; the field type definition rules include defining the field type of each field as a first field type, and / or setting the corresponding field type based on different field classifications; and filling the field type into the adjusted database stored function generation template.

[0244] In another embodiment provided in this disclosure, the code type includes data operation code. The data processing module 320 is further configured to, when the code type is data operation code, obtain data operation operator mapping rules; the data operation operator mapping rules include the mapping relationship between the original code and the database storage function; based on the data operation operator mapping rules and the attribute information, adjust the database storage function generation template to obtain an adjusted database storage function generation template; obtain the second identification information in the code block corresponding to the data operation code, and fill the second identification information into the adjusted database storage function generation template to obtain the database storage function sub-result corresponding to the data operation code.

[0245] In another embodiment provided in this disclosure, the data processing module 320 is further configured to obtain the database storage function generation pattern in the attribute information; the database storage function generation pattern includes a single statement generation pattern or a nested query statement generation pattern; and based on the database storage function generation pattern and the adjusted database storage function generation template, obtain the database storage function sub-result corresponding to the data operation code.

[0246] In another embodiment provided in this disclosure, the code type includes user-defined function code. The data processing module 320 is further configured to: obtain third identification information in the code block corresponding to the user-defined function code when the code type is user-defined function code; fill the third identification information into the database storage function generation template to obtain an adjusted database storage function generation template; determine the field type corresponding to each field in the user-defined function code based on the attribute information; and obtain the database storage function sub-result corresponding to the user-defined function code based on the field type and the adjusted database storage function generation template.

[0247] Figure 4 This is a schematic block diagram of a chip provided as an example of this disclosure. Figure 4 As shown, the chip 400 includes one or more (including two) processors 401 and a communication interface 402. The communication interface 402 can support the server in performing the data transmission and reception steps in the above method, and the processor 401 can support the server in performing the data processing steps in the above method.

[0248] Optional, such as Figure 4 As shown, the chip 400 also includes a memory 403, which may include read-only memory and random access memory, and provides operation instructions and data to the processor. A portion of the memory may also include non-volatile random access memory (NVRAM).

[0249] In some implementations, such as Figure 4 As shown, processor 401 executes corresponding operations by calling operation instructions stored in memory (which may be stored in the operating system). Processor 401 controls the processing operations of any terminal device; processor can also be called a central processing unit (CPU). Memory 403 may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory 403 may also include NVRAM. For example, in applications, memory, communication interfaces, and other components are coupled together via a bus system, which may include, in addition to a data bus, a power bus, a control bus, and a status signal bus, etc. However, for clarity, in... Figure 4 The general designated all buses as Bus System 404.

[0250] The methods disclosed in the embodiments of this disclosure can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above methods can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0251] Exemplary embodiments of this disclosure also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the electronic device to perform a method according to an embodiment of this disclosure.

[0252] Exemplary embodiments of this disclosure also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to embodiments of this disclosure.

[0253] Exemplary embodiments of this disclosure also provide a computer program product, including a computer program, wherein, when executed by a processor of a computer, the computer program is used to cause the computer to perform a method according to an embodiment of this disclosure.

[0254] Figure 5 The present disclosure provides an exemplary structural block diagram of an electronic device. The structural block diagram of an electronic device 500 that can serve as a server or client of the present disclosure is now described, and it is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0255] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 may also store various programs and data required for the operation of the electronic device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0256] Multiple components in electronic device 500 are connected to I / O interface 505, including: input unit 506, output unit 507, storage unit 508, and communication unit 509. Input unit 506 can be any type of device capable of inputting information to electronic device 500. Input unit 506 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 507 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 508 may include, but is not limited to, disk and optical disk. Communication unit 509 allows electronic device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0257] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above. The various methods described above can all be implemented as computer software programs, which are tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 500 via ROM 502 and / or communication unit 509.

[0258] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0259] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0260] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0261] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0262] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0263] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0264] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this disclosure are performed, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a terminal, a user equipment, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center integrating one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD).

[0265] Although this disclosure has been described in conjunction with specific features and embodiments, it will be apparent that various modifications and combinations can be made therein without departing from the spirit and scope of this disclosure. Accordingly, this specification and drawings are merely exemplary illustrations of the disclosure as defined by the appended claims and are to be considered as covering any and all modifications, variations, combinations, or equivalents within the scope of this disclosure. It is obvious that those skilled in the art can make various alterations and modifications to this disclosure without departing from its spirit and scope. Thus, this disclosure is also intended to include any such modifications and modifications that fall within the scope of the claims of this disclosure and their equivalents.< / udftype> < / udftype> < / udftype> < / udftype> < / udftype> < / sqltype> < / dataloadtype> < / dataloadtype> < / dataloadtype> < / genmode> < / genmode> < / genmode> < / udftype> < / udftype> < / udftype> < / udftype> < / udftype> < / udftype> < / udftype> < / udftype> < / sqltype> < / dataloadtype> < / dataloadtype> < / dataloadtype> < / dataloadtype> < / dataloadtype> < / dataloadtype> < / dataloadtype> < / content> < / functionname> < / content> < / functionname> < / dbtype> < / content> < / functionname> < / dbtype>

Claims

1. A method for generating database stored functions, characterized in that, The method includes: Receive the raw code to be processed and obtain the syntax structure information corresponding to the raw code; Obtain user-preset attribute information and database stored function generation template; the attribute information is used to define the parameter attribute values ​​of the database stored function, and the attribute information includes at least one of database type, code generation mode, data loading method and user-defined function; The database storage function generation template, the attribute information, and the original code are input into the database storage function generation module to obtain multiple database storage function sub-results. The database storage function generation module includes a code classification sub-module and a database storage function conversion sub-module. The code classification sub-module splits the original code into multiple code blocks based on the code type. The database storage function conversion sub-module adjusts the database storage function generation template based on the code type and the attribute information, and converts the code blocks corresponding to each code type into corresponding database storage function sub-results based on the adjusted database storage function generation template. Based on the grammatical structure information, the multiple database storage function sub-results are combined to obtain the target database storage function.

2. The method according to claim 1, characterized in that, The process of splitting the original code into multiple code blocks based on code type includes: Obtain preset code classification rules; the code classification rules include key parameters corresponding to multiple code types, and / or regular expressions; Iterate through the lines of code in the original code; If the code line includes the key parameter and / or the regular expression, determine the code type corresponding to the code line; Merge lines of code of the same type into code blocks to obtain code blocks corresponding to each code type.

3. The method according to claim 1, characterized in that, The code type includes data loading code; The method further includes: When the code type is data loading code, the data loading mode is determined based on the attribute information; Based on the data loading mode, the database storage function generation template is adjusted to obtain the adjusted database storage function generation template. Obtain the first identifier information in the code block corresponding to the data loading code, and fill the first identifier information into the adjusted database storage function generation template to obtain the database storage function sub-result corresponding to the data loading code.

4. The method according to claim 3, characterized in that The method further includes: Retrieve pre-defined field type rules; The field type definition rules are used to determine the field type corresponding to each field in the database storage function; the field type definition rules include defining the field type of each field as the first field type, and / or setting the corresponding field type based on different field classifications; Fill the field type into the adjusted database stored function generation template.

5. The method according to claim 1, characterized in that, The code type includes data manipulation code; the method further includes: When the code type is data manipulation code, obtain the data manipulation operator mapping rule; the data manipulation operator mapping rule includes the mapping relationship between the original code and the database storage function; Based on the data operation operator mapping rules and the attribute information, the database storage function generation template is adjusted to obtain the adjusted database storage function generation template. Obtain the second identifier information from the code block corresponding to the data operation code, and fill the second identifier information into the adjusted database storage function generation template to obtain the database storage function sub-result corresponding to the data operation code.

6. The method according to claim 5, characterized in that, The method further includes: Obtain the database stored function generation pattern from the attribute information; the database stored function generation pattern includes a single statement generation pattern or a nested query statement generation pattern. Based on the database storage function generation mode and the adjusted database storage function generation template, the database storage function sub-results corresponding to the data operation code are obtained.

7. The method according to claim 1, characterized in that, The code type includes user-defined function code; the method further includes: When the code type is user-defined function code, obtain the third identification information in the code block corresponding to the user-defined function code; The third identification information is filled into the database storage function generation template to obtain the adjusted database storage function generation template. Based on the attribute information, determine the field type corresponding to each field in the user-defined function code; Based on the field type and the adjusted database storage function generation template, the database storage function sub-result corresponding to the user-defined function code is obtained.

8. A database stored function generation device, characterized in that, The device includes: The data acquisition module is used to receive the raw code to be processed and to acquire the syntax structure information corresponding to the raw code; The data processing module is used to obtain user-preset attribute information and database storage function generation templates; the attribute information is used to define the parameter attribute values ​​of the database storage functions, and the attribute information includes at least one of database type, code generation mode, data loading method, and user-defined functions; The data processing module is further configured to input the database storage function generation template, the attribute information, and the original code into the database storage function generation module to obtain multiple database storage function sub-results; the database storage function generation module includes a code classification sub-module and a database storage function conversion sub-module, wherein the code classification sub-module splits the original code into multiple code blocks based on the code type; the database storage function conversion sub-module adjusts the database storage function generation template based on the code type and the attribute information, and converts the code blocks corresponding to each code type into corresponding database storage function sub-results based on the adjusted database storage function generation template; The data processing module is also used to combine the multiple database storage function sub-results based on the syntax structure information to obtain the target database storage function.

9. An electronic device, characterized in that, include: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the steps of the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1-7.

Images