Data processing method and device and electronic equipment
By generating SQL statements and Groovy code, optimizing query logic and dynamically specifying databases and tables, the problem of performance bottlenecks in traditional databases in large-scale data processing is solved, the flexibility and efficiency of data processing is improved, and the maintenance of the system is enhanced. and scalability.
Patent Information
- Application Number
- CN202411078148.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-26
AI Technical Summary
Traditional database storage methods encounter performance bottlenecks when processing large-scale data, especially in query operations across multiple databases and tables, resulting in low performance and difficulty in improving the flexibility and efficiency of data processing.
Generate SQL statements and Groovy code through configuration instructions and preset logical rules, dynamically specify the target database and data tables, optimize query logic, reduce unnecessary table connections and data retrieval, realize cross-border query, and logically transform database operations into Reduce code coupling.
Improves the flexibility and efficiency of data processing, reduces development time and human errors, enhances the maintainability and scalability of the system, and adapts to changing business needs and user preferences.
Smart Images

Figure CN119025541A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, device and electronic device. Background Art
[0002] As the amount of data and access volume increases, when the amount of database data becomes very large, the traditional database storage method can no longer meet the needs of large-scale data processing. It is inevitable that data performance bottlenecks will be encountered, and data reading and writing performance will become very low.
[0003] In the prior art, database and table sharding is used to improve the data reading and writing performance, but query operations across multiple databases and tables are very time-consuming. How to improve the flexibility and efficiency of data processing has become a question worth discussing. Summary of the invention
[0004] The embodiments of the present application provide a data processing method, device and electronic device for improving the flexibility and efficiency of data processing.
[0005] In a first aspect, an embodiment of the present application provides a data processing method, the method comprising:
[0006] In response to the configuration instruction, an SQL statement is generated based on the configuration instruction and preset logic rules, the configuration instruction includes a plurality of interest-based screening conditions, and the logic rules are used to define how to convert the interest-based screening conditions into SQL statements;
[0007] Use logical operators to logically combine multiple interest-based screening conditions to obtain a logical expression, and generate Groovy code based on the logical expression and the preset configuration expression template. The logical expression is used to represent the business logic;
[0008] The target data table of the target database is determined based on the Groovy code, and the data in the target data table is processed using SQL statements.
[0009] In the above method, by generating SQL statements and Groovy codes based on configuration instructions, the data in the database can be flexibly processed according to business needs. When business needs change, there is no need to make large-scale modifications to the database structure or code. Only the configuration instructions need to be adjusted to achieve accurate data screening and operation. At the same time, when generating SQL statements, logical rules can be used to optimize the query logic, reduce unnecessary table connections and data retrieval, and thus improve query efficiency. In addition, Groovy code can also implement more complex logical judgments and data processing operations in the code to improve the flexibility of data processing. In the scenario of sharding databases and tables, the target database and data table for the query can be dynamically specified through configuration instructions, thereby realizing query operations across multiple databases and tables. At the same time, query efficiency can be further improved by optimizing query logic and reducing unnecessary cross-database queries. By logicalizing database operations and encapsulating them in Groovy code, the coupling of the code can be reduced and maintainability can be improved. At the same time, since the configuration instructions and logical rules are independent of the code itself, new screening conditions can be added or existing conditions can be modified based on user needs in the future to adapt to changes in business needs and improve user experience.
[0010] Optionally, the above-mentioned generation of SQL statements based on configuration instructions and preset logical rules specifically includes:
[0011] Map the interest screening conditions to strings in SQL statements to obtain multiple query statements;
[0012] Combine query statements to get SQL statements.
[0013] In the above method, by mapping the filter conditions into SQL strings, various complex query requirements can be flexibly handled. When business requirements change, only the filter conditions need to be updated without modifying the structure of the SQL statement, thereby improving the scalability of the system.
[0014] Optionally, the above-mentioned interest screening conditions are mapped into strings in SQL statements to obtain multiple query statements, including:
[0015] Select the preset SQL statement template according to the query type;
[0016] Insert the character string corresponding to the interesting screening condition into the SQL statement template to obtain multiple query statements.
[0017] In the above method, when processing multiple data sources, mapping the filter conditions to strings in SQL statements can more easily implement cross-database queries. Through a common SQL statement template, dynamically adjusting the query part in the SQL statement can implement data processing between different data sources.
[0018] Optionally, the configuration expression template includes placeholders, which are replaced by logical expressions during the Groovy code generation process. The Groovy code is generated based on the logical expression and the preset configuration expression template, specifically including:
[0019] Fill the logical expression into the corresponding position of the placeholder to get the Groovy code.
[0020] In the above method, by using the configuration expression template, a large amount of Groovy code can be quickly generated without manually writing each line of code. This can significantly reduce development time, improve the degree of automation of the development process, and reduce the possibility of human errors. In addition, business logic can be easily adjusted by modifying configuration instructions without large-scale modification of the code. It is easy to adapt to changing business needs and user preferences.
[0021] Optionally, the above method further includes:
[0022] Get the configuration file and read the logical service information defined in the configuration file. The logical service information includes logical rules and configuration expressions.
[0023] In the above method, the logical service information is defined through the configuration file, so that the database processing logic can be flexibly configured. When the business requirements change, only the logical service information in the configuration file needs to be modified without modifying the program code, which improves the maintainability and scalability of the system. It is convenient for users to easily view, modify and verify the logical service information in the configuration file in the future to ensure the correctness and consistency of the database processing logic. In addition, the configuration file can be configured for different environments. By reading the configuration files of different environments, the system can automatically adapt to different database processing requirements, reducing errors and conflicts caused by environmental differences.
[0024] In a second aspect, an embodiment of the present application provides a data processing device, the device comprising:
[0025] A generation module, for generating SQL statements in response to configuration instructions based on the configuration instructions and preset logic rules, wherein the configuration instructions include a plurality of interest-based screening conditions, and the logic rules are used to define how to convert the interest-based screening conditions into SQL statements;
[0026] The generation module is also used to use logical operators to logically combine multiple interest screening conditions to obtain a logical expression, and generate Groovy code based on the logical expression and a preset configuration expression template. The logical expression is used to represent the business logic;
[0027] The processing module is used to determine the target data table of the target database based on the Groovy code, and use SQL statements to perform data processing in the target data table.
[0028] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the processor implements any one of the data processing methods in the first aspect.
[0029] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the data processing method of any one of the first aspects is implemented.
[0030] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program, which is executed by a processor to implement a data processing method as described in any one of the first aspects above.
[0031] The technical effects brought about by any one of the implementation methods in the second to fifth aspects can refer to the technical effects brought about by the corresponding implementation method in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 A schematic diagram of an application scenario of data processing provided in an embodiment of the present application;
[0033] Figure 2 A flow chart of a data processing method provided in an embodiment of the present application;
[0034] Figure 3 An exemplary flow chart of data processing provided in an embodiment of the present application;
[0035] Figure 4 A schematic diagram of a data processing device provided in an embodiment of the present application;
[0036] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0037] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0038] The application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems. In the description of the present application, unless otherwise specified, the meaning of "multiple" is two or more.
[0039] As the amount of data and access volume increases, when the amount of database data becomes very large, the traditional database storage method can no longer meet the needs of large-scale data processing. It is inevitable that data performance bottlenecks will be encountered, and data reading and writing performance will become very low.
[0040] In the prior art, database and table sharding is used to improve the data reading and writing performance, but query operations across multiple databases and tables are very time-consuming. How to improve the flexibility and efficiency of data processing has become a question worth discussing.
[0041] In order to solve the above problems, an embodiment of the present application provides a data processing method. In the method, in response to a configuration instruction, an SQL statement is generated based on the configuration instruction and a preset logical rule. The configuration instruction includes a plurality of interest-based filtering condition logical rules for defining how to convert the interest-based filtering conditions into SQL statements. A logical expression is generated based on the configuration instruction, and a preset configuration expression template is used to generate Groovy code, and the logical expression is used to determine the program behavior according to the filtering condition. The target data table of the target database is determined based on the Groovy code, and data processing is performed in the target data table using an SQL statement.
[0042] In this way, by generating SQL statements and Groovy code based on configuration instructions, the data in the database can be processed flexibly according to business needs. When business needs change, there is no need to make large-scale modifications to the database structure or code. You only need to adjust the configuration instructions to achieve accurate data screening and operation. At the same time, when generating SQL statements, you can use logical rules to optimize the query logic, reduce unnecessary table connections and data retrieval, and thus improve query efficiency. In addition, Groovy code can also implement more complex logical judgments and data processing operations in the code to improve the flexibility of data processing. In the scenario of sharding, the target database and data table of the query can be dynamically specified through configuration instructions, thereby realizing query operations across multiple databases and tables. At the same time, by optimizing the query logic and reducing unnecessary cross-database queries, the query efficiency can be further improved. By logicalizing database operations and encapsulating them in Groovy code, the coupling of the code can be reduced and the maintainability can be improved. At the same time, since the configuration instructions and logical rules are independent of the code itself, new filtering conditions can be added or existing conditions can be modified based on user needs in the future to adapt to changes in business needs and improve user experience.
[0043] A schematic diagram of an application scenario of an optional data processing method in an embodiment of the present application. Figure 1 As shown, it includes a server 100 and a terminal 101. The server 100 and the terminal 101 can be communicatively connected through a network to implement the data processing method of the present application.
[0044] The user can use the server 100 to interact with the terminal 101 through the network, such as receiving or sending messages, etc. Various client applications can be installed on the terminal 101, such as program writing applications, web browser applications, search applications, etc.
[0045] It is understandable that in the embodiment of the present application, the server 100 can be implemented as an independent server or a server cluster composed of multiple servers. The terminal 101 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, desktop computers, etc.
[0046] The server 100 can be used to generate SQL statements based on the configuration instructions and preset logic rules in response to configuration instructions, wherein the configuration instructions include multiple interest-based screening conditions, and the logic rules are used to define how to convert the interest-based screening conditions into SQL statements. Generate logical expressions based on the configuration instructions, and generate Groovy code based on the preset configuration expression template, wherein the logical expressions are used to determine program behavior based on the screening conditions. Determine the target data table of the target database based on the Groovy code, and use SQL statements to process data in the target data table.
[0047] like Figure 2 As shown, a flowchart of a data processing method provided in an embodiment of the present application may specifically include the following steps.
[0048] Step S201: In response to a configuration instruction, generate an SQL statement based on the configuration instruction and preset logical rules.
[0049] The configuration instruction includes a plurality of interest-based screening conditions, and the logic rule is used to define how to convert the interest-based screening conditions into SQL statements.
[0050] Optionally, before the server responds to the configuration instruction, the server may also obtain a configuration file set by the user and read the logical service information defined in the configuration file, wherein the logical service information includes logical rules and configuration expressions.
[0051] In this way, the logical service information is defined through the configuration file, so that the database processing logic can be flexibly configured. When business needs change, only the logical service information in the configuration file needs to be modified without modifying the program code, which improves the maintainability and scalability of the system. It is convenient for users to easily view, modify and verify the logical service information in the configuration file in the future to ensure the correctness and consistency of the database processing logic. In addition, the configuration file can be configured for different environments. By reading the configuration files of different environments, the system can automatically adapt to different database processing requirements, reducing errors and conflicts caused by environmental differences.
[0052] In an optional embodiment, when a user needs to create a data processing task, the user can select multiple interest-based screening conditions through a client for creating data processing tasks. The client sends a configuration instruction including multiple interest-based screening conditions to a server, and the server executes the creation of the data processing task.
[0053] For example, the user can double-click the icon of the client to start the corresponding client. The client display interface may include multiple screening conditions, such as time period, location, and user age group.
[0054] In an optional embodiment, the server can map the interest screening conditions into strings in SQL statements to obtain multiple query statements, and combine the query statements to obtain SQL statements.
[0055] In this way, by mapping the filter conditions to SQL strings, various complex query requirements can be flexibly handled. When business requirements change, only the filter conditions need to be updated without modifying the structure of the SQL statement, thereby improving the scalability of the system.
[0056] For example, the server can parse a configuration directive containing multiple interest-based filters. These filters may be stored in key-value pairs, lists, dictionaries, or any other structured format. The server can extract all the user-selected interest-based filters, including field names, operators (such as equal to, greater than, contains, etc.), and values. Map the fields of the interest-based filters to strings in SQL statements. For example, the conversion of field values to strings that are appropriately formatted in SQL queries (such as date, time, escape of special characters, etc.).
[0057] Optionally, after completing the string conversion, the server can also select a preset SQL statement template according to the query type, and insert the string corresponding to the interest screening condition into the SQL statement template to obtain multiple query statements.
[0058] For example, the preset SQL statement template may include a where clause, a select clause, a from clause, and other clauses. The select clause is used to specify the columns or expressions to be returned by the query. The select clause can be used to select one or more fields in a database table, and aggregate calculations or data conversions can also be performed. The from clause is used to specify the table or view from which the query retrieves data. The where clause is used to specify the conditions for filtering records.
[0059] When the server combines query statements to obtain an SQL statement, it can combine other clauses such as the select clause, where clause, from clause, etc. to form a complete SQL statement.
[0060] In this way, when processing multiple data sources, mapping the filter conditions to strings in SQL statements can make it easier to implement cross-database queries. Through a common SQL statement template, dynamically adjusting the query part in the SQL statement can realize data processing between different data sources.
[0061] Optionally, after the SQL statement is generated, it should be verified to ensure that its syntax is correct and meets the requirements of the database. For example, unit testing or integration testing can be used to verify whether the execution results of the SQL statement on the preset database meet the expectations.
[0062] Step S202: logically combine multiple interest-based screening conditions to obtain a logical expression, and generate Groovy code based on the logical expression and a preset configuration expression template.
[0063] Among them, logical expressions are used to express business logic.
[0064] In an optional embodiment, the server can extract multiple interest-based screening conditions from the configuration instruction. Multiple interest-based screening conditions are logically combined to obtain a logical expression. Among them, when performing logical combination, three basic logical operators can be used: and (and), or (or), and not (not). Multiple interest-based screening conditions are combined using logical operators to form a more complex logical expression.
[0065] Optionally, the preset configuration expression template can contain placeholders. The placeholders are replaced by actual logical expressions during the Groovy code generation process. The logical expressions are filled into the corresponding positions in the configuration expression template to generate Groovy code.
[0066] In this way, by using configuration expression templates, a large amount of Groovy code can be quickly generated without having to manually write every line of code. This can significantly reduce development time, improve the degree of automation in the development process, and reduce the possibility of human errors. In addition, business logic can be easily adjusted by modifying configuration instructions without large-scale modifications to the code. It is easy to adapt to changing business needs and user preferences.
[0067] Step S203: determine the target data table of the target database based on the Groovy code, and use SQL statements to perform data processing in the target data table.
[0068] In an optional embodiment, since the target data table is dynamically specified by configuration instructions, the target data table can be extracted by parsing the Groovy code, and the database connection object in Groovy is used to execute SQL statements to perform data processing in the target data table, such as data update, data deletion, etc.
[0069] like Figure 3 As shown, an exemplary flow chart of a data processing method provided in an embodiment of the present application may include the following steps:
[0070] Step S301, obtaining a configuration file, and reading the logical service information defined in the configuration file;
[0071] Step S302: In response to the configuration instruction, generate an SQL statement based on the configuration instruction and preset logical rules;
[0072] Step S303: Select a preset SQL statement template according to the query type;
[0073] Step S304: Select a preset SQL statement template according to the query type;
[0074] Step S305: insert the character string corresponding to the interest screening condition into the SQL statement template to obtain multiple query statements;
[0075] Step S306, combining query statements to obtain SQL statements;
[0076] Step S307: logically combine multiple interest screening conditions using logical operators to obtain a logical expression;
[0077] Step S308: Fill the logical expression into the corresponding position of the placeholder to obtain the Groovy code;
[0078] Step S309: determine the target data table of the target database based on the Groovy code, and use SQL statements to perform data processing in the target data table.
[0079] A schematic diagram of the structure of a data processing device provided in an embodiment of the present application. Figure 4 As shown, the device includes: a generating module 401 and a processing module 402.
[0080] A generation module 401 is used to generate SQL statements in response to configuration instructions based on the configuration instructions and preset logic rules, where the configuration instructions include multiple interest-based screening conditions, and the logic rules are used to define how to convert the interest-based screening conditions into SQL statements;
[0081] The generation module 401 is further used to use a logical operator to logically combine multiple interest screening conditions to obtain a logical expression, and generate Groovy code based on the logical expression and a preset configuration expression template, where the logical expression is used to represent the business logic;
[0082] The processing module 402 is used to determine the target data table of the target database based on the Groovy code, and use SQL statements to perform data processing in the target data table.
[0083] Optionally, the above-mentioned SQL statement is generated based on the configuration instruction and the preset logical rules, and the generation module 401 is specifically used for:
[0084] Map the interest screening conditions to strings in SQL statements to obtain multiple query statements;
[0085] Combine query statements to get SQL statements.
[0086] Optionally, the above-mentioned mapping of the interest screening conditions into a string in an SQL statement to obtain multiple query statements, the generation module 401 is specifically used for:
[0087] Select the preset SQL statement template according to the query type;
[0088] Insert the character string corresponding to the interesting screening condition into the SQL statement template to obtain multiple query statements.
[0089] Optionally, the configuration expression template includes placeholders, which are replaced by logical expressions during the Groovy code generation process. The Groovy code is generated based on the logical expression and the preset configuration expression template. The generation module 401 is specifically used for:
[0090] Fill the logical expression into the corresponding position of the placeholder to get the Groovy code.
[0091] Optionally, the processing module 402 is further configured to:
[0092] Get the configuration file and read the logical service information defined in the configuration file. The logical service information includes logical rules and configuration expressions.
[0093] Based on the same technical concept, an electronic device is also provided in an embodiment of the present application, and the electronic device can realize the functions of the aforementioned distributed training device.
[0094] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
[0095] At least one processor 501, and a memory 502 connected to the at least one processor 501. The specific connection medium between the processor 501 and the memory 502 is not limited in the embodiment of the present application. Figure 5 In the example, the processor 501 and the memory 502 are connected via a bus 500. The bus 500 is Figure 5 The connections between other components are shown in bold lines, and are not intended to be limiting. The bus 500 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 501 can also be called a controller, and there is no limitation on the name.
[0096] In the embodiment of the present application, the memory 502 stores instructions that can be executed by at least one processor 501. The at least one processor 501 can execute a distributed training method discussed above by executing the instructions stored in the memory 502. The processor 501 can implement Figure 4 The functions of each module in the device shown.
[0097] Among them, the processor 501 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 502 and calling data stored in the memory 502, the various functions of the device and processing data, the device can be monitored as a whole.
[0098] In one possible design, the processor 501 may include one or more processing units, and the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a driver interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 501. In some embodiments, the processor 501 and the memory 502 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.
[0099] Processor 501 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of a distributed training method disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware processor execution, or can be executed by a combination of hardware and software modules in the processor.
[0100] The memory 502 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 502 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 502 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 502 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.
[0101] By programming the processor 501, the code corresponding to the distributed training method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 2 The embodiment shown is a distributed training method. How to design and program the processor 501 is a technique known to those skilled in the art and will not be described in detail here.
[0102] It should be noted here that the above-mentioned electronic device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.
[0103] An embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a distributed training method in the above embodiment.
[0104] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0105] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 Flow or flows and / or boxes Figure 1 A device that performs the functions specified in a box or multiple boxes.
[0106] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 Flow or flows and / or boxes Figure 1 The functions specified in one or more boxes.
[0107] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 Flow or flows and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0108] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A data processing method, It is characterized in that The method comprises: In response to a configuration instruction, generating an SQL statement based on the configuration instruction and a preset logic rule, the configuration instruction includes a plurality of interest-based screening conditions, and the logic rule is used to define how to convert the interest-based screening conditions into an SQL statement; Using logical operators to logically combine the multiple interest-based screening conditions to obtain a logical expression, and generating Groovy code based on the logical expression and a preset configuration expression template, wherein the logical expression is used to represent the business logic; A target data table of a target database is determined based on the Groovy code, and the SQL statement is used to perform data processing in the target data table.
2. The method according to claim 1, It is characterized in that The generating of SQL statements based on the configuration instructions and preset logical rules specifically includes: Mapping the interest screening conditions into strings in SQL statements to obtain multiple query statements; The query statements are combined to obtain the SQL statement.
3. The method according to claim 1, It is characterized in that Mapping the interest screening conditions into strings in SQL statements to obtain multiple query statements specifically includes: Select the preset SQL statement template according to the query type; Insert the character string corresponding to the interest-based screening condition into the SQL statement template to obtain the multiple query statements.
4. The method according to claim 1, It is characterized in that The configuration expression template includes a placeholder, and the placeholder is replaced by a logical expression during the Groovy code generation process. The Groovy code is generated based on the logical expression and the preset configuration expression template, specifically including: The logical expression is filled into the corresponding position of the placeholder to obtain the Groovy code.
5. The method according to claim 1, It is characterized in that Before responding to the configuration instruction, the method further includes: A configuration file is obtained, and logical service information defined in the configuration file is read, wherein the logical service information includes the logical rule and the configuration expression.
6. A data processing device, It is characterized in that The device comprises: A generation module, configured to generate an SQL statement in response to a configuration instruction based on the configuration instruction and a preset logic rule, wherein the configuration instruction includes a plurality of interest-based screening conditions, and the logic rule is used to define how to convert the interest-based screening conditions into an SQL statement; The generation module is further used to use a logical operator to logically combine the multiple interest-based screening conditions to obtain a logical expression, and generate Groovy code based on the logical expression and a preset configuration expression template, wherein the logical expression is used to determine program behavior according to the interest-based screening conditions; The processing module is used to determine the target data table of the target database based on the Groovy code, and use the SQL statement to perform data processing in the target data table.
7. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.
8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and the computer program is used to cause the computer to execute the method according to any one of claims 1 to 5.
9. A computer program product, It is characterized in that When the computer program product is called by a computer, the computer executes the method according to any one of claims 1 to 5.
Citation Information
Cited By
Layered data processing method and system for rail transit
WO2026144462A1
