Data processing method, device, equipment and medium

Abstract

The disclosed embodiments relate to a data processing method, apparatus, device, and medium, wherein the method comprises: obtaining a data processing statement from a local database, and extracting a target function and target data parameters from the processing statement; determining whether the target function is a local function corresponding to the local database; if it is not a local function, obtaining the function attributes of the target function; determining whether the function attributes include a preset first attribute; if the first attribute is included, determining whether the target data parameter includes a preset parameter corresponding to the first attribute; if the preset parameter is included, determining that the query value of the data processing statement is a preset value corresponding to the preset parameter. Thus, for external processing functions, function optimization can be performed in advance, and the corresponding execution results can be directly obtained for functions that do not need to be executed externally, thereby improving data processing efficiency and reducing computing power consumption.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a data processing method, apparatus, device, and medium. Background Art

[0002] A database can be thought of as a warehouse that organizes, stores, and manages data according to a data structure. It's a collection of large amounts of data stored permanently on a computer in an organized, shareable, and uniformly managed manner. Database management of large amounts of data can be performed using local worksheet functions or by defining external functions to implement complex data management capabilities.

[0003] In related technologies, if the database supports access to external functions in SQL statements, it is necessary to directly call the external data source and pull the external function from the external data source to perform relevant data processing.

[0004] However, the above-mentioned method of directly calling the external data source when accessing the external function requires a large amount of computing power to call and run the external data source, and results in low efficiency in processing the relevant data. Summary of the Invention

[0005] In order to solve the above technical problems, the present disclosure provides a data processing method, device, equipment and medium, which can pre-optimize external processing functions and directly obtain corresponding execution results for functions that do not need to be executed externally, thereby improving data processing efficiency and reducing computing power consumption.

[0006] The present disclosure provides a data processing method, comprising: obtaining a data processing statement from a local database, and extracting a target function and target data parameters from the processing statement; determining whether the target function is a local function corresponding to the local database;

[0007] If it is not the local function, obtain the function properties of the target function; determine whether the function properties include a preset first property; if the first property is included, determine whether the target data parameter includes a preset parameter corresponding to the first property; if the preset parameter is included, determine that the query value of the data processing statement is a preset value corresponding to the preset parameter.

[0008] An embodiment of the present disclosure also provides a data processing device, which includes: an extraction module for obtaining a data processing statement from a local database and extracting a target function and target data parameters from the processing statement; a first judgment module for judging whether the target function is a local function corresponding to the local database; an acquisition module for obtaining function properties of the target function when it is not the local function; a second judgment module for judging whether the function properties include a preset first property; a third judgment module for judging whether the target data parameter includes a preset parameter corresponding to the first property when the first property is included; and a determination module for determining that the query value of the data processing statement is a preset value corresponding to the preset parameter when the preset parameter is included.

[0009] An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing instructions executable by the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the data processing method provided by the embodiment of the present disclosure.

[0010] An embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the data processing method provided by the embodiment of the present disclosure.

[0011] The technical solution provided by the embodiments of the present disclosure has the following advantages over the prior art:

[0012] The data processing solution provided by the embodiment of the present disclosure obtains the data processing statement of the local database, extracts the target function and target data parameters in the processing statement, determines whether the target function is a local function corresponding to the local database, and if it is not a local function, obtains the function attributes of the target function, and then determines whether the function attributes include a preset first attribute. If the first attribute is included, then determines whether the target data parameter includes a preset parameter corresponding to the first attribute. If the preset parameter is included, then determines that the query value of the data processing statement is a preset value corresponding to the preset parameter. As a result, the external processing function can be optimized in advance, and the corresponding execution results of the function that does not need to be executed externally can be directly obtained, thereby improving data processing efficiency and reducing computing power consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0014] Figure 1A flowchart of a data processing method provided in an embodiment of the present disclosure;

[0015] Figure 2 A flowchart of another data processing method provided by an embodiment of the present disclosure; and

[0016] Figure 3 A schematic diagram of the structure of a data processing device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0018] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0019] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0020] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] In order to solve the above problems, the embodiments of the present disclosure provide a data processing method, which is introduced below in conjunction with specific embodiments.

[0024] Figure 1 This is a flow chart of a data processing method provided by an embodiment of the present disclosure. The method can be executed by a data processing device, wherein the device can be implemented using software and / or hardware and can generally be integrated into an electronic device. Figure 1 As shown, the method includes:

[0025] Step 101: Obtain a data processing statement from a local database and extract the target function and target data parameters in the processing statement;

[0026] In this embodiment, a data processing statement, i.e., an SQL statement, is obtained from the local database. The statement includes a target function and target data parameters corresponding to the current data processing request. The target function is a specific function for performing data processing, and the target data parameters correspond to the data to be processed, or the working area (rows and columns, etc.) of the data to be processed in a corresponding table.

[0027] For example, when the data processing statement is Select func@dblink(column1,column2)fromtableA, the target function is func, dblink: an object created in the database that specifies how to connect to other databases, column1 and column2 are target data parameters, corresponding to the table and table range where the data to be processed is located.

[0028] Step 102, determining whether the target function is a local function corresponding to the local database;

[0029] In this embodiment, it is determined whether the target function is a local function corresponding to the local database. If it is not a local function, it means that an external data source may need to be called. Therefore, in order to reduce the consumption of computing power and improve data processing efficiency, the previous optimization processing can be further performed.

[0030] In one embodiment of the present disclosure, if the target function is a local function corresponding to the local database, optimization processing can also be performed. The optimization processing includes optimization processing of external functions. The implementation principle is similar and will not be repeated here.

[0031] It should be noted that in different application scenarios, the method of determining whether the target function is a local function corresponding to the local database is different. The following examples are examples:

[0032] Example 1:

[0033] In this example, a local function set of the local database is pre-built to determine whether the target function belongs to the local function set. If it does not belong to the local function set, the target function is determined to be an external function.

[0034] Example 2:

[0035] In this example, if each function carries the identifier of the database to which it belongs, the first identifier carried by the target function is extracted, and then the second identifier of the local database is obtained to determine whether the first identifier and the second identifier are consistent. If they are inconsistent, the target function is determined to be an external function.

[0036] Step 103: If it is not a local function, obtain the function attributes of the target function;

[0037] Among them, function attributes may include strict attributes and immutable attributes.

[0038] In some possible embodiments, the function properties of each function are pre-stored in a remote preset cloud server. Therefore, in this embodiment, a property query request of the target function is sent to the preset cloud server, for example, to obtain the function identifier of the target function, which includes but is not limited to the function name and other information that can uniquely represent the function, and then, the function properties fed back by the cloud server in response to the property query request are obtained.

[0039] During the actual execution process, the legitimacy of the target function can also be verified before obtaining the function properties of the target function, that is, obtaining the preset external data source function list, judging whether the target function belongs to the external data source function list, and reporting an error if it does not belong to the external data source function list; if it belongs to the external data source function list, obtaining the first parameter type of the target data parameter, querying the preset second parameter type of the target function, and determining whether the first parameter type and the second parameter type match; obtaining the function properties of the target function only when they match, otherwise reporting an error.

[0040] Step 104, determining whether the function attributes include a preset first attribute;

[0041] Step 105: If the first attribute is included, determine whether the target data parameter includes a preset parameter corresponding to the first attribute;

[0042] Step 106: If the preset parameter is included, determine that the query value of the data processing statement is a preset value corresponding to the preset parameter.

[0043] It can be understood that for the objective function of the first attribute, when the objective function performs data processing, its output value uniquely corresponds to the specific input value, that is, if the function attribute includes the preset first attribute, it is determined whether the target data parameter contains the preset parameter corresponding to the first attribute. If it contains the preset parameter, the output value of the objective function is fixed. Therefore, the query value of the data processing statement is directly determined to be the preset value corresponding to the preset parameter. Therefore, there is no need to execute the objective function again in the external data source, which improves the data processing efficiency and reduces the consumption of computing power.

[0044] For example, if the first attribute is a strict attribute, the preset parameter corresponding to the first attribute is a null value parameter, that is, NULL. When the target data parameter includes NULL, the query value of the data processing statement is directly determined to be NULL.

[0045] In one embodiment of the present disclosure, Figure 2 As shown, after determining whether the function attributes include the preset first attribute, the following steps are further included:

[0046] Step 201: If the preset first attribute is not included, it is determined whether the function attribute includes the preset second attribute.

[0047] In this embodiment, if the preset first attribute is not included, it is determined whether the function attribute includes the preset second attribute.

[0048] The objective function of the second attribute can obtain the same result execution value based on the same input target data parameter.

[0049] Step 202: If the second attribute is included, query the preset query result database according to the target data parameter.

[0050] In this embodiment, if the second attribute is included, a preset query result database is queried according to the target data parameter, and the query result database includes each external function of the second attribute and the execution value of each external function corresponding to different data parameters.

[0051] Step 203: If a matching value that matches the target data parameter is found, the query value of the data processing statement is determined to be a matching value.

[0052] In this embodiment, if a matching value that matches the target data parameter is found, the query value of the data processing statement is determined to be a matching value, thereby obtaining the corresponding data processing result value without executing the corresponding target function again.

[0053] For example, if the second attribute is immutable, then during the execution phase, for the same input target data parameter, the execution result value is the same, so only one historical execution is required. The same parameters can be executed once without feedback calling an external function, especially in scenarios where the data is local and the function is external.

[0054] In this embodiment, if no matching value matching the target data parameter is found, that is, the corresponding execution result value is not obtained in the recorded preset query result data, the external database corresponding to the target function is determined, a data processing statement is sent to the external database, and the execution result value fed back by the external database in response to the data processing statement is obtained. The correspondence between the execution result value and the target data parameter is stored in the preset query result database, so that when the target function of the second attribute is subsequently executed, the output result corresponding to the same target data parameter can be reused.

[0055] In one embodiment of the present disclosure, if the function attributes do not include the second attribute, it is determined whether the database corresponding to the target data parameter is a local database. For example, it can be determined whether the data table corresponding to the target data parameter is in the local database based on the table identifier corresponding to the target query statement (such as the above-mentioned tableA). If it is in the local database, the database corresponding to the target data parameter is considered to be a local database.

[0056] Furthermore, if it is in a local database, target data corresponding to the target data parameter is obtained in the local database, and the target data and data processing statements are sent to a database corresponding to the target function.

[0057] Of course, in this embodiment, if the database corresponding to the target data parameter is not a local database, the target data corresponding to the target data parameter can also be obtained from an external database, and the target data and data processing statements can be sent to the database corresponding to the target function.

[0058] In one embodiment of the present disclosure, each objective function and its corresponding input data and corresponding execution result value can also be stored in a preset cloud server. Therefore, when it is determined that the attributes of the objective function do not include the second attribute, it is also possible to obtain the data corresponding to the target data parameter of the objective function, obtain the function identifier of the objective function, and send a query request carrying the function identifier and the corresponding data to the cloud server. If the execution result value fed back by the cloud server is obtained, the result value can be directly reused without executing the corresponding external function.

[0059] In summary, the data processing method of the embodiment of the present disclosure obtains the data processing statement of the local database, extracts the target function and target data parameters in the processing statement, determines whether the target function is a local function corresponding to the local database, and if it is not a local function, obtains the function properties of the target function, and then determines whether the function properties include a preset first attribute. If the first attribute is included, then determines whether the target data parameter includes a preset parameter corresponding to the first attribute. If the preset parameter is included, then determines that the query value of the data processing statement is a preset value corresponding to the preset parameter. As a result, the external processing function can be optimized in advance, and the corresponding execution results can be directly obtained for functions that do not need to be executed externally, thereby improving data processing efficiency and reducing computing power consumption.

[0060] Figure 3 This is a structural diagram of a data processing device provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can generally be integrated into an electronic device. Figure 3 As shown, the device includes:

[0061] Extraction module 301, used to obtain data processing statements from the local database and extract target functions and target data parameters from the processing statements;

[0062] A first judging module 302 is configured to judge whether the target function is a local function corresponding to the local database;

[0063] An acquisition module 303 is configured to acquire function properties of the target function when the target function is not a local function;

[0064] A second judging module 304 is configured to judge whether the function attributes include a preset first attribute;

[0065] A third judgment module 305 is configured to judge whether the target data parameter includes a preset parameter corresponding to the first attribute when the first attribute is included;

[0066] Determination module 306 is configured to determine, when the preset parameters are included, that the query value of the data processing statement is a preset value corresponding to the preset parameters. The data processing device provided in the embodiments of the present disclosure can execute the data processing method provided in any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects of the execution method. The implementation principles thereof are similar and will not be further described here.

[0067] In order to implement the above embodiments, the present disclosure further proposes a computer program product, including a computer program / instruction, which implements the data processing method in the above embodiments when executed by a processor.

[0068] In order to implement the above embodiments, the present disclosure also proposes an electronic device, which includes: a processor; a memory for storing processor-executable instructions; and a processor for reading executable instructions from the memory and executing the executable instructions to implement the above data processing method.

[0069] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0070] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0071] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A data processing method, characterized in that: The following steps are involved: Obtaining a data processing statement from a local database, and extracting a target function and target data parameters from the processing statement; Determining whether the target function is a local function corresponding to the local database; If it is not the local function, obtaining the function properties of the target function; Determining whether the function attributes include a preset first attribute; If the first attribute is included, determining whether the target data parameter includes a preset parameter corresponding to the first attribute; If the preset parameter is included, determining the query value of the data processing statement to be a preset value corresponding to the preset parameter; wherein, when the objective function including the first attribute performs data processing, the output value uniquely corresponds to the specific input value; After determining whether the function attributes include the preset first attribute, the method further includes: If the preset first attribute is not included, determining whether the function attributes include a preset second attribute; wherein the target function of the second attribute can obtain the same result execution value based on the same input target data parameter; If the second attribute is included, querying a preset query result database according to the target data parameter; wherein the preset query result database includes each external function of the second attribute and the execution value of each external function corresponding to different data parameters; If a matching value that matches the target data parameter is found, determining the query value of the data processing statement to be the matching value; After querying the preset query result database according to the target data parameters, the method further includes: If no matching value matching the target data parameter is found, determining an external database corresponding to the target function; Sending the data processing statement to the external database, and obtaining an execution result value fed back by the external database in response to the data processing statement; The corresponding relationship between the execution result value and the target data parameter is stored in the preset query result database.

2. The method according to claim 1, wherein After determining whether the function attributes include the preset second attribute, the method further includes: If the second attribute is not included, determining whether the database corresponding to the target data parameter is the local database; If it is the local database, obtaining the target data corresponding to the target data parameter in the local database; The target data and the data processing statement are sent to a database corresponding to the target function.

3. The method according to claim 1, wherein The obtaining of the function attribute of the objective function includes: Sending a property query request of the target function to a preset cloud server; Acquire the function attributes fed back by the cloud server in response to the attribute query request.

4. The method according to claim 1, wherein If the first attribute is a strict attribute, the preset parameter corresponding to the first attribute is a null value parameter.

5. The method according to claim 1, wherein Before obtaining the function attribute of the objective function, the method further includes: Get the preset external data source function list; Determine whether the target function belongs to the external data source function list; If it belongs to the external data source function list, obtaining the first parameter type of the target data parameter; Querying a preset second parameter type of the objective function; It is determined that the first parameter type and the second parameter type match.

6. A data processing device, characterized in that: include: An extraction module is used to obtain a data processing statement from a local database and extract a target function and target data parameters from the processing statement; A first judging module, configured to judge whether the target function is a local function corresponding to the local database; An acquisition module, configured to acquire function properties of the target function when the target function is not a local function; A second judging module is used to judge whether the function attributes include a preset first attribute; a third judgment module, configured to judge whether the target data parameter includes a preset parameter corresponding to the first attribute when the first attribute is included; a determination module, configured to determine, when the preset parameter is included, that the query value of the data processing statement is a preset value corresponding to the preset parameter; wherein, when the objective function including the first attribute performs data processing, the output value uniquely corresponds to the specific input value; The apparatus further includes a fourth determining module, configured to: If the preset first attribute is not included, determining whether the function attributes include a preset second attribute; wherein the target function of the second attribute can obtain the same result execution value based on the same input target data parameter; If the second attribute is included, querying a preset query result database according to the target data parameter; wherein the preset query result database includes each external function of the second attribute and the execution value of each external function corresponding to different data parameters; If a matching value that matches the target data parameter is found, determining the query value of the data processing statement to be the matching value; The apparatus further includes a fifth judgment module, configured to: If no matching value matching the target data parameter is found, determining an external database corresponding to the target function; Sending the data processing statement to the external database, and obtaining an execution result value fed back by the external database in response to the data processing statement; The corresponding relationship between the execution result value and the target data parameter is stored in the preset query result database.

7. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the data processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is used to execute the data processing method according to any one of claims 1 to 5.

Images