Data processing method and device, equipment and storage medium

By analyzing and judging the number of executions of SQL query statements, dynamically adjusting the concurrency to execute query tasks, solving the abnormal problems during SQL query, reducing the risk of cluster failure, and improving the stability of data query.

CN118939664APending Publication Date: 2024-11-12BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202310531824.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-11
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

When the existing database query language (SQL) obtains data through concurrent execution scans, it is easy to cause abnormal execution results, which in turn causes large-scale failures of the cluster machine.

Method used

By obtaining the query statement expression sent by the user, parsing and generating an execution plan, we judge whether the cumulative number of executions of the query statement is less than or equal to the preset number. If so, determine the first concurrency degree. If otherwise, determine the second concurrency degree, according to the corresponding Concurrency performs query tasks to reduce the occurrence of exceptions.

Benefits of technology

It effectively reduces the probability of large-scale failure of cluster machines, reduces abnormalities in execution results, and improves the stability of data query.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939664A_ABST
    Figure CN118939664A_ABST
Patent Text Reader

Abstract

The invention provides a data processing method and device, equipment and a storage medium, relates to the technical field of databases, in particular to the technical fields of data analysis, data query and the like, and can be applied to a scene of querying data by utilizing a database query language. According to the specific implementation scheme, the method comprises the steps of obtaining a query statement expression sent by a user, and analyzing the query statement expression; generating a corresponding execution plan according to the query statement expression; judging whether the accumulative execution frequency of the query statement expression is smaller than or equal to a preset frequency or not; if yes, determining a first concurrency degree, and executing the query task according to the execution plan and the first concurrency degree; if not, a second concurrency degree is determined, the query task is executed according to the execution plan and the second concurrency degree, and the first concurrency degree is smaller than the second concurrency degree. Abnormal conditions of execution results can be reduced, and the probability of large-area faults of cluster machines is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of database technology, specifically to technical fields such as data analysis and data query, and can be applied to scenarios where data is queried using a database query language, and in particular to a data processing method, apparatus, device and storage medium. Background Art

[0002] The database stores a large amount of data, and users can use the database query language to query the required data. The database query language may include Structured Query Language (SQL).

[0003] Currently, when using SQL to query data, the required data is obtained by concurrently executing scans.

[0004] However, the current method of obtaining required data through concurrent execution scanning is prone to abnormal execution results, leading to large-scale failures of cluster machines. Summary of the invention

[0005] The present disclosure provides a data processing method, apparatus, device and storage medium, which can reduce abnormal situations of execution results and reduce the probability of large-scale failure of cluster machines.

[0006] According to a first aspect of the present disclosure, a data processing method is provided, the method comprising:

[0007] Obtain the query statement expression sent by the user and parse the query statement expression; generate a corresponding execution plan according to the query statement expression; determine whether the cumulative number of executions of the query statement expression is less than or equal to the preset number; if so, determine the first concurrency, and execute the query task according to the execution plan and the first concurrency; if not, determine the second concurrency, and execute the query task according to the execution plan and the second concurrency, and the first concurrency is less than the second concurrency.

[0008] According to a second aspect of the present disclosure, a data processing device is provided, the device comprising: an acquisition unit, a generation unit, a judgment unit, and a determination unit.

[0009] An acquisition unit is used to acquire a query statement expression sent by a user and parse the query statement expression; a generation unit is used to generate a corresponding execution plan according to the query statement expression; a judgment unit is used to judge whether the cumulative number of executions of the query statement expression is less than or equal to a preset number; a determination unit is used to determine a first concurrency if yes, and execute the query task according to the execution plan and the first concurrency; the determination unit is also used to determine a second concurrency if no, and execute the query task according to the execution plan and the second concurrency, the first concurrency being less than the second concurrency.

[0010] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect.

[0011] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause a computer to execute the method according to the first aspect.

[0012] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method according to the first aspect when executed by a processor.

[0013] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0015] Figure 1 A flowchart of a data processing method provided by an embodiment of the present disclosure;

[0016] Figure 2 A schematic diagram of the composition of a data processing device provided in an embodiment of the present disclosure;

[0017] Figure 3 The present disclosure is provided as a schematic block diagram of an exemplary electronic device 300 that can be used to implement the present disclosure. DETAILED DESCRIPTION

[0018] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0019] It should be understood that in the embodiments of the present disclosure, the character " / " generally indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", etc. are used only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features.

[0020] The database stores a large amount of data, and users can use the database query language to query the required data. The database query language may include Structured Query Language (SQL).

[0021] Currently, when using SQL to query data, the required data is obtained by concurrently executing scans.

[0022] However, the current method of obtaining required data through concurrent execution scanning is prone to abnormal execution results, leading to large-scale failures of cluster machines.

[0023] For example, databases can be divided into relational databases and non-relational databases. Generally speaking, database query languages ​​vary according to different database types. Relational databases use a database query language called Structured Query Language (SQL) for querying. Currently, when using SQL to query data, the required data is obtained through concurrent execution scanning. Users are prone to misoperation of SQL, causing execution anomalies. Moreover, when SQL is abnormal, it is easy to cause abnormalities in large-scale scanning and updating of data, resulting in large-scale failures of cluster machines.

[0024] Against this background, the present disclosure provides a data processing method that can reduce abnormal situations in execution results and reduce the probability of large-scale failures of cluster machines.

[0025] Exemplarily, the execution subject of the data storage method may be a computer or a server, or may be other electronic devices with data processing capabilities. The execution subject of the method is not limited here.

[0026] In some embodiments, the server may be a single server, or a server cluster composed of multiple servers. In some implementations, the server cluster may also be a distributed cluster. The present disclosure does not limit the specific implementation of the server.

[0027] Figure 1 The following is a flow chart of a data processing method provided by an embodiment of the present disclosure. Figure 1 As shown, the method may include S101-S105.

[0028] S101: Obtain a query statement expression sent by a user, and parse the query statement expression.

[0029] Exemplarily, the query statement expression includes SQL, and the query statement expression sent by the user can be parsed into the required result.

[0030] S102: Generate a corresponding execution plan according to the query statement expression.

[0031] Exemplarily, a logical plan in a form corresponding to the query statement expression can be generated according to the parsing result of the query statement expression, and a physical execution plan can be generated in combination with the logical plan. Specifically, the relevant technology of generating an execution plan according to a query statement expression and querying data in a database according to the execution plan can be used to generate a corresponding execution plan according to the query statement expression, which is not limited here.

[0032] S103: Determine whether the cumulative execution times of the query statement expression is less than or equal to a preset number.

[0033] It should be noted that the cumulative number of executions refers to the total number of times a query statement expression is actually executed and called in the server.

[0034] Exemplarily, a value of the cumulative number of executions of a query statement expression may be preset, and then based on the preset value of the cumulative number of executions, it is determined whether the cumulative number of executions of the query statement expression is less than or equal to the preset number of executions. That is, when the cumulative number of executions of the query statement expression is less than or equal to the value, it is determined whether the cumulative number of executions of the query statement expression is less than or equal to the preset number of executions. That is, the value of the preset cumulative number of executions is the preset number of executions.

[0035] By setting a preset number, it is possible to determine whether the corresponding query statement is a new query statement expression according to whether the cumulative number of executions of the query statement expression is less than or equal to the preset number. That is, when the cumulative number of executions of the query statement expression is small (less than the preset number), the query statement expression is considered to be a new query statement expression.

[0036] Therefore, the specific value of the above-mentioned preset number of times can be set to a smaller value according to actual experience, so as to determine the new query statement expression. For example, the preset number of times can be 1, 3, etc., which is not limited here.

[0037] For example, assuming that the preset number is 1, when the cumulative execution times of a query statement expression is less than 1, it indicates that the query statement expression is a new query statement expression.

[0038] S104: If yes, determine a first concurrency, and execute the query task according to the execution plan and the first concurrency.

[0039] Exemplarily, when it is determined that the cumulative number of executions of the query statement expression is less than or equal to the preset number of times, the first concurrency can be determined, and the data query task is executed according to the execution plan and the first concurrency, wherein the concurrency can be understood as the number of tasks that can be executed in parallel when executing the data query task (i.e., the number of regions that can be scanned in parallel when executing the data query task). Among them, in other possible implementations of the present application, the query statement expression with a cumulative number of executions less than or equal to the preset number of times can also be called manual SQL.

[0040] Based on the above implementation, for example, assuming that the preset number is 2, query statement expressions with a cumulative execution number less than or equal to 2 times need to execute the data query task according to the execution plan and the first concurrency.

[0041] S105: If not, determine the second concurrency, and execute the query task according to the execution plan and the second concurrency, wherein the first concurrency is less than the second concurrency.

[0042] Exemplarily, when it is determined that the cumulative number of executions of a query statement expression is greater than a preset number, a second concurrency can be determined, and the data query task can be executed according to the execution plan and the second concurrency, wherein the first concurrency is less than the second concurrency, and the second concurrency can be a concurrency determined according to the scanning rules for scanning various regions in the database in parallel in the relevant technology, or can be a concurrency determined empirically according to the number of different regions in different units (stores) in the database. Optionally, in other possible implementations of the present application, the second concurrency can also be referred to as the default single-machine concurrency, and the query statement expression with a cumulative number of executions greater than the preset number can be referred to as non-manual SQL.

[0043] Based on the above implementation, for example, assuming that the preset number of times is 2, then the query statement expressions with a cumulative execution number greater than 2 times will execute the data query task according to the execution plan and the second concurrency. If a machine has X tasks distributed, the rule of the prior art is that each machine executes as many tasks in parallel as it has, then the second concurrency is X; if a machine has X tasks distributed, but the user believes based on experience that executing X tasks in parallel is inefficient or there is a risk of anomalies, the second concurrency can also be set to any value less than X and greater than the first concurrency.

[0044] The present disclosure obtains the query statement expression sent by the user and parses the query statement expression, so that the corresponding execution plan can be generated according to the query statement expression. So that the data query task can be executed according to the execution plan later, so as to query the data specified by the query statement expression. Among them, the method also determines whether the cumulative execution times of the query statement expression are less than or equal to the preset times, so as to determine the new query statement expression with less execution times according to the preset times and the cumulative execution times. Thus, when the cumulative execution times are less than or equal to the preset times, the query statement expression is determined to be a new query statement expression, and then the first concurrency can be determined, and the query task is executed according to the execution plan and the first concurrency. Correspondingly, when the cumulative execution times are greater than the preset times, it can be determined that the query statement expression is not a new query statement expression, so the second concurrency can be determined, and the query task is executed according to the execution plan and the second concurrency. Since the new query statement expression is used less times, the possibility of anomalies is high. Therefore, the method of the present disclosure can be used for the new query statement expression, and the query task can be executed with a low concurrency to reduce the amount of data affected when the new query statement expression has anomalies, thereby reducing the probability of large-scale failure of cluster machines.

[0045] In some embodiments, before generating a corresponding execution plan according to the query statement expression, the method may further include: determining whether the query statement expression complies with a preset rule.

[0046] Usually, for different databases, the format rules of the query statement expression when querying the corresponding data are pre-set, so that the database can parse the corresponding query statement expression according to the rules to generate an execution plan to query the corresponding data. Therefore, it is possible to detect whether the query statement expression meets the preset rules (i.e., the format rules mentioned above). When the query statement expression meets the preset rules, the subsequent steps are executed. When the query statement expression does not meet the preset rules, the query task can be terminated. Thus, the format of the query statement expression can be verified. When the query statement expression does not meet the preset rules, the probability of the query statement expression being abnormal will be very high. Therefore, it is necessary to terminate the query task to avoid abnormal data query tasks caused by query statement expressions that do not meet the preset rules.

[0047] Based on the above implementation, for example, assuming that before generating an execution plan, it is detected that a certain query statement expression meets the preset rules, then continue to determine whether the cumulative number of executions of the query statement expression is less than or equal to the preset number, and execute S104 or S105 according to the judgment result; if it is detected that a certain query statement expression does not meet the preset rules, then directly terminate the task, and it can be considered that the query task has failed.

[0048] The preset rules are query statement expression rules corresponding to database specifications, which can be set according to actual conditions.

[0049] This embodiment verifies the format of the query statement expression and screens out the query statement expressions that meet the preset rules. When the query statement expression meets the preset rules, the subsequent steps are executed, thereby avoiding the situation where the data query task is abnormal due to the query statement expression that does not meet the preset rules, and reducing the waste of resources for further operations on the query statement expressions that do not meet the preset rules.

[0050] In some embodiments, the above-mentioned preset rules may include: the query statement expression includes a Boolean expression.

[0051] Exemplarily, the query statement expression includes a conditional expression. When determining that the query statement expression meets the preset rules, it is necessary to analyze the left and right nodes of the conditional expression node of the query statement expression. When the left and right nodes of each conditional expression node are Boolean expressions, it can be determined that the query statement expression meets the preset rules; when the left and right nodes of the conditional expression node of a query statement expression have non-Boolean expressions, it is considered that the query statement expression does not meet the preset rules, and the query task can be terminated at this time. Among them, the Boolean expression can be a code statement, which ultimately has only two values: true and false.

[0052] Based on the above implementation, for example, assuming that you need to determine whether a query statement expression complies with the preset rules, you can first find the predicates contained in the query statement expression (such as AND, OR, NOT), and determine whether the left and right nodes of each predicate are both Boolean expressions. If so, then the query statement expression complies with the preset rules. If the left and right nodes of a predicate contain non-Boolean expressions, then the query statement expression does not comply with the preset rules.

[0053] This embodiment improves the accuracy of verifying query statement expressions by limiting the preset rules to include query statement expressions including Boolean expressions, thereby further reducing the probability of large-scale machine cluster failures caused by abnormal query statement expressions.

[0054] In some embodiments, when the above method is applied to an electronic device, before determining whether the cumulative number of executions of the query statement expression is less than or equal to a preset number of times, the method may also include: after the electronic device is restarted, determining that the current time is after a preset time after the electronic device is restarted.

[0055] Exemplarily, when the above method is applied to an electronic device, a time length can be set in advance. After the electronic device is restarted, it is determined whether the current time is after the preset time length. If the current time is after the preset time length, the step of determining whether the cumulative number of executions of the queried statement expression is less than or equal to the preset number is executed.

[0056] Based on the above implementation, for example, assuming that the preset time length is 1 minute, then when an electronic device is restarted, it is determined whether the current time is after 1 minute. If it is determined that the current time is after 1 minute, the step of determining whether the cumulative number of executions of the query statement expression is less than or equal to the preset number is executed.

[0057] This embodiment ensures that the electronic device operates normally after restarting, thereby reducing the probability of electronic device failure, by determining whether the current time is after a preset time length after the electronic device is restarted, and executing a step of determining whether the cumulative number of executions of the queried statement expression is less than or equal to a preset number when the current time is after the preset time length.

[0058] In some embodiments, the method may further include: after the electronic device is restarted, determining that the current time is within a preset time length after the electronic device is restarted, and executing the query task according to the execution plan and the second concurrency.

[0059] Exemplarily, when the above method is applied to an electronic device, a time length can be set in advance. After the electronic device is restarted, it is determined whether the current time is after the preset time length. If it is determined that the current time is within the preset time length, the query task is executed according to the execution plan and the second concurrency.

[0060] Based on the above implementation, for example, assuming that the preset duration is 1 minute, then when an electronic device is restarted, it is determined whether the current time is after 1 minute. If it is determined that the current time is within 1 minute, the query statement expression is not judged or restricted, and the query task is executed directly according to the execution plan and the second concurrency.

[0061] This embodiment determines whether the current time is after the preset time after the electronic device is restarted. When the current time is within the preset time, the query task is executed according to the execution plan and the second concurrency, thereby ensuring that the electronic device can run normally after restarting and maintaining the stability of the operation of the electronic device.

[0062] In some embodiments, the first concurrency is 1.

[0063] Exemplarily, when the cumulative number of executions of the query statement expression is less than or equal to the preset number of times, the data query task can be executed according to the execution plan and the first concurrency. The first concurrency is 1, which means that when executing the data query task, only one area is scanned each time, that is, only one copy of data is queried each time. After querying this copy of data, if this copy of data is the data required by the user, the query result is returned. If this copy of data is not the data required by the user, the next copy of data is queried until the task is completed.

[0064] Therefore, when it is determined that the query statement expression is a new query statement expression, the execution plan generated according to the query statement expression is limited to perform data query, and the data in each area is scanned in a serial manner, thereby reducing the amount of data affected by the query statement expression being abnormal, and avoiding large-scale failures of cluster machines caused by data query anomalies.

[0065] In an exemplary embodiment, the present disclosure also provides a data processing device that can be used to implement the above Figure 1 The data processing method in the embodiment shown.

[0066] Figure 2 The following is a schematic diagram of the composition of a data processing device provided in an embodiment of the present disclosure. Figure 2 As shown, the device may include: an acquisition unit 201, a generation unit 202, a judgment unit 203, and a determination unit 204.

[0067] The acquisition unit 201 is used to acquire the query statement expression sent by the user and parse the query statement expression.

[0068] The generating unit 202 is used to generate a corresponding execution plan according to the query statement expression.

[0069] The judging unit 203 is used to judge whether the accumulated execution times of the query statement expression is less than or equal to a preset number.

[0070] The determining unit 204 is configured to determine a first concurrency, and execute the query task according to the execution plan and the first concurrency.

[0071] The determining unit 204 is further configured to determine a second concurrency if no, and execute the query task according to the execution plan and the second concurrency, wherein the first concurrency is less than the second concurrency.

[0072] Optionally, the determining unit 204 is further configured to determine whether the query statement expression complies with a preset rule.

[0073] Optionally, the preset rule includes: the query statement expression includes a Boolean expression.

[0074] Optionally, the determination unit 204 is further configured to determine, after the electronic device is restarted, that the current time is after a preset time period after the electronic device is restarted.

[0075] Optionally, the determination unit 204 is further configured to: after the electronic device is restarted, determine that the current time is within a preset time period after the electronic device is restarted, and then execute the query task according to the execution plan and the second concurrency.

[0076] Optionally, the first concurrency is 1.

[0077] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0078] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0079] In an exemplary embodiment, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the above embodiments.

[0080] In an exemplary embodiment, the readable storage medium may be a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method according to the above embodiments.

[0081] In an exemplary embodiment, a computer program product includes a computer program, and when the computer program is executed by a processor, the method according to the above embodiments is implemented.

[0082] Figure 3 A schematic block diagram of an example electronic device 300 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0083] like Figure 3As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0084] A number of components in the electronic device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0085] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 301 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform the data processing method in any other appropriate manner (e.g., by means of firmware).

[0086] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0087] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0088] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0089] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0090] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0091] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0092] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0093] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A data processing method, the method include: Obtaining a query statement expression sent by a user, and parsing the query statement expression; Generate a corresponding execution plan according to the query statement expression; Determine whether the cumulative execution times of the query statement expression is less than or equal to a preset number; If yes, determine a first concurrency, and execute the query task according to the execution plan and the first concurrency; If not, a second concurrency is determined, and the query task is executed according to the execution plan and the second concurrency, wherein the first concurrency is less than the second concurrency.

2. The method according to claim 1, before generating a corresponding execution plan according to the query statement expression, the method further include: Determine whether the query statement expression complies with preset rules.

3. The method according to claim 2, wherein the preset rule: include: The query statement expression includes a Boolean expression.

4. The method according to any one of claims 1 to 3, applied to an electronic device, before determining whether the cumulative number of executions of the query statement expression is less than or equal to a preset number of times, the method further include: After the electronic device is restarted, it is determined that the current time is after a preset time period after the electronic device is restarted.

5. The method according to claim 4, further comprising: include: After the electronic device is restarted, it is determined that the current time is within a preset time period after the electronic device is restarted, and the query task is executed according to the execution plan and the second concurrency.

6. According to the method according to any one of claims 1-5, the first concurrency is 1.

7. A data processing device, the device include: An acquisition unit, used to acquire a query statement expression sent by a user and parse the query statement expression; A generating unit, used for generating a corresponding execution plan according to the query statement expression; A judging unit, used to judge whether the cumulative number of executions of the query statement expression is less than or equal to a preset number; a determining unit, configured to determine a first concurrency, and execute the query task according to the execution plan and the first concurrency; The determining unit is further configured to, if not, determine a second concurrency, and execute the query task according to the execution plan and the second concurrency, wherein the first concurrency is less than the second concurrency.

8. The apparatus according to claim 7, wherein the determining unit is further configured to: Determine whether the query statement expression complies with preset rules.

9. The device according to claim 8, wherein the preset rule: include: The query statement expression includes a Boolean expression.

10. The apparatus according to any one of claims 7 to 9, wherein the determining unit is further configured to: After the electronic device is restarted, it is determined that the current time is after a preset time period after the electronic device is restarted.

11. The apparatus according to claim 10, wherein the determining unit is further configured to: After the electronic device is restarted, it is determined that the current time is within a preset time period after the electronic device is restarted, and the query task is executed according to the execution plan and the second concurrency.

12. The device according to claims 7-11, wherein the first concurrency is 1.

13. An electronic device, include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method according to any one of claims 1 to 6.

15. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • SQL statement execution frequency dynamic regulation method

    CN104933190A

  • Data query method, device and system based on structured query language

    CN109145015A

  • Service preheating method, preheating device and service system

    CN114519582A

  • Predicting execution times of concurrent queries

    US20160203404A1