Data query method and device, electronic equipment and computer readable medium

Abstract

The invention discloses a data query method and device, electronic equipment and a computer readable medium, and relates to the technical field of computers.The specific implementation mode comprises the steps that a data query request is received, and a corresponding user identifier and a corresponding scene identifier are obtained; determining a corresponding query instance and a sub-scheduler identifier according to the user identifier and the scene identifier; calling a sub-scheduling method corresponding to the sub-scheduler identifier to determine a corresponding resource group according to the user identifier; and based on the resource group, executing the query instance and returning corresponding query result data. Through resource isolation, the resource utilization rate is improved, and the data query rate and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data query method, device, electronic device and computer-readable medium. Background Art

[0002] The existing data service analysis platform is very inflexible when using data and cannot cope with large amounts of user data and frequent changes. When executing queries, unreasonable queries occupy the vast majority of computing resources. Although resource groups are divided, some queries that read a small number of rows and occupy little resources have a running time that is much longer than the running time when the cluster is idle because there is no resource tilt, and even normal queries cannot be executed smoothly. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a data query method, device, electronic device and computer-readable medium, which can solve the existing problem of low data query efficiency when the amount of user data is large and changes frequently.

[0004] To achieve the above object, according to one aspect of an embodiment of the present application, a data query method is provided, comprising:

[0005] Receive data query request and obtain corresponding user ID and scene ID;

[0006] Determine the corresponding query instance and sub-scheduler identifier according to the user identifier and the scenario identifier;

[0007] Calling the sub-scheduling method corresponding to the sub-scheduler identifier to determine the corresponding resource group according to the user identifier;

[0008] Based on the resource group, execute the query instance and return the corresponding query result data.

[0009] Optionally, determining a corresponding query instance and a sub-scheduler identifier according to the user identifier and the scenario identifier includes:

[0010] Determine the corresponding user group according to the user ID and the scenario ID;

[0011] Matching the user group with each preset user group in the preset user group set, and in response to a matching failure, obtaining demand data and resource quota data corresponding to the data query request;

[0012] Create target user groups and target resource groups based on demand data and resource quota data;

[0013] According to the target user group and target resource group, a corresponding sub-scheduler is created and a corresponding sub-scheduler identifier is generated;

[0014] According to the user identifier and the scenario identifier, the corresponding operation data and machine data are determined, and then a query instance is generated according to the operation data and the machine data.

[0015] Optionally, determining a corresponding user group according to the user identifier and the scenario identifier includes:

[0016] Determine the corresponding data permissions based on the user ID and scenario ID;

[0017] Determine the corresponding user group based on data permissions.

[0018] Optionally, execute a query instance, including:

[0019] Get the resource usage corresponding to the query instance;

[0020] Based on resource usage, query limit operations are performed.

[0021] Optionally, query restriction operations are performed based on resource usage, including:

[0022] When the resource usage is greater than the preset threshold, the corresponding query instance is blacklisted and restricted.

[0023] Optionally, determine a corresponding resource group, including:

[0024] Determine the corresponding resource queue according to the user ID;

[0025] The resource occupancy rate corresponding to each candidate resource group in the resource queue is obtained, and then the candidate resource group corresponding to the lowest resource occupancy rate is determined as the resource group corresponding to the user identifier.

[0026] Optionally, determining a corresponding query instance and a sub-scheduler identifier according to the user identifier and the scenario identifier includes:

[0027] Determine the number of user identifiers, and in response to the number being multiple, obtain priorities corresponding to the user identifiers, and then determine the target user identifier according to the priorities;

[0028] According to the target user identifier and the scenario identifier, the corresponding query instance and sub-scheduler identifier are determined.

[0029] In addition, the present application also provides a data query device, including:

[0030] A receiving unit is configured to receive a data query request and obtain a corresponding user identifier and a scene identifier;

[0031] A determination unit configured to determine a corresponding query instance and a sub-scheduler identifier according to the user identifier and the scenario identifier;

[0032] A resource group determination unit is configured to call a sub-scheduling method corresponding to the sub-scheduler identifier to determine a corresponding resource group according to the user identifier;

[0033] The execution unit is configured to execute the query instance based on the resource group and return the corresponding query result data.

[0034] Optionally, the determining unit is further configured to:

[0035] Determine the corresponding user group according to the user ID and the scenario ID;

[0036] Matching the user group with each preset user group in the preset user group set, and in response to a matching failure, obtaining demand data and resource quota data corresponding to the data query request;

[0037] Create target user groups and target resource groups based on demand data and resource quota data;

[0038] According to the target user group and target resource group, a corresponding sub-scheduler is created and a corresponding sub-scheduler identifier is generated;

[0039] According to the user identifier and the scenario identifier, the corresponding operation data and machine data are determined, and then a query instance is generated according to the operation data and the machine data.

[0040] Optionally, the determining unit is further configured to:

[0041] Determine the corresponding data permissions based on the user ID and scenario ID;

[0042] Determine the corresponding user group based on data permissions.

[0043] Optionally, the execution unit is further configured to:

[0044] Get the resource usage corresponding to the query instance;

[0045] Based on resource usage, query limit operations are performed.

[0046] Optionally, the execution unit is further configured to:

[0047] When the resource usage is greater than the preset threshold, the corresponding query instance is blacklisted and restricted.

[0048] Optionally, the resource group determining unit is further configured to:

[0049] Determine the corresponding resource queue according to the user ID;

[0050] The resource occupancy rate corresponding to each candidate resource group in the resource queue is obtained, and then the candidate resource group corresponding to the lowest resource occupancy rate is determined as the resource group corresponding to the user identifier.

[0051] Optionally, the determining unit is further configured to:

[0052] Determine the number of user identifiers, and in response to the number being multiple, obtain priorities corresponding to the user identifiers, and then determine the target user identifier according to the priorities;

[0053] According to the target user identifier and the scenario identifier, the corresponding query instance and sub-scheduler identifier are determined.

[0054] In addition, the present application also provides a data query electronic device, including: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by one or more processors, the one or more processors implement the data query method as described above.

[0055] In addition, the present application also provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the data query method as described above is implemented.

[0056] One embodiment of the above invention has the following advantages or beneficial effects: the application receives a data query request to obtain a corresponding user identifier and a scenario identifier; determines a corresponding query instance and a sub-scheduler identifier according to the user identifier and the scenario identifier; calls a sub-scheduling method corresponding to the sub-scheduler identifier to determine a corresponding resource group according to the user identifier; executes the query instance based on the resource group and returns the corresponding query result data. Through resource isolation, resource utilization is improved, and data query rate and accuracy are improved.

[0057] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific implementation examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings are used to better understand the present application and do not constitute an improper limitation on the present application.

[0059] Figure 1 It is a schematic diagram of the main process of the data query method provided according to an embodiment of the present application;

[0060] Figure 2 It is a schematic diagram of the main process of the data query method provided according to an embodiment of the present application;

[0061] Figure 3 is a schematic diagram of the system structure of a data query method provided according to an embodiment of the present application;

[0062] Figure 4 This is a schematic diagram of a user change process of a data query method provided according to an embodiment of the present application;

[0063] Figure 5 It is a query execution flow diagram of a data query method provided according to an embodiment of the present application;

[0064] Figure 6 is a schematic diagram of main units of a data query device according to an embodiment of the present application;

[0065] Figure 7 is an exemplary system architecture diagram to which the embodiments of the present application can be applied;

[0066] Figure 8 It is a structural diagram of a computer system of a terminal device or server suitable for implementing an embodiment of the present application. DETAILED DESCRIPTION

[0067] The following is an explanation of the exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description. It should be noted that in the technical solution of the present disclosure, the collection, collection, update, analysis, processing, use, transmission, storage and other aspects of user personal information involved are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Take necessary measures for user personal information to prevent illegal access to user personal information data and maintain user personal information security, network security and national security.

[0068] Figure 1 is a schematic diagram of the main process of a data query method provided according to an embodiment of the present application, such as Figure 1 As shown, the data query method includes:

[0069] Step S101, receiving a data query request, and obtaining a corresponding user identifier and scene identifier.

[0070] In this embodiment, the execution subject of the data query method (for example, it can be a server) can receive the data query request through a wired connection or a wireless connection. After receiving the data query request, the execution subject can obtain the user identifier and the scene identifier carried in the request. Among them, the user identifier can be A, B, C, D, E, or 1, 2, 3. The embodiment of the present application does not specifically limit the user identifier. Different user identifiers can represent different accessible data permissions. The scene identifier is used to characterize the scene when the data query request is initiated. The scene may include express lines and express transportation lines, etc. The embodiment of the present application does not specifically limit the scene. For example, it can be a data query request initiated on an express line or a data query request initiated on an express transportation line.

[0071] Step S102: Determine the corresponding query instance and sub-scheduler identifier according to the user identifier and the scenario identifier.

[0072] The execution subject can determine the CPU, memory and other resources that can be used for querying data according to the user identifier and scenario identifier, and then determine the corresponding query instance and sub-scheduler identifier according to the available CPU, memory and other resources and the user identifier.

[0073] In some embodiments, a corresponding query instance and a sub-scheduler identifier are determined based on a user identifier and a scenario identifier, including: determining a corresponding user group based on the user identifier and the scenario identifier; matching the user group with each preset user group in a preset user group set, and in response to a matching failure, obtaining demand data and resource quota data corresponding to the data query request; creating a target user group and a target resource group based on the demand data and the resource quota data; creating a corresponding sub-scheduler and generating a corresponding sub-scheduler identifier based on the target user group and the target resource group; determining corresponding operation data and machine data based on the user identifier and the scenario identifier, and then generating a query instance based on the operation data and the machine data.

[0074] When a new user accesses the system, the user can view the current user groups through user group management to see whether there is a group that meets the user requirements corresponding to the user ID. When all current user groups cannot meet the user's requirements, that is, when the match fails, the user can also apply for a new user group. When applying for a new user group, the demand data and the resource quota data applied for must be obtained. After the application for a new user group is successfully submitted, the system administrator will review whether the scenario and resource application are reasonable. After approval, the execution entity will create the corresponding user group and the corresponding resource group at the resource management layer, and authorize them to the corresponding user group. At the same time, the corresponding sub-scheduler is created at the scheduling layer, the corresponding sub-scheduler ID is generated, and the applicant user is set as the administrator of the new user group. According to the user ID and scenario ID, determine the operations to be performed and which machines need to perform these operations, and then generate the corresponding query instance.

[0075] Specifically, according to the user identifier and the scene identifier, the corresponding query instance and sub-scheduler identifier are determined, including: determining the number of user identifiers, and in response to the number being multiple, obtaining the priority corresponding to the user identifier, and then determining the target user identifier according to the priority; according to the target user identifier and the scene identifier, determining the corresponding query instance and sub-scheduler identifier.

[0076] When there are multiple users submitting data query requests, priorities can be assigned to multiple different users to prioritize the execution of queries from users with high priorities. The user identifier corresponding to the user with high priority (e.g., the user ranked first or second in priority, and the embodiment of the present application does not specifically limit the user with high priority) is determined as the target user identifier.

[0077] Specifically, determining a corresponding user group according to the user identifier and the scenario identifier includes: determining a corresponding data permission according to the user identifier and the scenario identifier; and determining a corresponding user group according to the data permission.

[0078] Data permissions, such as access permissions for express delivery line-related data, access permissions for express transportation line-related data, etc., are not specifically limited in the present application embodiment. Users are divided into different user groups according to data permissions, and users in each user group have the same data access permissions.

[0079] Step S103: calling a sub-scheduling method corresponding to the sub-scheduler identifier to determine a corresponding resource group according to the user identifier.

[0080] Specifically, determining the corresponding resource group includes: determining the corresponding resource queue according to the user identifier; obtaining the resource occupancy rate corresponding to each candidate resource group in the resource queue, and then determining the candidate resource group corresponding to the lowest resource occupancy rate as the resource group corresponding to the user identifier.

[0081] The corresponding data permissions can be determined according to the user ID, and the corresponding accessible resource queue can be determined according to the determined data permissions. Each resource queue can be divided into various sub-queues, and the sub-queues can be, for example, various candidate resource groups. The resource occupancy rate of each candidate resource group corresponding to the corresponding accessible resource queue is obtained, and the candidate resource group corresponding to the lowest resource occupancy rate is determined as the resource group corresponding to the user ID.

[0082] Step S104: Based on the resource group, execute the query instance and return the corresponding query result data.

[0083] The data query is executed in the determined resource group. Specifically, the query instance can be executed in the determined resource group and the obtained query result data can be returned.

[0084] This embodiment receives a data query request, obtains the corresponding user ID and scenario ID; determines the corresponding query instance and sub-scheduler ID according to the user ID and scenario ID; calls the sub-scheduling method corresponding to the sub-scheduler ID to determine the corresponding resource group according to the user ID; executes the query instance based on the resource group and returns the corresponding query result data. Through resource isolation, resource utilization is improved, and data query speed and accuracy are improved.

[0085] Figure 2 is a schematic diagram of the main flow of a data query method provided according to an embodiment of the present application, such as Figure 2 As shown, the data query method includes:

[0086] Step S201, receiving a data query request, and obtaining a corresponding user identifier and scenario identifier.

[0087] The data query request may be a request to query the express delivery line data or a request to query the express transportation line data.

[0088] The user identifier can be a number corresponding to the user at the application level, such as A, B, C, D, E, or a number corresponding to the user at the engine level, such as 1, 2, 3. The scene identifier can be KD, meaning the express line, or KY, meaning the express line. The embodiment of the present application does not specifically limit the scene identifier.

[0089] Step S202: Determine the corresponding query instance and sub-scheduler identifier according to the user identifier and the scenario identifier.

[0090] According to the user ID and scenario ID, the operations to be performed and the machines to be performed are determined, thereby generating the corresponding query instance. When the user ID has a corresponding existing user group, the corresponding sub-scheduler ID can be determined according to the sub-scheduler corresponding to the existing user group.

[0091] Step S203: calling the sub-scheduling method corresponding to the sub-scheduler identifier to determine the corresponding resource group according to the user identifier.

[0092] Call the sub-scheduling method corresponding to the sub-scheduler identifier to determine the query type corresponding to the data query request, and determine the corresponding resource group based on the query type and user identifier. Query types can include short queries, normal queries, and slow queries. Normal queries are allowed to share resources with each other when idle, while short queries are hard occupation of resources, and slow queries will take corresponding measures to limit them. The corresponding user group can be determined based on the user identifier, the corresponding engine-level user can be determined based on the user group, and the corresponding resource group can be determined based on the query type and the user identifier of the engine-level user.

[0093] Step S204: based on the resource group, obtain the resource occupancy rate corresponding to the query instance.

[0094] Step S205: performing a query restriction operation based on the resource occupancy rate.

[0095] Specifically, based on the resource usage, query restriction operations are performed, including:

[0096] When the resource usage is greater than the preset threshold, the corresponding query instance is blacklisted and restricted.

[0097] The execution entity can collect statistics on query execution information during the query process. It can set query duration thresholds, CPU time thresholds, memory thresholds, etc. to blacklist certain SQL statements with high resource usage to prevent them from occupying too many cluster resources.

[0098] Step S206, returning corresponding query result data.

[0099] The embodiments of the present application can improve resource utilization and increase data query speed and accuracy through resource isolation.

[0100] Figure 3 1 is a schematic diagram of an application scenario of a data query method provided according to an embodiment of the present application. The data query method of the embodiment of the present application is applied to a data query scenario in which the amount of user data is large and changes frequently. Figure 3As shown in the figure, the execution subject that the data query method relies on can be divided into five layers, namely, the application layer, the permission layer, the scheduling layer, the resource management layer, and the execution layer from top to bottom. The application layer is the top-level application, which submits various query requests under the restriction of the permission layer. At the same time, it does not need to pay attention to the implementation details of the lower layer, and the method can also be adapted to various applications that need to allocate query resources to cope with different scenarios. The permission layer is a mapping of the underlying resource management layer, which solves the problem that traditional resource management methods need to frequently perform risky operations at the database level. When a new user joins the application, there is generally a user group to which the user belongs in the business. At this time, the user is operated at the upper layer and added to the corresponding user group. In the case of frequent user changes, this operation is very lightweight and will not cause any risk to the underlying engine, thereby avoiding dangerous operations on the entire cluster. In order to prevent a user from frequently submitting queries or submitting unreasonable large queries, which affects the query of the entire group, a scheduling layer is set up. It can be understood as a set of schedulers composed of multiple tiny schedulers. The execution logic of a sub-scheduler is as follows: First, the resources are divided into different resource queues, and sub-queues can be divided under the resource queues. When a user submits a query, the depth-first algorithm is used to prioritize the queues with low resource occupancy rates for resource allocation. In more detail, different users can be prioritized to ensure the execution of queries of users with high priority. In addition, the engine will count the query execution information during the query process. The administrator can set the query duration threshold, CPU time threshold, memory threshold, etc., so as to blacklist and restrict certain SQLs with high resource occupancy rates to prevent them from occupying too many cluster resources. The resource layer is a hard limit on resources at the engine level. The method is to create different users on the engine to map different user groups at the upper level. At the same time, query types are divided into short queries, ordinary queries, and slow queries. Ordinary queries are allowed to share resources with each other when idle, while short queries are hard occupation of resources. Slow queries will take corresponding measures to limit them. For example, there are four resource groups α, β, γ, and δ, and γ is the short query resource group. On a 16-core machine, 6 / 4 / 4 / 2 resources are allocated respectively. When β is idle, α resources = (16-4)*6 / (6+2). In the actual execution process, the resource isolation operation actually takes effect when the physical execution plan is generated, that is, before the actual execution, the engine has already decided which query instances will be executed on which machines, thereby avoiding the possibility of mutual encroachment on resources. The execution layer is responsible for executing the actual physical execution plan to execute the query, and during the query process, it records the resource usage of the query and reports it to the upper layer. Resource groups are different resource quotas divided according to the different resource requirements of users (here refers to users 1, 2, and 3). Resource groups α and β may have differences in resource quotas. Resource group γ is a resource group dedicated to short queries. Its resources are highly independent and will not be encroached by queries from other groups.In the logistics scenario, A, B, C, and D can refer to users of different business lines. In essence, they are data divided according to different data permissions. For example, the express delivery line cannot see the data of the express delivery line. Users A, B, and C refer to users at the application level. They may be specific users under a certain line, such as Zhang San and Li Si. They are assigned to different user groups. The user group can be an express user group (such as user group 1 or user group 2 or user group 3). This group has the query permission for express delivery data. Users 1, 2, and 3 refer to users at the engine level, such as root users and jack users. They can be simply understood as having different resource quotas. User A is not mapped to user 1 because the modification of user permissions at the engine level is relatively heavy. It can grant some operation permissions on tables and libraries. It can even grant permissions to add and delete cluster machines, but ordinary users cannot master this knowledge and may not even use SQL queries. User A can be understood as front-line employees in logistics. There are a large number of them and they may change frequently. New employees join and employees leave. It is impossible to delete and create users frequently at the engine level. By mapping it to an application-level application, it can be frequently and easily removed and moved into user groups, and the administrator of the corresponding user group can perform related operations.

[0101] Figure 4 This is a schematic diagram of the user change process of the data query method provided in an embodiment of the present application. The user change scenario is illustrated by the operation of adding a new user. Since the deletion operation is relatively simple, that is, removing the user from the corresponding group, it will not be repeated here. The process of new user access in the embodiment of the present application is as follows:

[0102] When a new user accesses the system, the new user can view the existing user groups through the user group management to see if there is a group that meets their needs. If there is a user group that meets the user's needs, the new user can apply to join the corresponding user group, and the execution subject will send the approval process to the administrator of the corresponding group. The group administrator is responsible for managing the permissions within the group, adding and deleting users within the group, and can apply to the system administrator for resource quota modification, resource type modification, blacklist addition and lifting, and other related operations. After the group administrator approves and agrees, the new user can join the current user group and share the resources of the current group.

[0103] When all current user groups cannot meet the needs of new users, the new user can also apply for a new user group. When applying for a new user group, the requirements and resource quotas applied for must be described in detail and submitted. After successful submission, the system administrator will review whether the scenario and resource application are reasonable. After approval, the execution entity will create the corresponding user group and the corresponding resource group at the resource management layer, and grant the management rights of the user group to the new user. At the same time, the corresponding sub-scheduler is created at the scheduling layer, and the applicant user is set as the administrator of the group.

[0104] Figure 5 FIG. 1 is a flowchart of a query execution method according to an embodiment of the present application. Figure 5 The execution process of data query is shown in the figure, which can be divided into three stages: submission execution, scheduling execution, and actual execution:

[0105] Submit for execution: User A submits a query request in user group 1, and the request is then sent to the query scheduler. The execution subject can call the query scheduler to verify the SQL submitted by the user to see if it is a disabled and unreasonable SQL. If it is a disabled SQL, it will directly return a failure to the user and give a reasonable prompt. Otherwise, the legal SQL will be distributed to the sub-scheduler corresponding to the user group under the planning of the query scheduler. The sub-scheduler has multiple resource queues, so that it can limit the query requests submitted by a single user. The sub-scheduler will use the depth-first algorithm to traverse the resource queues, and select the queue with the lowest resource occupancy rate to allocate resources. Thus, the query enters the stage of waiting for scheduling execution. The sub-scheduler here can be flexibly selected according to different business scenarios or query execution characteristics, and even different sub-scheduling algorithms can be set for different user groups. Scheduling execution: The query waits in the queue until it is scheduled. The execution subject will find the corresponding user at the engine level according to the user group corresponding to the user who submitted the query, and set the corresponding resource group for the query. Only at this point will the data query request be actually submitted to the OLAP engine for execution. Actual execution: The query goes through the first two processes and is actually submitted to the OLAP engine for execution. Before being executed, a SQL statement goes through lexical analysis, syntax analysis, logical plan, physical plan, and distributed physical plan stages, and then it is submitted to the distributed cluster for execution. By classifying different submitting users, the execution engine can obtain the resource quota of this SQL statement. After lexical analysis and syntax analysis, the SQL statement will be parsed into an abstract syntax tree. Lexical analysis can identify the SQL string as a token, and syntax analysis can convert the token into an abstract syntax tree. The logical plan can convert the abstract syntax tree into an operator tree. Each node in the tree represents a way of calculating data, and the entire tree represents the calculation logic and flow direction of the data. The process related to resource management is to generate a physical plan. The physical plan will determine which machines to perform which operations based on the distribution of machines and data. When generating a query instance, the engine will determine the CPU, memory and other resources that the query instance can use based on the resource quota, thereby achieving mutual isolation between resources. Finally, the query is assigned to different machines for execution and returns the results. If the execution status of this statement exceeds the preset blacklist limit, the statement will be added to the blacklist to limit the next execution. Reasonable resource scheduling is achieved, resource conflicts are reduced, and the query speed is improved overall. In the context of a fixed user group and relatively unstable users, and users with low SQL levels who cannot be tuned. Through the data query method of the embodiment of the present application, a permission management and scheduling execution system is built on the upper layer of the query engine to avoid frequent risk operations of creating users and authorizations at the engine layer, and to classify users in different scenarios to avoid mutual influence between them.Solve the problem that the current engine only divides resources but does not schedule them, distinguish different types of queries, and restrict unreasonable queries, improve resource utilization and query efficiency, thereby improving user experience.

[0106] Figure 6 Schematic diagram of the main units of the data query device according to an embodiment of the present application. Figure 6 As shown, the data query device 600 includes a receiving unit 601 , a determining unit 602 , a resource group determining unit 603 and an executing unit 604 .

[0107] The receiving unit 601 is configured to receive a data query request and obtain a corresponding user identifier and a scene identifier.

[0108] The determination unit 602 is configured to determine a corresponding query instance and a sub-scheduler identifier according to the user identifier and the scenario identifier.

[0109] The resource group determining unit 603 is configured to call a sub-scheduling method corresponding to the sub-scheduler identifier to determine a corresponding resource group according to the user identifier.

[0110] The execution unit 604 is configured to execute the query instance based on the resource group and return the corresponding query result data.

[0111] In some embodiments, the determination unit 602 is further configured to: determine the corresponding user group based on the user identifier and the scenario identifier; match the user group with each preset user group in the preset user group set, and in response to a matching failure, obtain the demand data and resource quota data corresponding to the data query request; create a target user group and a target resource group based on the demand data and the resource quota data; create a corresponding sub-scheduler and generate a corresponding sub-scheduler identifier based on the target user group and the target resource group; determine the corresponding operation data and machine data based on the user identifier and the scenario identifier, and then generate a query instance based on the operation data and the machine data.

[0112] In some embodiments, the determination unit 602 is further configured to: determine the corresponding data authority according to the user identification and the scenario identification; and determine the corresponding user group according to the data authority.

[0113] In some embodiments, the execution unit 604 is further configured to: obtain a resource occupancy rate corresponding to the query instance; and perform a query restriction operation based on the resource occupancy rate.

[0114] In some embodiments, the execution unit 604 is further configured to: perform a blacklist restriction operation on the corresponding query instance when the resource occupancy rate is greater than a preset threshold.

[0115] In some embodiments, the resource group determination unit 603 is further configured to: determine the corresponding resource queue according to the user identifier; obtain the resource occupancy rate corresponding to each candidate resource group in the resource queue, and then determine the candidate resource group corresponding to the lowest resource occupancy rate as the resource group corresponding to the user identifier.

[0116] In some embodiments, the determination unit 602 is further configured to: determine the number of user identifiers, and in response to the number being multiple, obtain the priority corresponding to the user identifier, and then determine the target user identifier based on the priority; determine the corresponding query instance and sub-scheduler identifier based on the target user identifier and the scene identifier.

[0117] It should be noted that the data query method and the data query device of the present application have a corresponding relationship in terms of specific implementation contents, so the repeated contents will not be described again.

[0118] Figure 7 An exemplary system architecture 700 to which the data query method or data query device according to the embodiment of the present application can be applied is shown.

[0119] like Figure 7 As shown, system architecture 700 may include terminal devices 701, 702, 703, network 704 and server 705. Network 704 is used to provide a medium for communication links between terminal devices 701, 702, 703 and server 705. Network 704 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0120] Users can use terminal devices 701, 702, and 703 to interact with server 705 through network 704 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 701, 702, and 703, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0121] The terminal devices 701 , 702 , and 703 may be various electronic devices having a data query processing screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0122] Server 705 may be a server that provides various services, such as a background management server that provides support for data query requests submitted by users using terminal devices 701, 702, and 703 (only as an example). The background management server may receive a data query request, obtain a corresponding user identifier and a scene identifier; determine a corresponding query instance and a sub-scheduler identifier according to the user identifier and the scene identifier; call a sub-scheduler identifier corresponding to a sub-scheduler identifier to determine a corresponding resource group according to the user identifier; execute a query instance based on the resource group and return the corresponding query result data. Through resource isolation, resource utilization is improved, and data query rate and accuracy are improved.

[0123] It should be noted that the data query method provided in the embodiment of the present application is generally executed by the server 705 , and accordingly, the data query device is generally arranged in the server 705 .

[0124] It should be understood that Figure 7 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0125] Reference below Figure 8 , which shows a schematic diagram of the structure of a computer system 800 of a terminal device suitable for implementing an embodiment of the present application. Figure 8 The terminal device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0126] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the computer system 800 are also stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0127] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.

[0128] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the system of the present application are executed.

[0129] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. Computer-readable storage media may include, for example, but are not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0130] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0131] The units involved in the embodiments described in the present application may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a receiving unit, a determining unit, a resource group determining unit, and an executing unit. The names of these units do not constitute limitations on the units themselves in some cases.

[0132] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes receiving a data query request, obtaining a corresponding user identifier and a scene identifier; determining a corresponding query instance and a sub-scheduler identifier according to the user identifier and the scene identifier; calling a sub-scheduling method corresponding to the sub-scheduler identifier to determine a corresponding resource group according to the user identifier; and executing the query instance based on the resource group and returning the corresponding query result data.

[0133] According to the technical solution of the embodiment of the present application, resource utilization is improved through resource isolation, and the data query rate and accuracy are improved.

[0134] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A data query method, characterized in that: include: Receive data query request and obtain corresponding user ID and scene ID; Determine a corresponding query instance and a sub-scheduler identifier according to the user identifier and the scenario identifier; Calling a sub-scheduling method corresponding to the sub-scheduler identifier to determine a corresponding resource group according to the user identifier; Based on the resource group, the query instance is executed and corresponding query result data is returned.

2. The method according to claim 1, characterized in that The determining, according to the user identifier and the scenario identifier, a corresponding query instance and a sub-scheduler identifier includes: Determining a corresponding user group according to the user identifier and the scenario identifier; Matching the user group with each preset user group in a preset user group set, and in response to a match failure, obtaining demand data and resource quota data corresponding to the data query request; Creating a target user group and a target resource group according to the demand data and the resource quota data; According to the target user group and the target resource group, create a corresponding sub-scheduler and generate a corresponding sub-scheduler identifier; According to the user identifier and the scenario identifier, corresponding operation data and machine data are determined, and then according to the operation data and the machine data, a query instance is generated.

3. The method according to claim 2, characterized in that The determining a corresponding user group according to the user identifier and the scenario identifier includes: Determining corresponding data permissions according to the user identifier and the scenario identifier; According to the data permission, a corresponding user group is determined.

4. The method according to claim 1, characterized in that: The executing the query instance includes: Obtaining resource occupancy rate corresponding to the query instance; Based on the resource occupancy rate, a query restriction operation is performed.

5. The method according to claim 4, characterized in that The performing a query restriction operation based on the resource occupancy rate includes: When the resource usage is greater than the preset threshold, the corresponding query instance is blacklisted and restricted.

6. The method according to claim 1, characterized in that The determining of the corresponding resource group includes: Determine a corresponding resource queue according to the user identifier; The resource occupancy rate corresponding to each candidate resource group in the resource queue is obtained, and then the candidate resource group corresponding to the lowest resource occupancy rate is determined as the resource group corresponding to the user identifier.

7. The method according to claim 1, characterized in that The determining, according to the user identifier and the scenario identifier, a corresponding query instance and a sub-scheduler identifier includes: Determine the number of the user identifiers, and in response to the number being multiple, obtain priorities corresponding to the user identifiers, and then determine the target user identifier according to the priorities; According to the target user identifier and the scenario identifier, a corresponding query instance and a sub-scheduler identifier are determined.

8. A data query device, characterized in that: include: A receiving unit is configured to receive a data query request and obtain a corresponding user identifier and a scene identifier; A determination unit, configured to determine a corresponding query instance and a sub-scheduler identifier according to the user identifier and the scenario identifier; a resource group determination unit configured to call a sub-scheduling method corresponding to the sub-scheduler identifier to determine a corresponding resource group according to the user identifier; The execution unit is configured to execute the query instance based on the resource group and return corresponding query result data.

9. The device according to claim 8, characterized in that The determining unit is further configured to: Determining a corresponding user group according to the user identifier and the scenario identifier; Matching the user group with each preset user group in a preset user group set, and in response to a match failure, obtaining demand data and resource quota data corresponding to the data query request; Creating a target user group and a target resource group according to the demand data and the resource quota data; According to the target user group and the target resource group, create a corresponding sub-scheduler and generate a corresponding sub-scheduler identifier; According to the user identifier and the scenario identifier, corresponding operation data and machine data are determined, and then according to the operation data and the machine data, a query instance is generated.

10. The device according to claim 9, characterized in that The determining unit is further configured to: Determining corresponding data permissions according to the user identifier and the scenario identifier; According to the data permission, a corresponding user group is determined.

11. A data query electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

12. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images