Advanced user resource access operation guarantee method and device, equipment and medium
By identifying user levels and adjusting permission settings, the problem of resource competition among advanced users in the Apache Doris engine is solved, ensuring query performance for advanced users and improving user satisfaction.
Patent Information
- Application Number
- CN202411358773.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-26
- Publication Date
- 2025-10-10
- Estimated Expiration
- 2044-09-26
AI Technical Summary
In engines such as Apache Doris, advanced users cannot guarantee their query performance when cluster resources are tight, resulting in query performance degradation, affecting user satisfaction, and lack of user-level resource competition management.
By querying the current memory usage of users in the cluster, identifying user levels, and based on the preset user resource configuration information, generating a judgment result on whether the memory is sufficient, adjusting the permission settings of ordinary users to ensure resource access operations of advanced users, including blocking excessive user operations, sending prompts and warnings.
Effectively guarantee resource access performance for advanced users, reduce resource preemption, improve response time, and enhance user satisfaction.
Smart Images

Figure CN119312301B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of big data technology, and in particular to a method, apparatus, device, and medium for ensuring access to resources by high-level users. Background Art
[0002] In the era of big data, real-time data is crucial to business operations. This is especially true in the financial sector, where high-priority users have an even more pressing need for extremely fast responses and decision-making. For example, the Apache Doris engine, a high-performance MPP analytical database, with its millisecond-level query response, high concurrency, high availability, and easy scalability, has become an ideal choice for real-time data warehouses in big data scenarios.
[0003] However, current engines like Doris lack user tiering. When multiple users simultaneously perform queries and computations in a cluster (within the same resource group), resource competition and mutual impact can occur. This can lead to queries for advanced users being unable to guarantee performance when cluster resources are limited, slowing down user decision-making and impacting their satisfaction. Summary of the Invention
[0004] The purpose of the embodiments of the present application is to provide a method, apparatus, device and medium for guaranteeing resource access operations for advanced users, the main purpose of which is to ensure that advanced users' operations can obtain sufficient cluster resources.
[0005] To solve the above technical problems, the present invention provides a method for guaranteeing access to resources for advanced users, which adopts the following technical solutions:
[0006] A method for ensuring access to resources by high-level users, comprising the following steps:
[0007] Query the current memory usage of all users performing the current resource access operation in the cluster;
[0008] Get preset user resource configuration information;
[0009] identifying user levels of all users who perform the current resource access operation based on the user resource configuration information;
[0010] When it is recognized that the user levels include both ordinary users and advanced users, generating a determination result of whether the current memory is sufficient based on the current memory usage of all the users;
[0011] Based on the judgment result, corresponding permission setting is performed on the current resource access operation of the ordinary user to ensure the current resource access operation of the advanced user.
[0012] Furthermore, the performing of corresponding permission setting on the current resource access operation of the ordinary user based on the judgment result includes the following steps:
[0013] If the judgment result is that the current memory is insufficient, blocking the current resource access operation of excess users among the ordinary users; the excess users are ordinary users whose current memory usage exceeds a preset proportion of the total memory of all resources in the cluster;
[0014] A blocking prompt is sent to the excess user.
[0015] Furthermore, when it is identified that the user levels also include the advanced user, the performing of corresponding permission setting on the current resource access operation of the ordinary user based on the determination result further includes the following steps:
[0016] If the judgment result is that the current memory is sufficient, determining whether the current memory usage of the ordinary user exceeds a preset range;
[0017] If it is determined that the current resource usage does not exceed the preset range, the current resource access operation of the ordinary user is allowed, and whether the current memory usage of the ordinary user exceeds a preset ratio of the preset range is determined. If the current memory usage exceeds the preset ratio, an early warning prompt is sent to the ordinary user.
[0018] If it is determined that the resource access exceeds the preset range, the current resource access operation of the ordinary user is blocked, and a blocking prompt is sent to the ordinary user.
[0019] Furthermore, generating a judgment result of whether the current memory is sufficient based on the current memory usage of all the users includes the following steps:
[0020] Determine whether the current memory usage of each user who performs the current resource access operation satisfies a preset ratio less than the total memory of all resources in the cluster;
[0021] If it is determined that the preset ratio is less than the total memory of all resources in the cluster, a determination result of sufficient current memory is generated;
[0022] If it is determined that the preset proportion less than the total memory of all resources in the cluster is not satisfied, a determination result that the current memory is insufficient is generated.
[0023] Furthermore, the user resource configuration information includes a user name and a corresponding memory quota; and the identifying the user levels of all the users who perform the current resource access operation based on the user resource configuration information includes the following steps:
[0024] Matching a corresponding memory quota in the user resource configuration information based on the user's username;
[0025] If the matched memory quota is greater than 0, the user is regarded as a common user;
[0026] If the matched memory quota is equal to 0, the user is considered as the advanced user;
[0027] If the memory quota is not matched, the user is regarded as the advanced user.
[0028] Further, upon identifying that the user levels include common users, keywords are extracted from the operation log of the current resource access operation of the common users;
[0029] Based on the keyword, determining whether the current memory usage of the current resource access operation of the ordinary user exceeds a warning level;
[0030] If it is determined that the memory usage exceeds the warning level, a feedback indicating that the memory usage exceeds the warning level is sent to the ordinary user.
[0031] In order to solve the above technical problems, the embodiment of the present application further provides a device for ensuring access to resources by advanced users, which adopts the following technical solutions:
[0032] A high-level user resource access operation guarantee device, comprising:
[0033] An information acquisition module is used to query the current memory usage of all users performing the current resource access operation in the cluster;
[0034] Configuration acquisition module, used to obtain preset user resource configuration information;
[0035] a level identification module, configured to identify the user levels of all the users who perform the current resource access operation based on the user resource configuration information;
[0036] a memory determination module, configured to, upon recognizing that the user level includes the common user, generate a determination result on whether the current memory is sufficient based on the current memory usage of all the users;
[0037] The authority setting module is used to perform corresponding authority setting on the current resource access operation of the ordinary user based on the judgment result, so as to ensure the current resource access operation of the advanced user.
[0038] Furthermore, when it is identified that the user levels also include the advanced users, the permission setting module includes:
[0039] an operation blocking submodule, configured to block the current resource access operation of the ordinary user if the judgment result is that the current memory is insufficient;
[0040] The blocking sending submodule is used to send a blocking prompt to the ordinary user.
[0041] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solutions:
[0042] A computer device includes a memory and a processor. The memory stores computer-readable instructions. When the processor executes the computer-readable instructions, the processor implements the steps of any one of the above methods for ensuring access to advanced user resources.
[0043] In order to solve the above technical problems, the embodiments of the present application further provide a computer-readable storage medium, which adopts the following technical solutions:
[0044] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of any one of the above methods for ensuring access to resources by advanced users.
[0045] Compared with the prior art, the embodiments of the present application have the following beneficial effects:
[0046] The embodiment of the present application queries the current memory usage of all users who perform current resource access operations in the cluster; identifies the user levels of all users who perform current resource access operations based on preset user resource configuration information; when the user levels are identified to include ordinary users, generates a judgment result on whether the current memory is sufficient based on the current memory usage of all users; based on the judgment result, performs corresponding permission settings on the current resource access operations of ordinary users to ensure that operations such as query requests of high-priority users can obtain sufficient cluster resources, reduce resource preemption, and ensure good response time even during cluster peak periods, so that advanced users can obtain a better service experience, thereby enhancing user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] In order to more clearly illustrate the solutions in this application, a brief introduction will be given below to the drawings required for use in the description of the embodiments of this application. Obviously, the drawings described below are some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0048] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;
[0049] Figure 2 A flowchart of an embodiment of a method for ensuring access to resources by advanced users according to the present application;
[0050] Figure 3 yes Figure 2 A flowchart of a specific implementation of step S205;
[0051] Figure 4 It is a structural diagram of an embodiment of a device for ensuring access to resources by advanced users according to the present application;
[0052] Figure 5 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION
[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.
[0054] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0055] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.
[0056] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.
[0057] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.
[0058] Terminal devices 101, 102, and 103 can be various electronic devices with display screens and support web browsing, including but not limited to smartphones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV), laptop computers, desktop computers, etc.
[0059] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .
[0060] It should be noted that the advanced user resource access operation guarantee method provided in the embodiment of the present application is generally executed by a server, and accordingly, the advanced user resource access operation guarantee device is generally set in the server.
[0061] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.
[0062] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for guaranteeing access to resources for advanced users according to the present application. The method for guaranteeing access to resources for advanced users according to the embodiment of the present application comprises the following steps:
[0063] Step S201: query the current memory usage of all users who perform current resource access operations in the cluster.
[0064] In this embodiment, the electronic device (eg Figure 1The server shown in FIG. 1 can query the current memory usage of all users performing the current resource access operation in the cluster via a wired connection or a wireless connection. It should be noted that the wireless connection methods mentioned above may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra wideband) connections, and other wireless connection methods currently known or to be developed in the future.
[0065] Taking the financial insurance business scenario as an example, users can log in to the financial insurance system through the client of the terminal device by entering information such as username and password, and submit operations to query various business resources (such as transaction data, payment data, business data) through the client financial insurance system interface.
[0066] For example, taking the Doris engine as an example, the Doris engine can set the advanced user resource access operation guarantee function of this application by adding the DorisFE dynamic parameter enable_resource_isolation. This dynamic parameter is a function switch that can dynamically turn on or off the function implemented by the advanced user resource access operation guarantee method of the embodiment of this application without restarting the server; in addition, the advanced user resource access operation guarantee method of the embodiment of this application can also be implemented by setting the Doris static parameters. Doris can query all query information of a user in the cluster through the API, such as: the user's current memory usage, as well as FE_IP, query_id, connection_id, Database, Statement and other information.
[0067] Specifically, the current memory usage of all users performing the current resource access operation in the query cluster can be refreshed at a preset interval (for example, every 10 seconds), and the refresh interval can be based on user needs. In one implementation, the Doris engine FE dynamic parameter mem_quota_delay can be used to adjust it.
[0068] Step S202: Obtain preset user resource configuration information.
[0069] In one implementation, the user resource configuration information may include a user name and a corresponding memory quota.
[0070] For example, the configuration file content is as follows:
[0071] "User":[
[0072] {"UserName":"leader",
[0073] "memQuota": "0"},
[0074] {"UserName":"test",
[0075] "memQuota": "300"},
[0076] {"UserName: root",
[0077] "memQuota: 500"} ]
[0079] Where userName is the user name, and memQuota (memory quota) is the corresponding user memory quota (in bytes).
[0080] In one embodiment, continuing to take the Doris engine as an example, a new $DORIS_HOME / fe / conf / customer.json configuration can be added through FE. This configuration is a user level and resource quota configuration file, which is hot-loaded and can be dynamically modified without restarting the service.
[0081] Step S203: identifying all user levels that perform the current resource access operation based on the preset user resource configuration information.
[0082] In an optional embodiment, the user resource configuration information includes a user name and a corresponding memory quota; step S203, based on the preset user resource configuration information, identifying all user levels that perform the current resource access operation may include the following method steps:
[0083] Match the corresponding memory quota in the user resource configuration information based on the user's username;
[0084] If the matched memory quota is greater than 0, the user is considered as a normal user;
[0085] If the matched memory quota is equal to 0, the user is considered as a premium user;
[0086] If the memory quota is not matched, the user is considered a premium user.
[0087] Continuing with the configuration file described in the example above, for other users, if the quota is > 0, they are ordinary users; otherwise, they are advanced users. When a user is created in Doris but not configured in the user resource configuration customer.json file, it will also be considered an advanced user. In addition, this quota will not take effect on root and admin administrator users.
[0088] The embodiment of the present application matches the corresponding memory quota in the user resource configuration information based on the user's username, identifies whether the user is an ordinary user or an advanced user based on the matched memory quota, and directly links ordinary users and advanced users with memory quotas, thereby effectively ensuring the effective use of memory resources by advanced users.
[0089] Step S204 : after identifying that the user levels include both common users and advanced users, a determination result of whether the current memory is sufficient is generated based on the current memory usage of all users.
[0090] In one embodiment, in step S204, generating a determination result of whether the current memory is sufficient based on the current memory usage of all users may include the following method steps:
[0091] Determine whether the current memory usage of each user performing the current resource access operation meets the preset ratio of the total memory of all resources in the cluster;
[0092] If the judgment is that the memory usage is less than the preset ratio of the total memory of all resources in the cluster, a judgment result of sufficient memory is generated;
[0093] If the judgment does not meet the preset ratio less than the total memory of all resources in the cluster, a judgment result of insufficient current memory is generated.
[0094] For example, assuming that a user submits a request to query various business resources through the client financial insurance system interface, it can be set that when the current used memory of all users performing the current resource access operation in the cluster is less than the preset proportion (for example, 30%) of the total memory of all resources in the cluster (for example, transaction data, payment data, and business data), it is considered that there is sufficient memory, otherwise it is considered that there is insufficient memory.
[0095] The embodiment of the present application determines whether the memory usage of the current resource access operations of all users meets the requirement of being less than the total memory of all resources in the cluster. When the memory usage of the current resource access operations of all users meets the requirement of being less than the total memory of all resources in the cluster, the embodiment of the present application determines that the current memory is sufficient. In this way, the memory usage of each user in the cluster is fully taken into consideration, thereby obtaining a higher standard judgment result on whether the memory is sufficient, so as to effectively guarantee the current resource access operations of advanced users.
[0096] Step S205: Based on the judgment result, corresponding permission settings are performed on the current resource access operations of the common user to ensure the current resource access operations of the advanced user.
[0097] The embodiment of the present application flexibly performs corresponding different permission settings for the current resource access operations of ordinary users based on the judgment result of whether the memory is sufficient, thereby ensuring the current resource access operation permissions of advanced users.
[0098] The embodiment of the present application queries the current memory usage of all users who perform current resource access operations in the cluster; identifies the user levels of all users who perform current resource access operations based on preset user resource configuration information; when the user levels are identified to include ordinary users, generates a judgment result on whether the current memory is sufficient based on the current memory usage of all users; based on the judgment result, performs corresponding permission settings on the current resource access operations of ordinary users to ensure that operations such as query requests of high-priority users can obtain sufficient cluster resources, reduce resource preemption, and ensure good response time even during cluster peak periods, so that advanced users can obtain a better service experience, thereby enhancing user satisfaction.
[0099] like Figure 3 As shown, Figure 3 yes Figure 2 Flowchart of a specific implementation of step S205.
[0100] In an optional embodiment, when it is identified that the user level also includes advanced users, step S205, based on the judgment result, performing corresponding permission setting for the current resource access operation of the ordinary user may include the following method steps:
[0101] In step S2051, if the judgment result is that the current memory is insufficient, the current resource access operation of excess users among ordinary users is blocked; excess users are ordinary users whose current memory usage exceeds a preset proportion of the total memory of all resources in the cluster.
[0102] Step S2052: Send a blocking prompt to the excess user.
[0103] The embodiment of the present application identifies that user levels include not only ordinary users but also advanced users, and when it is determined that memory resources are insufficient, the current query requests and other operations of ordinary users can be blocked, thereby effectively ensuring the current resource access and other operations of advanced users; in addition, by sending a blocking prompt to the user, ordinary users can effectively understand the reasons for being blocked from the current resource access operation, thereby facilitating ordinary users to adjust their operation strategies in a timely manner.
[0104] For example, if a high-level user is currently performing a query operation in the Doris cluster and it is determined that the cluster memory is insufficient, ordinary users will be blocked from querying and a prompt will be displayed: "The current system has a high-power user querying and a lot of memory is being used. Please wait."
[0105] Continue as Figure 3 As shown, in an optional embodiment, step S205, based on the judgment result, performing corresponding permission setting on the current resource access operation of the common user may also include the following method steps:
[0106] Step S2053: If the judgment result is that the current memory is sufficient, determine whether the current memory usage of the ordinary user exceeds the preset range.
[0107] Step S2054: If it is determined that it does not exceed the preset range, the current resource access operation of the ordinary user is allowed. Step S2055: It is determined whether the current memory usage of the ordinary user exceeds the preset ratio of the preset range. Step S2056: If it exceeds the preset ratio, an early warning prompt is sent to the ordinary user.
[0108] Step S2057: If it is determined that the resource access exceeds the preset range, the current resource access operation of the ordinary user is blocked, and a blocking prompt is sent to the ordinary user.
[0109] For example, if the current memory is sufficient, the user's used memory is checked. If it has not reached 80% of the quota (i.e., the preset proportion within the preset range), the query is allowed but a log alert is displayed: "User XXX mem usage has reached loadwater of quota!" If the quota is reached, the user is blocked from querying and a message is displayed: "User XXX's memory usage has reached the quota, quota: XXX, current usage: XXX."
[0110] The embodiment of the present application determines whether the current memory usage of ordinary users exceeds a preset range when it is determined that memory resources are sufficient. If it does not exceed the preset range, the current query of the ordinary user is allowed, and determines whether the current memory usage of the ordinary user exceeds a preset proportion of the preset range. If it exceeds the preset proportion, an early warning prompt is sent to the ordinary user; if it exceeds the preset range, the current query of the ordinary user is blocked, and a blocking prompt is sent to the ordinary user. Therefore, a certain amount of memory can be reserved to ensure that it will not affect the subsequent operations of newly added advanced users to execute queries, etc. In addition, an early warning is given to ordinary users of possible blocked operations, which facilitates ordinary users to adjust their operation strategies in time to improve the user experience of ordinary users.
[0111] In an optional embodiment, after identifying all user levels performing the current resource access operation based on the user resource configuration information in step S202, the high-level user resource access operation guarantee method of the present application further includes the following steps:
[0112] The user levels identified include common users, and keywords are extracted from the operation logs of the common users' current resource access operations;
[0113] Based on the keyword, determine whether the current memory usage of the current resource access operation of the ordinary user exceeds the warning level;
[0114] If it is determined that the memory usage exceeds the warning limit, a feedback indicating that the memory usage exceeds the warning limit is sent to ordinary users.
[0115] For example, by scanning the operation log of an ordinary user, it is found that there is a need to pre-access business data that occupies a large amount of memory in the operation log. Based on this and combined with certain preset rules, it can be judged that the ordinary user's current resource access operation may or has exceeded the warning level, so that feedback that the memory usage exceeds the warning level can be sent to the user in a timely manner through various means such as email, text message, and phone call, so as to facilitate the ordinary user to adjust the use of memory resources in a timely manner.
[0116] The embodiment of the present application extracts keywords from the operation log of the current resource access operation of the ordinary user. After judging based on the keywords that the memory usage of the current resource access operation of the ordinary user has or may exceed the preset warning level, the feedback of the ordinary user's memory usage exceeding the warning level can be sent to the user in a timely manner. This can facilitate the ordinary user to adjust the memory resource usage strategy in a timely manner, reduce the impact on the ordinary user due to the actual memory usage limit, and reduce the impact on the entire resource group and even other tasks in the cluster.
[0117] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.
[0118] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.
[0119] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware via computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes in the above-described method embodiments. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).
[0120] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
[0121] Further references Figure 4 , as a response to the above Figure 2 The present application provides an embodiment of a device for guaranteeing access to resources of a high-level user. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0122] like Figure 4 As shown, the advanced user resource access operation guarantee device 300 of the embodiment of the present application includes: an information query module 301, a configuration acquisition module 302, a level identification module 303, a memory judgment module 304 and a matrix generation module 305. Among them:
[0123] Information acquisition module 301, used to query the current memory usage of all users performing current resource access operations in the cluster;
[0124] Configuration acquisition module 302, used to obtain preset user resource configuration information; level identification module 303, used to identify all user levels performing the current resource access operation based on the user resource configuration information;
[0125] The memory determination module 304 is configured to determine whether the current memory is sufficient based on the current memory usage of all users after identifying that the user levels include both ordinary users and advanced users;
[0126] The authority setting module 305 is used to perform corresponding authority setting on the current resource access operation of the common user based on the judgment result, so as to ensure the current resource access operation of the advanced user.
[0127] The embodiment of the present application queries the current memory usage of all users who perform current resource access operations in the cluster; identifies the user levels of all users who perform current resource access operations based on preset user resource configuration information; when the user levels are identified to include ordinary users, generates a judgment result on whether the current memory is sufficient based on the current memory usage of all users; based on the judgment result, performs corresponding permission settings on the current resource access operations of ordinary users to ensure that operations such as query requests of high-priority users can obtain sufficient cluster resources, reduce resource preemption, and ensure good response time even during cluster peak periods, so that advanced users can obtain a better service experience, thereby enhancing user satisfaction.
[0128] In some optional implementations of this embodiment, the permission setting module 405 includes:
[0129] An operation blocking submodule is used to block the current resource access operation of excess users among ordinary users if the result of the judgment is that the current memory is insufficient; the excess users are ordinary users whose current memory usage exceeds a preset proportion of the total memory of all resources in the cluster;
[0130] The block sending submodule is used to send block prompts to excess users.
[0131] The embodiment of the present application identifies that user levels include not only ordinary users but also advanced users, and when it is determined that memory resources are insufficient, the current query requests and other operations of ordinary users can be blocked, thereby effectively ensuring the current resource access and other operations of advanced users; in addition, by sending a blocking prompt to the user, ordinary users can effectively understand the reasons for being blocked from the current resource access operation, thereby facilitating ordinary users to adjust their operation strategies in a timely manner.
[0132] In some optional implementations of this embodiment, the permission setting module 405 further includes:
[0133] The range judgment submodule is used to judge whether the current memory usage of ordinary users exceeds the preset range if the judgment result is that the current memory is sufficient;
[0134] An operation permission submodule is used to allow the common user's current resource access operation if it is determined that the current memory usage does not exceed the preset range, and to determine whether the common user's current memory usage exceeds a preset ratio of the preset range. If it exceeds the preset ratio, a warning prompt is sent to the common user;
[0135] The operation blocking submodule is used to block the current resource access operation of ordinary users if it is judged to exceed the preset range, and send a blocking prompt to the ordinary users.
[0136] In an embodiment of the present application, when memory resources are sufficient, it is determined whether the current memory usage of ordinary users exceeds a preset range (for example, a preset proportion). If it does not exceed the preset proportion, the current query of the ordinary user is allowed, and an early warning prompt is sent to the ordinary user. If it exceeds the preset proportion, the current query of the ordinary user is blocked, and a blocking prompt is sent to the ordinary user. A certain amount of memory can be reserved to ensure that it will not affect the subsequent query execution and other operations of new trained and advanced users.
[0137] In some optional implementations of this embodiment, the memory determination module 404 includes:
[0138] The condition judgment submodule is used to determine whether the current memory usage of each user performing the current resource access operation meets the preset ratio of the total memory of all resources in the cluster;
[0139] A sufficiency judgment submodule is used to generate a judgment result of current memory sufficiency if it is judged that the current memory is less than a preset ratio of the total memory of all resources in the cluster;
[0140] The insufficient judgment submodule is used to generate a judgment result of insufficient current memory if the judgment does not meet the preset ratio of the total memory of all resources in the cluster.
[0141] The embodiment of the present application determines whether the memory usage of the current resource access operations of all users meets the requirement of being less than the total memory of all resources in the cluster. When the memory usage of the current resource access operations of all users meets the requirement of being less than the total memory of all resources in the cluster, the embodiment of the present application determines that the current memory is sufficient. In this way, the memory usage of each user in the cluster is fully considered, thereby obtaining a higher standard judgment result on whether the memory is sufficient, so as to effectively guarantee the current resource access operations of advanced users.
[0142] In some optional implementations of this embodiment, the user resource configuration information includes a user name and a corresponding memory quota; the level identification module 403 includes:
[0143] A quota matching submodule, configured to match a corresponding memory quota in the user resource configuration information based on the user's username;
[0144] A normal determination submodule, configured to determine that the user is a normal user if the matched memory quota is greater than 0;
[0145] A first determining submodule, configured to determine that the user is a premium user if the matched memory quota is equal to 0;
[0146] The second determining submodule is configured to determine that if the memory quota is not matched, the user is regarded as the advanced user.
[0147] The embodiment of the present application matches the corresponding memory quota in the user resource configuration information based on the user's username, identifies whether the user is an ordinary user or an advanced user based on the matched memory quota, and directly links ordinary users and advanced users with memory quotas, thereby effectively ensuring the effective use of memory resources by advanced users.
[0148] In some optional implementations of this embodiment, the apparatus further includes:
[0149] A key extraction module is used to extract keywords from the operation log of the current resource access operation of the ordinary user when the user level is identified to include the ordinary user;
[0150] The warning judgment module is used to judge whether the current memory usage of the current resource access operation of ordinary users exceeds the warning level based on keywords;
[0151] The feedback sending module is used to send feedback that the memory usage exceeds the warning level to ordinary users if it is determined that the memory usage exceeds the warning level.
[0152] The embodiment of the present application can facilitate ordinary users to adjust their memory resource usage strategies in a timely manner after determining that the memory resources of ordinary users exceed the warning level, thereby reducing the impact on ordinary users caused by memory usage restrictions and reducing the impact on the entire resource group and even other tasks in the cluster.
[0153] To solve the above technical problems, the present application also provides a computer device. Figure 5 , Figure 5 This is a basic structural block diagram of the computer device in this embodiment.
[0154] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 6 with components 61-63, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.
[0155] The computer device can be a desktop computer, a notebook computer, a palm computer, a cloud server, or the like. The computer device can interact with a user through a keyboard, a mouse, a remote controller, a touchpad, a voice control device, or the like.
[0156] The memory 61 includes at least one type of readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, or the like), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, or the like. In some embodiments, the memory 61 can be an internal storage unit of the computer device 6, such as a hard disk or a memory of the computer device 6. In other embodiments, the memory 61 can also be an external storage device of the computer device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, or the like. Of course, the memory 61 can include both an internal storage unit and an external storage device of the computer device 6. In this embodiment, the memory 61 is generally used to store an operating system and various application software installed in the computer device 6, such as computer readable instructions of the advanced user resource access operation guarantee method, or the like. In addition, the memory 61 can also be used to temporarily store various data that has been output or will be output.
[0157] The processor 62 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run computer readable instructions or process data stored in the memory 61, such as computer readable instructions of the advanced user resource access operation guarantee method.
[0158] The network interface 63 can include a wireless network interface or a wired network interface, and is generally used to establish a communication connection between the computer device 6 and other electronic devices.
[0159] The embodiment of the present application queries the current memory usage of all users who perform current resource access operations in the cluster; identifies the user levels of all users who perform current resource access operations based on preset user resource configuration information; when the user levels are identified to include ordinary users, generates a judgment result on whether the current memory is sufficient based on the current memory usage of all users; based on the judgment result, performs corresponding permission settings on the current resource access operations of ordinary users to ensure that operations such as query requests of high-priority users can obtain sufficient cluster resources, reduce resource preemption, and ensure good response time even during cluster peak periods, so that advanced users can obtain a better service experience, thereby enhancing user satisfaction.
[0160] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions. The computer-readable instructions can be executed by at least one processor to enable at least one processor to perform the steps of the advanced user resource access operation guarantee method as described above.
[0161] The embodiment of the present application queries the current memory usage of all users who perform current resource access operations in the cluster; identifies the user levels of all users who perform current resource access operations based on preset user resource configuration information; when the user levels are identified to include ordinary users, generates a judgment result on whether the current memory is sufficient based on the current memory usage of all users; based on the judgment result, performs corresponding permission settings on the current resource access operations of ordinary users to ensure that operations such as query requests of high-priority users can obtain sufficient cluster resources, reduce resource preemption, and ensure good response time even during cluster peak periods, so that advanced users can obtain a better service experience, thereby enhancing user satisfaction.
[0162] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of each embodiment of the present application.
[0163] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.
[0164] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.
Claims
1. A method for ensuring access to resources by advanced users, characterized in that: The method comprises the following steps: Query the current memory usage of all users performing current resource access operations in the cluster; Get preset user resource configuration information; identifying user levels of all users who perform the current resource access operation based on the user resource configuration information; When it is recognized that the user levels include both ordinary users and advanced users, generating a determination result of whether the current memory is sufficient based on the current memory usage of all the users; Based on the judgment result, performing corresponding permission setting on the current resource access operation of the ordinary user to ensure the current resource access operation of the advanced user; If the judgment result is that the current memory is insufficient, preventing excess users among the ordinary users from accessing the current resource; The excess user is an ordinary user whose current memory usage exceeds a preset proportion of the total memory of all resources in the cluster; Sending a blocking prompt to the excess user; If the judgment result is that the current memory is sufficient, determining whether the current memory usage of the ordinary user exceeds a preset range; If it is determined that the current resource usage does not exceed the preset range, the current resource access operation of the ordinary user is allowed, and whether the current memory usage of the ordinary user exceeds a preset ratio of the preset range is determined. If the current memory usage exceeds the preset ratio, an early warning prompt is sent to the ordinary user. If it is determined that the resource access exceeds the preset range, the current resource access operation of the ordinary user is blocked, and a blocking prompt is sent to the ordinary user.
2. The method for ensuring access to resources by advanced users according to claim 1, wherein: The generating of a judgment result of whether the current memory is sufficient based on the current memory usage of all the users comprises the following steps: Determine whether the current memory usage of each user who performs the current resource access operation satisfies a preset ratio less than the total memory of all resources in the cluster; If it is determined that the preset ratio is less than the total memory of all resources in the cluster, a determination result of sufficient current memory is generated; If it is determined that the preset proportion less than the total memory of all resources in the cluster is not satisfied, a determination result that the current memory is insufficient is generated.
3. The method for ensuring access to resources by advanced users according to claim 1, wherein: The user resource configuration information includes a user name and a corresponding memory quota; and identifying the user levels of all users who perform the current resource access operation based on the user resource configuration information includes the following steps: Matching a corresponding memory quota in the user resource configuration information based on the user's username; If the matched memory quota is greater than 0, the user is regarded as a common user; If the matched memory quota is equal to 0, the user is considered as the advanced user; If the memory quota is not matched, the user is regarded as the advanced user.
4. The method for ensuring access to resources by high-level users according to claim 1, wherein: After identifying the user levels of all the users who perform the current resource access operation based on the user resource configuration information, the method further includes the following steps: Upon identifying that the user levels include common users, extracting keywords from the operation log of the current resource access operation of the common users; Based on the keyword, determining whether the current memory usage of the current resource access operation of the ordinary user exceeds a warning level; If it is determined that the memory usage exceeds the warning level, a feedback indicating that the memory usage exceeds the warning level is sent to the ordinary user.
5. A high-level user resource access operation guarantee device, characterized in that: The device implements the steps of the method for ensuring access to resources by advanced users according to any one of claims 1 to 4, and the device comprises: The information acquisition module is used to query the current memory usage of all users performing current resource access operations in the cluster; Configuration acquisition module, used to obtain preset user resource configuration information; a level identification module, configured to identify the user levels of all the users who perform the current resource access operation based on the user resource configuration information; a memory determination module, configured to, upon recognizing that the user levels include ordinary users, generate a determination result on whether current memory is sufficient based on the current memory usage of all the users; The authority setting module is used to perform corresponding authority setting on the current resource access operation of the ordinary user based on the judgment result, so as to ensure the current resource access operation of the advanced user.
6. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the advanced user resource access operation guarantee method according to any one of claims 1 to 4 when executing the computer-readable instructions.
7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the method for ensuring access to resources by advanced users according to any one of claims 1 to 4.
Citation Information
Patent Citations
Multi-user task scheduling method and device for calculation cluster
CN107291545A
Resource access method, cache server, storage medium and electronic equipment
CN111510491A