Resource management system
By designing a resource management system, using internal resource management modules to perceive resource usage and generate resource request instructions, the problem of inefficiency in resource management of the stream processing framework is solved, and fine control and efficient management of resource usage is achieved.
Patent Information
- Application Number
- CN202411918821.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-24
- Publication Date
- 2025-03-25
AI Technical Summary
The existing stream processing framework is difficult to control the resource usage during task execution, resulting in low resource management efficiency.
A resource management system is designed, including internal resource management module, product-side interface module and external resource management module. The internal resource management module generates resource request instructions by perceiving resource usage, and the interactive module forwards it to the product-side interface module, which then forwards the instructions to the external resource management module for processing.
It realizes fine control of the resource usage of task execution, improves resource management efficiency, and manages and monitors resource usage through product-side interfaces.
Smart Images
Figure CN119690679A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a resource management system. Background Art
[0002] Stream processing frameworks can be used to process real-time data streams to achieve rapid data processing, analysis, and response. Commonly used stream processing frameworks include Flink, Kafka Streams, and Spark Streaming. However, in the prior art, stream processing frameworks usually interact with external resource managers through programming interfaces provided by external resource managers, and are unable to control resource usage during task execution, making it difficult to manage task resources, and resource management efficiency is also low. Summary of the invention
[0003] In view of this, an embodiment of the present disclosure provides a resource management system that can control resource usage during task execution and improve resource management efficiency.
[0004] In a first aspect, an embodiment of the present disclosure provides a resource management system, including an internal resource management module, a product-side interface module, and an external resource management module;
[0005] The internal resource management module is used to obtain the resource usage of the system when executing tasks, generate resource request instructions based on the resource usage, and forward the resource request instructions to the product interface module;
[0006] The product-side interface module is used to receive the resource request instruction forwarded by the internal resource management module and forward the resource request instruction to the external resource management module;
[0007] The external resource management module is used to receive the resource request instructions forwarded by the product-side interface module, adjust the system's resource status according to the resource request instructions, and return the system's resource status information to the product-side interface module.
[0008] Optionally, the internal resource management module includes a perception module and an interaction module;
[0009] The perception module is used to obtain the resource usage of the system when executing the task; in response to the resource usage corresponding to the task indicating that the operating resources are insufficient, a resource request instruction for requesting capacity expansion is generated; in response to the resource usage corresponding to the task indicating that the operating resources are idle, a resource request instruction for requesting capacity reduction is generated; and the resource request instruction is forwarded to the interaction module;
[0010] The interaction module is used to receive the resource request instruction forwarded by the perception module, and call the method corresponding to the request in the resource request instruction to forward the resource request instruction to the product-side interface module.
[0011] Optionally, the internal resource management module further includes a scheduling module;
[0012] The scheduling module is used to determine the status information of the container in the system when the interaction module receives the resource request instruction forwarded by the perception module; and generate a container request instruction based on the status information of the container in the system; and forward the container request instruction to the product-side interface module, so that the product-side interface module forwards the container request instruction to the external resource management module.
[0013] Optionally, the product-side interface module is used to receive resource request instructions forwarded by the internal resource management module, and forward the resource request instructions to the external resource management module; and save the resource usage for executing the task, check the resource usage of the task, and adjust the resources used to execute the task according to the resource usage of the task.
[0014] Optionally, the external resource management module is used to receive resource request instructions forwarded by the product-side interface module; in response to the resource request instruction being a resource request instruction requesting capacity expansion, a container is created in the system based on the resource request instruction; in response to the resource request instruction being a resource request instruction requesting capacity reduction, the container in the system is deleted based on the resource request instruction to release system resources; and the system resource status information is returned to the product-side interface module according to a preset time period.
[0015] Optionally, the internal resource management module is also used to record the creation time of the external resource management module to create a container in the system. In response to the creation time exceeding a preset time threshold, the status information of the container is updated to creation failure, and the data used to create the container is deleted, and the container is re-created in the system based on the resource request instruction.
[0016] Optionally, the system further comprises a storage module;
[0017] The storage module is used to obtain the status information of each container in the system from the internal resource management module, and based on the status information, generate and store different container lists to record the status information of each container in the system.
[0018] Optionally, the container list includes a list of containers requested to be created, a list of containers requested to be deleted, a list of all system containers, and a list of containers that do not exist after multiple queries.
[0019] Optionally, the full list of system containers includes container information that should be in normal operating state; the internal resource management module is also used to compare the queried running container information with the container information that should be in normal operating state in the full list of system containers; in response to different comparison results, a list of containers that do not exist in multiple queries is generated based on the comparison results.
[0020] Optionally, the product-side interface module is further used to verify the resource application amount in the resource request instruction for requesting capacity expansion according to the resource application amount in the resource request instruction for requesting capacity expansion and the resource usage of the system.
[0021] The resource management system provided by the embodiment of the present disclosure no longer uses the stream processing framework to interact directly with the external resource manager when resource adjustment is required. Instead, the resource adjustment request is first forwarded to the product side, and then the product side forwards the request to the external resource manager, and the external resource manager implements the actual resource processing. At the same time, within the stream processing framework, the internal resource manager can also be used to perceive the resource usage during task execution, thereby being able to respond to state changes of resource containers in a timely manner, ensuring the flexibility and dynamic adjustment capabilities of resource management, thereby improving resource management efficiency. In addition, when forwarding resource processing requests, the product-side interface can also associate the container corresponding to the task, so as to achieve management and monitoring of task resource usage on the product side, further improving the efficiency of resource management. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.
[0023] Figure 1 A schematic diagram of a resource management system provided by an embodiment of the present disclosure is shown;
[0024] Figure 2 An exemplary system architecture is shown in which embodiments of the present disclosure may be applied. DETAILED DESCRIPTION
[0025] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.
[0026] It should be noted that in the technical solution of the present invention, the collection, use, storage, sharing and transfer of user personal information involved are in compliance with the provisions of relevant laws and regulations, and it is necessary to inform the user and obtain the user's consent or authorization. When applicable, the user's personal information is de-identified and / or anonymized and / or encrypted.
[0027] The raising of the above problems and the solutions are the results obtained by the inventor after practice and careful research. The discovery process of the above problems and the solutions proposed for the above problems should be the contributions made by the inventor to the present disclosure during the disclosure process.
[0028] The technical solutions in the present disclosure will be clearly and completely described below in conjunction with the drawings in the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.
[0029] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0030] To facilitate understanding of this embodiment, a resource management system disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the resource management system provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities. In some possible implementation methods, the resource management system can be implemented by a processor calling computer-readable instructions stored in a memory.
[0031] See also Figure 1 As shown in FIG. 1 , a structure diagram of a resource management system provided by an embodiment of the present disclosure is shown. It should be noted in advance that: Figure 1 The arrows in the figure represent a possible interaction mode between the modules in each layer, and do not constitute an improper limitation on the present invention. Figure 1 As shown, the embodiment of the present disclosure provides a structure of a resource management system, including an internal resource management module, a product-side interface module and an external resource management module, wherein:
[0032] The internal resource management module is used to obtain the resource usage of the system when executing tasks, generate resource request instructions based on the resource usage, and forward the resource request instructions to the product-side interface module.
[0033] In the disclosed embodiment, the internal resource management module includes a perception module. Specifically, the perception module is used to obtain the resource usage of the system when executing the task; in response to the resource usage corresponding to the task indicating that the operating resources are insufficient, a resource request instruction for requesting capacity expansion is generated; in response to the resource usage corresponding to the task indicating that the operating resources are idle, a resource request instruction for requesting capacity reduction is generated; and the resource request instruction is forwarded to the interaction module.
[0034] In a specific implementation, when submitting a task, the perception module can first check whether the resources are sufficient. If the resources are insufficient, a resource request instruction requesting capacity expansion can be generated, thereby creating a new resource for task execution. In another possible implementation, the perception module can also obtain the resource usage in the current cluster through top-down queries and implementation detection, such as the operating status of the container, resource occupancy rate, resource idle time, etc. When the perception module detects that the idle time of the container exceeds the specified time, it can generate a corresponding resource request instruction requesting to delete the container, thereby releasing redundant resource containers in the system.
[0035] In a specific implementation, the perception module can periodically query the status of the container through the interface of the internal resource management module. The interface here depends on each driver (resource management driver). By communicating with the external resource management module through the driver, the perception module can obtain the status of the container and related resource usage in real time. When the perception module finds that the resource usage corresponding to the task indicates insufficient resources, it can generate a resource request instruction to request capacity expansion; for example, if it is detected that the current number of containers cannot meet the needs of task execution, the perception module can generate a corresponding instruction to trigger the startWorker method to request the creation of a new container. The resource request instruction here can include information about the resources, such as CPU (Central Processing Unit) occupancy, memory, etc., and then the perception module can forward the resource request instruction to the interaction module for subsequent processing.
[0036] Similarly, when the perception module determines that certain containers are idle or no longer needed within a certain period of time, a resource request instruction requesting capacity reduction can be generated. At this time, the interaction module can be notified by calling the releaseResource method to request the release of container resources that are no longer in use. For example, if a container has not processed any tasks in the past 10 minutes, it can be considered idle, thereby triggering a capacity reduction request. It should be noted that the method mentioned above for determining whether a container is idle is only used as an example of a feasible implementation method in the embodiment of the present disclosure, and does not constitute an improper limitation on the present invention. In actual applications, other judgment methods can be set according to needs. The embodiment of the present disclosure does not make specific limitations on this, and its function can be realized.
[0037] In the disclosed embodiment, the internal resource management module further includes an interaction module. Specifically, the interaction module is used to receive the resource request instruction forwarded by the perception module, call the method corresponding to the request in the resource request instruction, and forward the resource request instruction to the product-side interface module.
[0038] In a specific implementation, after the perception module perceives the resource usage in the system and generates the corresponding resource request instruction, it can forward the generated resource request instruction to the interaction module. After receiving the resource request instruction, the interaction module can call the corresponding method according to the type of underlying containerization in the current system to request the corresponding interface to interact with the external resource management module. After completing the processing of the request, the interaction module can forward the final request to the product-side interface module so that the module can interact with the external resource management module to ensure that steps such as resource application, resource release and status update can proceed smoothly.
[0039] Exemplarily, the interaction module can select the corresponding driver method for processing according to the received resource request instruction. For example, for expansion request, the interaction module can call the resource application interface implemented in driverA; for reduction request, the interaction module can call the stopWorker method to notify the external resource management module through the HTTP API to clean up related resources.
[0040] In the embodiment of the present disclosure, the internal resource management module also includes a scheduling module. Specifically, the scheduling module is used to determine the status information of the container in the system when the interaction module receives the resource request instruction forwarded by the perception module; and generate a container request instruction according to the status information of the container in the system; and forward the container request instruction to the product-side interface module, so that the product-side interface module forwards the container request instruction to the external resource management module.
[0041] In the specific implementation, when the interaction module receives the resource request instruction forwarded by the perception module, the scheduling module can call the container query interface to obtain the status information of all containers in the current system and determine whether the container is in a normal operating state. Then, the resource request instruction received by the interaction module can be checked according to the queried container status. When the detection passes, the resource request instruction can be forwarded to the product interface module.
[0042] In another possible implementation, the scheduling module may also periodically call the container query interface according to a preset time period to query the status of all containers in the current system and determine whether the container is in normal operation. According to the queried container status, it is decided whether it is necessary to call the expansion or reduction process. For example, if a container creation failure or an exception is detected, the expansion process will be triggered to reapply for the creation of a new container. Alternatively, after sensing that the state of the container has changed to running, it is necessary to check whether the Future associated with the container created this time is in a completed state. If it is still not completed, it can be set to completed and the current related container information can be stored. If the container state changes to an abnormal final state such as failure, the reduction process can be called to process the abnormal container, and the expansion process can be called to create a new available container to ensure that there are sufficient resources in the system to perform tasks.
[0043] In an embodiment of the present disclosure, the internal resource management module is also used to record the creation time of the external resource management module to create a container in the system. In response to the creation time exceeding a preset time threshold, the status information of the container is updated to creation failure, and the data used to create the container is deleted, and the container is re-created in the system based on the resource request instruction.
[0044] In a specific implementation, when the product-side interface module forwards a resource request instruction requesting capacity expansion to the external resource management module so that the external resource management module creates a new container, the internal resource management module can save the creation request information of the new container, such as the resource serial number, request time, container status, etc., and associate the Future object requested to be created (in asynchronous operations, the program can immediately return a Future object without waiting for the operation to be completed. When the operation is completed, the Future object will store the calculation result or exception), which is used to query the creation status of the new container. At the same time, the internal resource management module can record the creation time of the new container. When the creation time exceeds the preset time threshold, the status information of the container, that is, the Future object, can be marked as abnormal, indicating that the current new container creation failed. Then, the deletion interface can be called to notify the external resource management module to delete the relevant creation resources, and the corresponding Future object is set to an abnormal end, and according to the number of containers that failed to be created, the container creation process is restarted to notify the external resource management module to re-create a new container based on the resource request instruction requesting capacity expansion. This can achieve rapid processing when resource creation is abnormal.
[0045] It should be noted that the above-mentioned method of determining that the creation of a new container fails is only used as an example of a possible implementation method of the embodiment of the present disclosure. In actual applications, it is also possible to determine whether the new container is successfully created by monitoring the container status and other methods. The embodiment of the present disclosure does not make specific limitations on this, and the function can be realized.
[0046] The product-side interface module is used to receive the resource request instruction forwarded by the internal resource management module and forward the resource request instruction to the external resource management module.
[0047] In this step, the internal resource management module forwards the generated resource request instruction to the product-side interface module, and then the product-side interface module forwards the resource request instruction to the external resource management module. As a result, it is no longer necessary to interact with the external resource management module through the programming interface provided by the external resource management module, and the container resources can be managed and verified on the product side of the stream processing framework.
[0048] In the embodiment of the present disclosure, the product-side interface module is used to receive resource request instructions forwarded by the internal resource management module, and forward the resource request instructions to the external resource management module; and save the resource usage for executing the task, check the resource usage of the task, and adjust the resources used to execute the task according to the resource usage of the task.
[0049] In the disclosed embodiments, the product-side interface module can be understood as the external interface module of the stream processing framework, which is the part that communicates and interacts with the external system or service. In the product-side interface module, the stream processing framework can apply for, release, and monitor resources with the external resource management module. In a specific implementation, the stream processing framework can communicate with the external resource management module through the product side to dynamically apply for and release resources. In addition, the stream processing framework can also use different drivers in the product-side interface module to connect to different external systems. Each driver is responsible for calling the interface of the corresponding external system to complete resource management operations.
[0050] In the disclosed embodiment, the product-side interface module is further used to verify the resource application amount in the resource request instruction for requesting capacity expansion according to the resource application amount in the resource request instruction for requesting capacity expansion and the resource usage of the system.
[0051] In a specific implementation, when the product-side interface module forwards a resource request instruction to the external resource management module, it can obtain the resource request amount in the resource request instruction, and at the same time compare it with the resource usage limit stored in the product-side interface itself or in the database to check whether the newly applied resources exceed the maximum resource usage limit. This can prevent a task from occupying too many resources and affecting the execution of other tasks.
[0052] The external resource management module is used to receive the resource request instruction forwarded by the product-side interface module, adjust the system resource status according to the resource request instruction, and return the system resource status information to the product-side interface module.
[0053] In this step, the external resource management module can create new containers or delete redundant containers in the system based on the resource request instructions forwarded by the product-side interface module, and return the creation or deletion results to the product-side interface module so that the product-side interface module can know the execution status of the resource request instructions.
[0054] In the embodiment of the present disclosure, the external resource management module is used to receive resource request instructions forwarded by the product-side interface module; in response to the resource request instruction being a resource request instruction requesting capacity expansion, a container is created in the system based on the resource request instruction; in response to the resource request instruction being a resource request instruction requesting capacity reduction, the container in the system is deleted based on the resource request instruction to release system resources; and the resource status information of the system is returned to the product-side interface module according to a preset time period.
[0055] In a specific implementation, after receiving the resource request instruction forwarded by the product-side interface module, the external resource management module can create or delete a container according to the parameters in the resource request instruction. Then, the external resource management module can also periodically return the current resource status information to the product-side interface module, such as whether the container is successfully created, whether it is in normal operation, and so on. For a new container that is successfully created, its Future can be set to a completed state so that task deployment can continue within the stream processing framework; for a container that fails to be created, its Future can be marked as an exception, and the steps of container creation can be re-executed. The perception method of creation failure and the steps of re-creating the container have been explained above and will not be repeated here. For containers that need to execute deletion instructions, you can first check whether the container status is idle, and then you can combine the resource usage in the current system to determine whether to perform the deletion operation on the container to ensure that the tasks currently being executed in the system will not have insufficient execution resources after deleting the container. Alternatively, when executing a container deletion instruction, a grayscale deletion method can be used. For example, if a resource request instruction indicates that the four containers in the current system are needed, they can be deleted one by one. After deleting each container, it can be determined whether the next container needs to be deleted based on the resource usage in the current system to ensure that all tasks have sufficient execution resources.
[0056] The above has described the steps of creating or deleting containers in the system using the resource management system provided by the embodiment of the present disclosure. In addition, the embodiment of the present disclosure can also monitor the status of each container resource in the system in real time through the query process, and record it as different container lists to achieve efficient management of resources in the system.
[0057] In the embodiment of the present disclosure, the system also includes a storage module; the storage module is used to obtain status information of each container in the system from the internal resource management module, and based on the status information, generate and store different container lists to record the status information of each container in the system.
[0058] In specific implementation, since the embodiment of the present disclosure adopts the interface calling method to create and delete container resources, and the creation and deletion of container resources requires a certain amount of time to process, the call of the interface only indicates that the other end receives the request and processes it, and does not mean that the processing is completed. Therefore, the embodiment of the present disclosure can store the status of each container resource by creating different container lists, so as to ensure that the containers in the system should be in normal operating state. When obtaining the status of each container, the internal resource management module can achieve docking with the external resource management module through the product-side interface module. In this process, the internal resource management module can generate corresponding instructions and forward them to the product-side interface module, and the product-side interface module can forward the instructions to the external resource management module, and then receive the container status information returned by the external resource management module based on the above instructions, and return it to the internal resource management module.
[0059] In a possible implementation, the container list may include a list of containers requested to be created. When a resource request instruction for requesting capacity expansion is submitted, the internal resource management module may record all container information requested to be created, such as resource encoding, request implementation, etc. Based on this information, a list of containers requested to be created may be obtained, in which the container information and status information of each container resource requested to be created may be recorded to determine whether the container is successfully created; in addition, the container creation timeout detection mechanism mentioned above may also be executed based on the container status information recorded in the list.
[0060] In another possible implementation, the container list may also include a list of containers requested to be deleted. The list of containers requested to be deleted may store information about each container for which the shrinking process has been called. Since container deletion takes a certain amount of time, there may be a situation where the container requested to be deleted still exists when the container list is queried next time. In this case, the list can ensure that the deletion process is not called repeatedly. In addition, in order to avoid the storage space of the list from being too large after the system has been running for a long time, the information currently stored in the list of containers requested to be deleted can be cleared after the list has not been updated for a specified period of time.
[0061] In another possible implementation, the container list may also include a full list of system containers, which may store information of all containers in the current system that should be in normal operation, so as to compare with the queried container list and process abnormal containers in a timely manner.
[0062] In another possible implementation, the container list may also include a list of containers that do not exist after multiple queries. Specifically, the list may be generated by the following steps: the internal resource management module is further used to compare the queried running container information with the container information that should be in a normal operating state in the full list of system containers; in response to different comparison results, a list of containers that do not exist after multiple queries is generated based on the comparison results.
[0063] In a specific implementation, when querying the container list, there may be some container information that should be in a normal state but cannot be queried in the full list of system containers, such as containers that have exited abnormally. At this time, it is necessary to delete the abnormal container and apply for a new container to ensure that the tasks in the system can be executed normally. However, since the system may have potential abnormal problems such as abnormal network fluctuations, these containers may still be in normal operation, but they are not queried temporarily. Therefore, a maximum tolerance number can be set. When a container is still not queried after exceeding the maximum tolerance number of queries, the container can be regarded as an abnormally exited container. For such a container, it can be recorded in the list of containers that do not exist in multiple queries, so that the containers recorded in it can be deleted based on the list, thereby ensuring the stability of the system.
[0064] It should be noted that the above-mentioned different container lists and methods for generating various container lists are all exemplified as feasible implementation methods in the embodiments of the present disclosure. In actual applications, other container lists can be set according to needs to implement corresponding functions, and the above-mentioned various lists can also be selected or combined. The embodiments of the present disclosure do not specifically limit this.
[0065] Figure 2 An exemplary system architecture 200 of a resource management system to which the present disclosure may be implemented is shown.
[0066] like Figure 2 As shown, the system architecture 200 may include terminal devices 201, 202, 203, a network 204 and a server 205. The network 204 is used to provide a medium for communication links between the terminal devices 201, 202, 203 and the server 205. The network 204 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0067] Users can use terminal devices 201, 202, 203 to interact with server 205 through network 204 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 201, 202, 203, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).
[0068] The terminal devices 201 , 202 , and 203 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.
[0069] The server 205 may be a server that provides various services, such as a background management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 201, 202, and 203. The background management server may process the received resource management request and feed back the processing result (only an example) to the terminal device.
[0070] It should be noted that the resource management system provided in the embodiment of the present invention is generally executed by the server 205, and accordingly, the resource management system is generally set in the server 205. The resource management system provided in the embodiment of the present invention can also be executed by the terminal devices 201, 202, and 203, and accordingly, the resource management system can be set in the terminal devices 201, 202, and 203.
[0071] It should be understood that Figure 2 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.
[0072] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0073] The modules involved in the embodiments of the present invention may be implemented in software or hardware, and the modules described may also be arranged in a processor.
[0074] Finally, it should be noted that the above embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. A resource management system, characterized in that: Includes internal resource management module, product interface module and external resource management module; The internal resource management module is used to obtain resource usage when the system executes tasks, generate resource request instructions based on the resource usage, and forward the resource request instructions to the product-side interface module; The product-side interface module is used to receive the resource request instruction forwarded by the internal resource management module, and forward the resource request instruction to the external resource management module; The external resource management module is used to receive the resource request instruction forwarded by the product-side interface module, adjust the resource status of the system according to the resource request instruction, and return the resource status information of the system to the product-side interface module.
2. The system according to claim 1, characterized in that The internal resource management module includes a perception module and an interaction module; The sensing module is used to obtain resource usage of the system when executing the task; in response to the resource usage corresponding to the task indicating that the operating resources are insufficient, a resource request instruction for requesting capacity expansion is generated; in response to the resource usage corresponding to the task indicating that the operating resources are idle, a resource request instruction for requesting capacity reduction is generated; and forwarding the resource request instruction to the interaction module; The interaction module is used to receive the resource request instruction forwarded by the perception module, and call the method corresponding to the request in the resource request instruction to forward the resource request instruction to the product-side interface module.
3. The system according to claim 1, characterized in that The internal resource management module also includes a scheduling module; The scheduling module is used to determine the status information of the container in the system when the interaction module receives the resource request instruction forwarded by the perception module; and generate a container request instruction based on the status information of the container in the system; and forward the container request instruction to the product-side interface module, so that the product-side interface module forwards the container request instruction to the external resource management module.
4. The system according to claim 1, characterized in that The product-side interface module is used to receive the resource request instruction forwarded by the internal resource management module, and forward the resource request instruction to the external resource management module; The resource usage of executing the task is saved, the resource usage of the task is checked, and the resources used to execute the task are adjusted according to the resource usage of the task.
5. The system according to claim 1, characterized in that The external resource management module is used to receive the resource request instruction forwarded by the product-side interface module; in response to the resource request instruction being a resource request instruction requesting capacity expansion, a container is created in the system based on the resource request instruction; In response to the resource request instruction being a resource request instruction requesting capacity reduction, based on the resource request instruction, the container in the system is deleted to release system resources; and the resource status information of the system is returned to the product-side interface module according to a preset time period.
6. The system according to claim 1, characterized in that The internal resource management module is also used to record the creation time of the external resource management module to create a container in the system. In response to the creation time exceeding a preset time threshold, the status information of the container is updated to creation failure, and the data used to create the container is deleted, and the container is re-created in the system based on the resource request instruction.
7. The system according to claim 1, characterized in that The system also includes a storage module; The storage module is used to obtain the status information of each container in the system from the internal resource management module, and based on the status information, generate and store different container lists to record the status information of each container in the system.
8. The system according to claim 7, characterized in that The container list includes a list of containers requested to be created, a list of containers requested to be deleted, a list of all system containers, and a list of containers that do not exist after multiple queries.
9. The system according to claim 8, characterized in that The full list of system containers includes container information that should be in a normal operating state; the internal resource management module is also used to compare the queried running container information with the container information that should be in a normal operating state in the full list of system containers; in response to the comparison results being different, a list of containers that do not exist in the multiple queries is generated based on the comparison results.
10. The system according to claim 2, characterized in that The product-side interface module is further used to verify the resource application amount in the resource request instruction for requesting capacity expansion according to the resource application amount in the resource request instruction for requesting capacity expansion and the resource usage of the system.