Method and device for cluster-based model loading

Abstract

The present invention discloses a method and device for cluster-based model loading, and relates to the field of computer technology. A specific implementation of the method includes: for each cluster, obtaining the index data of each device in the cluster respectively; determining the performance index of the cluster according to the index data of each device in the cluster; determining the model list loaded by each cluster according to the performance index of each cluster, so as to adjust the model loaded by each cluster. This implementation does not require manual intervention, can dynamically adjust the model loaded by each cluster according to the index data of the cluster, and supports custom indicators, which greatly improves the system availability, standardization and flexibility.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for clustering loading models. Background Art

[0002] In the field of artificial intelligence, model files are usually loaded into the local memory of the computer to achieve rapid prediction of model results. In the SaaS (Software-as-a-Service) platform solution, a single system needs to provide services to a large number of users. In most scenarios, each user requires an independent business model to provide model prediction services. The memory of a single computer is limited and cannot support the consumption of memory space on a single computer due to the growing number of models. The common solution is to manually select the cluster to load the new model by referring to some fixed indicators when creating a user, and route the traffic to the correct cluster through the mapping relationship between the model and the cluster.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] 1. Manually selecting a cluster will significantly increase operation and maintenance costs and reduce system availability;

[0005] 2. Cluster traffic and memory indicators change dynamically, and existing technologies cannot support dynamic cluster adjustment;

[0006] 3. The cluster selection algorithms and algorithm input parameters of different business systems may vary. The input parameters of existing technologies are fixed and lack flexibility and standardization. Summary of the invention

[0007] In view of this, an embodiment of the present invention provides a method and device for cluster-based model loading, which does not require human intervention, can dynamically adjust the model loaded by each cluster according to the cluster's indicator data, and supports custom indicators, greatly improving system availability, standardization and flexibility.

[0008] To achieve the above object, according to one aspect of an embodiment of the present invention, a method for clustering model loading is provided, comprising:

[0009] For each cluster, obtain the indicator data of each device in the cluster respectively;

[0010] Determine the performance index of the cluster according to the index data of each device in the cluster;

[0011] A list of models loaded by each cluster is determined according to performance indicators of each cluster to adjust the models loaded by each cluster.

[0012] Optionally, after respectively obtaining the indicator data of each device in the cluster, the method further includes:

[0013] Determine whether there is a message queue corresponding to the device in a preset cache queue; if so, save the indicator data of the device to the message queue; otherwise, create a message queue corresponding to the device in the cache queue, and then save the indicator data of the device to the message queue;

[0014] Before the step of determining the performance index of the cluster according to the index data of each device in the cluster, the method further includes: acquiring the index data of the device from the message queue.

[0015] Optionally, after the step of obtaining the indicator data of the device from the message queue, it also includes: if the acquisition is successful, updating the last pull time of the message queue; otherwise, judging and confirming whether the time difference between the current time and the last pull time of the message queue is greater than a preset time threshold; if greater, deleting the message queue from the cache queue, and deleting the mapping corresponding to the device in the mapping relationship between the model and the cluster.

[0016] Optionally, after the step of determining the model list loaded by each cluster according to the performance indicators of each cluster, the step further includes:

[0017] The mapping relationship between the model and the cluster is updated according to the model list loaded by each cluster, and the updated mapping relationship is pushed to each cluster.

[0018] Optionally, the method of the embodiment of the present invention further includes: when a new model needs to be loaded, determining the cluster for loading the new model according to performance indicators of each cluster, or determining the cluster for loading the new model according to a preset cluster selection algorithm.

[0019] According to a second aspect of an embodiment of the present invention, there is provided a device for clustering loading models, comprising:

[0020] An acquisition module, for each cluster, respectively acquires the indicator data of each device in the cluster;

[0021] A determination module, which determines the performance index of the cluster according to the index data of each device in the cluster;

[0022] The adjustment module determines a list of models loaded by each cluster according to performance indicators of each cluster, so as to adjust the models loaded by each cluster.

[0023] Optionally, the acquisition module is further configured to: after respectively acquiring the indicator data of each device in the cluster,

[0024] Determine whether there is a message queue corresponding to the device in a preset cache queue; if so, save the indicator data of the device to the message queue; otherwise, create a message queue corresponding to the device in the cache queue, and then save the indicator data of the device to the message queue;

[0025] The determination module is further configured to: before the step of determining the performance index of the cluster according to the index data of each device in the cluster, obtain the index data of the device from the message queue.

[0026] Optionally, the determination module is also used to: after the step of obtaining the indicator data of the device from the message queue, if the acquisition is successful, update the last pull time of the message queue; otherwise, determine whether the time difference between the current time and the last pull time of the message queue is greater than a preset time threshold; if greater, delete the message queue from the cache queue, and delete the mapping corresponding to the device in the mapping relationship between the model and the cluster.

[0027] Optionally, the adjustment module is further used for: after the step of determining the model list loaded by each cluster according to the performance indicators of each cluster,

[0028] The mapping relationship between the model and the cluster is updated according to the model list loaded by each cluster; and the updated mapping relationship is pushed to each cluster.

[0029] Optionally, the device of the embodiment of the present invention also includes a selection module, which is used to: when a new model needs to be loaded, determine the cluster used to load the new model according to performance indicators of each cluster, or determine the cluster used to load the new model according to a preset cluster selection algorithm.

[0030] According to a third aspect of an embodiment of the present invention, a system for clustering loading models is provided, including:

[0031] The collector is used to: for each cluster, respectively obtain the indicator data of each device in the cluster; determine the performance indicator of the cluster according to the indicator data of each device in the cluster; determine the model list loaded by each cluster according to the performance indicator of each cluster, so as to adjust the model loaded by each cluster;

[0032] A selector, used to: when a new model needs to be loaded, determine the cluster used to load the new model according to the performance indicators of each cluster, or determine the cluster used to load the new model according to a preset cluster selection algorithm;

[0033] The memory is used to store a data processing algorithm for determining the performance index of the cluster according to the index data of each device in the cluster, the performance index of each cluster, the mapping relationship between the model and the cluster, and the preset cluster selection algorithm.

[0034] According to a fourth aspect of an embodiment of the present invention, there is provided an electronic device for loading a model by cluster, including:

[0035] one or more processors;

[0036] a storage device for storing one or more programs,

[0037] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the embodiment of the present invention.

[0038] According to a fifth aspect of an embodiment of the present invention, there is provided a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method provided in the first aspect of the embodiment of the present invention is implemented.

[0039] An embodiment of the above invention has the following advantages or beneficial effects: by determining the performance index of the cluster according to the index data of each device in the cluster and then determining the model list loaded by each cluster, it is possible to dynamically adjust the model loaded by each cluster without manual intervention, and it is possible to support customized indicators, greatly improving the system availability, standardization and flexibility. By customizing the cluster selection algorithm and dynamic balancing strategy, the system availability, standardization and flexibility can be further improved.

[0040] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0042] Figure 1 is a schematic diagram of the main process of the method for clustering model loading according to an embodiment of the present invention;

[0043] Figure 2 is a schematic diagram of main modules of a device for clustering loading models according to an embodiment of the present invention;

[0044] Figure 3 is a schematic diagram of the main components of a system for cluster loading model according to an embodiment of the present invention;

[0045] Figure 4 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0046] Figure 5 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0048] According to one aspect of an embodiment of the present invention, a method for cluster-based model loading is provided.

[0049] Figure 1 FIG. 1 is a schematic diagram of the main process of the method for loading a model by cluster according to an embodiment of the present invention. Figure 1 As shown, the method for loading a model by cluster includes: step S101, step S102 and step S103.

[0050] In step S101, for each cluster, indicator data of each device in the cluster is obtained respectively.

[0051] A cluster is a group of independent computer devices that are interconnected through a high-speed network. A cluster often contains multiple computer devices.

[0052] Indicator data refers to the indicator value of each indicator. The selection of indicators can be selectively determined according to actual conditions, such as memory usage, CPU usage, number of node requests, etc. This example supports custom indicators. Compared with the fixed indicator parameters in the existing technology, custom indicators can greatly improve the standardization and flexibility of the system.

[0053] Step S102: determining the performance index of the cluster according to the index data of each device in the cluster.

[0054] The algorithm for determining the performance index of the cluster based on the index data of each device in the cluster can be selectively set according to the actual application scenario. For example, for cluster A, based on the index a values ​​of all devices in cluster A, the average index a value of all devices in cluster A is calculated by an averaging algorithm, or the sum of the index a values ​​of all devices in cluster A is calculated by a summing algorithm. This example supports custom data processing algorithms, thereby greatly improving the standardization and flexibility of the system.

[0055] Step S103: determining a list of models loaded by each cluster according to performance indicators of each cluster, so as to adjust the models loaded by each cluster.

[0056] Exemplarily, for cluster A, the number of models loaded by cluster A is determined according to the average index a value of cluster A. Alternatively, all models are allocated according to the performance indicators of each cluster, wherein clusters with higher performance load a larger number of models or models with a larger running load, and clusters with lower performance load a smaller number of models or models with a smaller running load. In actual application, a dynamic balancing strategy can be pre-set to determine the model list loaded by each cluster according to the preset dynamic balancing strategy and the performance indicators of each cluster. The dynamic balancing strategy can be customized.

[0057] The present invention determines the model list loaded by each cluster according to the indicator data of each device in the cluster, can dynamically adjust the model loaded by each cluster without manual intervention, can support customized indicators, and greatly improves the system availability, standardization and flexibility.

[0058] In some embodiments, after obtaining the index data of each device in the cluster respectively, it also includes: determining whether there is a message queue corresponding to the device in the preset cache queue; if so, saving the index data of the device to the message queue; otherwise, creating a message queue corresponding to the device in the cache queue, and then saving the index data of the device to the message queue. Before the step of determining the performance index of the cluster based on the index data of each device in the cluster, it also includes: obtaining the index data of the device from the message queue.

[0059] Exemplarily, the message queue of each device is named "cluster name + device unique identifier (such as IP)". For device b, after obtaining the indicator data of device b, determine whether there is a message queue named "cluster name + device b's unique identifier" in the cache queue. If it exists, save the currently obtained indicator data of device b to the message queue; otherwise, create a message queue named "cluster name + device b's unique identifier", and then save the currently obtained indicator data of device b to the message queue.

[0060] A cache queue is used to save the indicator data of each device in the cluster, which can perform message caching and asynchronous processing to improve system performance.

[0061] Furthermore, after the step of obtaining the indicator data of the device from the message queue, the method may further include: determining whether the indicator data of the device is successfully obtained. If the acquisition is successful, the last pull time of the message queue is updated; otherwise, determining whether the time difference between the current time and the last pull time of the message queue is greater than a preset time threshold; if greater, deleting the message queue from the cache queue, and deleting the mapping corresponding to the device in the mapping relationship between the model and the cluster.

[0062] Exemplarily, the indicator data of each device is pulled from the cache queue in a loop, and the last pull time of the message queue of each device is updated after pulling. For the indicator data of device c, the indicator data of device c is obtained from the message queue c named "cluster name + unique identifier of device c" in the cache queue, and then the last pull time of message queue c is updated. If the indicator data cannot be obtained from the message queue c, it is judged that device c is providing services. At this time, the message queue c named "cluster name + unique identifier of device c" is deleted from the cache queue, and the mapping corresponding to device c in the mapping relationship between the model and the cluster is deleted.

[0063] Through timeout judgment, it is possible to avoid using a device that does not provide a service to load a module, thereby improving the effect of cluster-based module loading.

[0064] Optionally, after the step of determining the model list loaded by each cluster according to the performance indicators of each cluster, the method further includes: updating the mapping relationship between the model and the cluster according to the model list loaded by each cluster, and pushing the updated mapping relationship to each cluster.

[0065] In this example, the mapping relationship between each cluster and the model loaded therein is preset. After the model list loaded by each cluster is determined according to the performance indicators of each cluster, the mapping relationship between the model and the cluster is updated according to the determined list. The updated mapping relationship is pushed to each cluster so that the devices in the cluster can update the loaded model according to the updated mapping relationship in a timely manner.

[0066] Optionally, the method of the embodiment of the present invention further includes: when a new model needs to be loaded, determining the cluster for loading the new model according to the performance indicators of each cluster, or determining the cluster for loading the new model according to a preset cluster selection algorithm. In this example, the new model refers to a model that is not currently loaded in each cluster.

[0067] For example, a blacklist and a whitelist of clusters are preset, where the blacklist is used to indicate that the cluster can no longer load new models, and the whitelist is used to indicate that the cluster must load new models. Of course, the cluster used to load the new model can also be determined based on the performance indicators of each cluster. For example, the cluster with higher performance loads the new model.

[0068] By customizing the cluster selection algorithm, it is possible to determine the cluster used to load new models, further improving system availability, standardization, and flexibility.

[0069] According to a second aspect of an embodiment of the present invention, a device for clustering loading models is provided.

[0070] Figure 2 is a schematic diagram of the main modules of the device for clustering loading model according to an embodiment of the present invention, such as Figure 2 As shown, the device 200 for loading a model by cluster includes:

[0071] The acquisition module 201 acquires the indicator data of each device in each cluster respectively;

[0072] A determination module 202, determining a performance indicator of the cluster according to indicator data of each device in the cluster;

[0073] The adjustment module 203 determines a list of models loaded by each cluster according to the performance indicators of each cluster, so as to adjust the models loaded by each cluster.

[0074] Optionally, the acquisition module is further configured to: after respectively acquiring the indicator data of each device in the cluster,

[0075] Determine whether there is a message queue corresponding to the device in a preset cache queue; if so, save the indicator data of the device to the message queue; otherwise, create a message queue corresponding to the device in the cache queue, and then save the indicator data of the device to the message queue;

[0076] The determination module is further configured to: before the step of determining the performance index of the cluster according to the index data of each device in the cluster, obtain the index data of the device from the message queue.

[0077] Optionally, the determination module is further used to: before the step of obtaining the indicator data of the device from the message queue, confirm that the time difference between the current time and the last pull time of the message queue is less than or equal to a preset time threshold; after the step of obtaining the indicator data of the device from the message queue, update the last pull time of the message queue; and,

[0078] If the time difference between the current time and the last pull time of the message queue is greater than a preset time threshold, the message queue is deleted from the cache queue, and the mapping corresponding to the device in the mapping relationship between the model and the cluster is deleted.

[0079] Optionally, the adjustment module is further used for: after the step of determining the model list loaded by each cluster according to the performance indicators of each cluster,

[0080] The mapping relationship between the model and the cluster is updated according to the model list loaded by each cluster; and the updated mapping relationship is pushed to each cluster.

[0081] Optionally, the device of the embodiment of the present invention also includes a selection module, which is used to: when a new model needs to be loaded, determine the cluster used to load the new model according to performance indicators of each cluster, or determine the cluster used to load the new model according to a preset cluster selection algorithm.

[0082] According to a third aspect of an embodiment of the present invention, a system for implementing the method provided in the first aspect of an embodiment of the present invention is provided.

[0083] Figure 3 is a schematic diagram of the main components of a system for cluster loading model according to an embodiment of the present invention, such as Figure 3 As shown, the system 300 for cluster loading model includes:

[0084] The collector 301 is used to: for each cluster, respectively obtain the indicator data of each device in the cluster; determine the performance indicator of the cluster according to the indicator data of each device in the cluster; determine the model list loaded by each cluster according to the performance indicator of each cluster, so as to adjust the model loaded by each cluster;

[0085] The selector 303 is used to: when a new model needs to be loaded, determine the cluster used to load the new model according to the performance indicators of each cluster, or determine the cluster used to load the new model according to a preset cluster selection algorithm;

[0086] The memory 302 is used to store a data processing algorithm for determining the performance index of the cluster according to the index data of each device in the cluster, the performance index of each cluster, a mapping relationship between a model and a cluster, and a preset cluster selection algorithm.

[0087] Figure 3In the model prediction service provider, the model prediction service provider is equivalent to the device in the cluster. The basic data collection node StatisticNode (collecting memory usage, CPU usage) and the query rate node QpsNode (the number of requests received by the node) in the model prediction service provider are connected in series to form a call chain tree. Users can inherit the DefaultNode abstract class and implement the entry method to develop the indicator data of the custom indicator data collection node. After all indicator data are collected, the message push node PushMsgNode sends the indicator data set collected by the call chain to the collector 301 through an RPC (Remote Procedure Call) call.

[0088] The model loading node LoadModelNode of the model prediction service provider obtains the latest mapping relationship between the model and the cluster from the data storage 302 and loads or updates the model. The entire business logic is completed through a scheduled task.

[0089] The collector 301 mainly includes a Disruptor cache queue and a Graph balanced tree. The Disruptor cache queue is used to achieve the decoupling of producers and consumers. The Graph (a data structure) balanced tree is used as a consumer of the collected indicators to complete the preprocessing, processing and storage of the indicators. The dynamic node can be customized to achieve the function of dynamically adjusting the cluster load during operation.

[0090] When the collector 301 receives the indicator data set sent by the model prediction service provider, it checks whether there is a message queue named by the client "cluster name + IP address" in the Disruptor cache queue. If it exists, the indicator data is directly stored in the corresponding message queue. If it does not exist, a message queue of "cluster name + IP address" is created, the indicator data is stored in the newly created queue, and the mapping relationship between the cluster and the IP address is saved in the Root node of the Graph balanced tree.

[0091] The Root node in the Graph balanced tree uses a scheduled task to execute business logic. First, loop the message queues in the Disruptor cache queue, consume messages from the corresponding message queue, and update the last pull time of the message queue if it is pulled. If it is not pulled, judge whether the current time - the last pull time of the message queue > the set time threshold is correct. If the return is correct, delete the mapping relationship between the cluster and the IP address, and delete the message queue named "cluster name + IP address". Secondly, get the basic parameters corresponding to the indicator data from the data storage (basic parameters refer to the merge strategy, data update strategy, data clearing strategy, etc.) and pass it backward. Finally, call its three child nodes Group, Storage, and Dynamic on the left, middle, and right in sequence.

[0092] The Group node aggregates messages by cluster, and calls its left and right child nodes Average and Add in sequence according to the basic parameters corresponding to the indicator data obtained from the data storage. Average is responsible for calculating the data indicator results of the current Group with the merge strategy of averaging, and Add is responsible for calculating the data indicator results of the current Group with the merge strategy of summing.

[0093] The Storage node will update the index data processed by the cluster to the data storage. The data update strategies are: overwrite / append without clearing / append with scheduled clearing. For each index, there is a corresponding storage strategy, which can be customized.

[0094] The Dynamic node is used to update the mapping relationship between the model and the cluster according to the basic parameters corresponding to the indicator data obtained from the data storage during runtime, so as to achieve dynamic balance of key indicator data such as model call volume and memory usage. Users can implement customized dynamic balancing strategies by inheriting the DynamicNode class and rewriting the execute method.

[0095] The selector 303 provides basic functions such as acquisition of collection indicators and persistence of selection results through a template mode. Users can implement abstract methods to complete the cluster selection business logic. The selector 303 can also serve as a model training platform for training new models. After the new model training is completed, the processed cluster collection indicators and cluster blacklists and whitelists are obtained from the data storage 302, and the user-defined cluster selection algorithm is executed to select the cluster for loading the new model, and the results are updated to the model and cluster mapping relationship in the data storage.

[0096] The data storage 302 may use a nosql (non-relational database) database such as Redis (a key-value storage system) and Zookeeper (a distributed, open source distributed application coordination service), which mainly stores the basic parameters of the collected indicators (merging strategy, data update strategy, data clearing strategy, etc.), processed indicator data, the mapping relationship between the model and the cluster, the cluster blacklist and whitelist, etc. When the mapping relationship between the model and the cluster in the data storage 302 changes, the local cache of the model prediction service consumer is updated through the active push mechanism (Redis Pubsub mechanism / Zookeeper message subscription).

[0097] The model prediction service consumer calls the model prediction services of different clusters according to the mapping relationship between the model and the cluster in the local cache. If there are multiple service providers for a model, polling is used for load balancing. The model prediction service consumer calls the computer node of the model prediction service predictor. When the model prediction service consumer calls the cluster provider, since there are multiple computers in the cluster, it will use polling to call different computers in turn to ensure that the requests can be evenly distributed among the computer nodes in the cluster to avoid overwhelming a certain computer node.

[0098] The existing technology of manually selecting model loading clusters has many shortcomings in terms of system availability, standardization, flexibility, etc. The present invention fully considers the usability, scalability and high performance of the cluster management model. First, a mapping table + timeout detection mechanism is used in the collector for cluster management, which is simple and easy to use; secondly, a large amount of business logic coupled with the business is abstracted to facilitate users to perform customized extensions; finally, Disruptor cache queues and Graph balanced trees are used for message caching and asynchronous processing to ensure high system performance. The present invention realizes a cluster model loading method that supports linear expansion of the number of models without any human intervention, making model cluster management easy.

[0099] According to a fourth aspect of an embodiment of the present invention, there is provided an electronic device for loading a model by cluster, including:

[0100] one or more processors;

[0101] a storage device for storing one or more programs,

[0102] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the embodiment of the present invention.

[0103] According to a fifth aspect of an embodiment of the present invention, there is provided a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method provided in the first aspect of the embodiment of the present invention.

[0104] Figure 4 An exemplary system architecture 400 is shown to which the method for loading a model by clustering or the apparatus for loading a model by clustering according to an embodiment of the present invention can be applied.

[0105] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, 403, network 404 and server 405. Network 404 is used to provide a medium for communication links between terminal devices 401, 402, 403 and server 405. Network 404 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0106] Users can use terminal devices 401, 402, 403 to interact with server 405 through network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, 403, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0107] The terminal devices 401 , 402 , and 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0108] The server 405 may be a server that provides various services, such as a backend management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 401, 402, and 403. The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information, product information - only an example) to the terminal device.

[0109] It should be noted that the method for loading a model by cluster provided in the embodiment of the present invention is generally executed by the server 405 , and accordingly, the device for loading a model by cluster is generally disposed in the server 405 .

[0110] It should be understood that Figure 4 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0111] Reference below Figure 5 , which shows a schematic diagram of the structure of a computer system 500 of a terminal device suitable for implementing an embodiment of the present invention. Figure 5 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0112] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0113] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0114] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present invention are executed.

[0115] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0116] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0117] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be set in a processor. For example, they may be described as: a processor comprising: an acquisition module, for each cluster, respectively acquiring the index data of each device in the cluster; a determination module, determining the performance index of the cluster according to the index data of each device in the cluster; an adjustment module, determining the model list loaded by each cluster according to the performance index of each cluster, so as to adjust the model loaded by each cluster. Among them, the names of these modules do not constitute a limitation on the module itself in some cases. For example, the acquisition module may also be described as "a module for determining the model list loaded by each cluster according to the performance index of each cluster".

[0118] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: for each cluster, obtaining the indicator data of each device in the cluster respectively; determining the performance indicator of the cluster according to the indicator data of each device in the cluster; determining the model list loaded by each cluster according to the performance indicator of each cluster, so as to adjust the model loaded by each cluster.

[0119] According to the technical solution of the embodiment of the present invention, by determining the performance index of the cluster based on the index data of each device in the cluster and then determining the model list loaded by each cluster, it is possible to dynamically adjust the model loaded by each cluster without manual intervention, and it is possible to support customized indicators, greatly improving the system availability, standardization and flexibility. By customizing the cluster selection algorithm and dynamic balancing strategy, the system availability, standardization and flexibility can be further improved.

[0120] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for clustering model loading, characterized in that: include: For each cluster, obtain the indicator data of each device in the cluster respectively; Determine the performance index of the cluster according to the index data of each device in the cluster; Determine a list of models loaded by each cluster according to performance indicators of each cluster to adjust the models loaded by each cluster; The method further comprises: before the step of determining the performance index of the cluster according to the index data of each device in the cluster, obtaining the index data of the device from a message queue; After the step of obtaining the indicator data of the device from the message queue, if the acquisition is successful, the last pull time of the message queue is updated; otherwise, it is determined whether the time difference between the current time and the last pull time of the message queue is greater than a preset time threshold; if greater, the message queue is deleted from the preset cache queue, and the mapping corresponding to the device in the mapping relationship between the model and the cluster is deleted.

2. The method according to claim 1, characterized in that After respectively obtaining the indicator data of each device in the cluster, it also includes: Determine whether there is a message queue corresponding to the device in the preset cache queue; if so, save the indicator data of the device in the message queue; otherwise, create a message queue corresponding to the device in the cache queue, and then save the indicator data of the device in the message queue.

3. The method according to claim 1, characterized in that After the step of determining the model list loaded by each cluster according to the performance indicators of each cluster, the method further includes: The mapping relationship between the model and the cluster is updated according to the model list loaded by each cluster, and the updated mapping relationship is pushed to each cluster.

4. The method according to claim 1, characterized in that Also includes: When a new model needs to be loaded, the cluster used to load the new model is determined according to the performance indicators of each cluster, or the cluster used to load the new model is determined according to a preset cluster selection algorithm.

5. A device for cluster loading model, characterized in that: include: An acquisition module, for each cluster, respectively acquires the indicator data of each device in the cluster; A determination module, which determines the performance index of the cluster according to the index data of each device in the cluster; An adjustment module, determining a list of models loaded by each cluster according to performance indicators of each cluster, so as to adjust the models loaded by each cluster; The determination module is also used for: before the step of determining the performance index of the cluster according to the index data of each device in the cluster, obtaining the index data of the device from the message queue; after the step of obtaining the index data of the device from the message queue, if the acquisition is successful, updating the last pull time of the message queue; otherwise, judging and confirming whether the time difference between the current time and the last pull time of the message queue is greater than a preset time threshold; if greater, deleting the message queue from the preset cache queue, and deleting the mapping corresponding to the device in the mapping relationship between the model and the cluster.

6. The device according to claim 5, characterized in that The acquisition module is also used to: after respectively acquiring the indicator data of each device in the cluster, Determine whether there is a message queue corresponding to the device in the preset cache queue; if so, save the indicator data of the device in the message queue; otherwise, create a message queue corresponding to the device in the cache queue, and then save the indicator data of the device in the message queue.

7. A system for cluster loading models, characterized in that: include: The collector is used to: for each cluster, obtain the indicator data of each device in the cluster respectively; Determine the performance index of the cluster according to the index data of each device in the cluster; determine the model list loaded by each cluster according to the performance index of each cluster to adjust the model loaded by each cluster; A selector, used to: when a new model needs to be loaded, determine the cluster used to load the new model according to the performance indicators of each cluster, or determine the cluster used to load the new model according to a preset cluster selection algorithm; A memory, used to store: a data processing algorithm for determining the performance index of the cluster according to the index data of each device in the cluster, the performance index of each cluster, a mapping relationship between the model and the cluster, and a preset cluster selection algorithm; The collector is also used to: before the step of determining the performance index of the cluster based on the index data of each device in the cluster, obtain the index data of the device from the message queue; after the step of obtaining the index data of the device from the message queue, if the acquisition is successful, update the last pull time of the message queue; otherwise, determine whether the time difference between the current time and the last pull time of the message queue is greater than a preset time threshold; if greater, delete the message queue from the preset cache queue, and delete the mapping corresponding to the device in the mapping relationship between the model and the cluster.

8. An electronic device with a cluster loading model, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images