Online service providing system based on go-tensorflow

Abstract

The application relates to an online service providing system, method, computer readable storage medium and computer program product based on go-tensorflow, which comprises an execution engine, a Redis cluster and an HDFS cluster. The execution engine obtains service model data of an online service from the Redis cluster, parses model identifiers of multiple algorithm models called by the online service, and forms a topology structure of calling paths of the algorithm models. The execution engine obtains algorithm model data from the HDFS cluster according to the model identifiers, combines the topology structure to build a model instance in the memory, traverses each algorithm model in the model instance, and executes each algorithm model based on go-tensorflow by using a coroutine, so that the performance of the online service is greatly improved, part of hardware resources is saved, and an efficient and feasible solution is provided for online services of a power grid service platform.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of power grids, in particular to an online service providing system and method based on go-tensorflow. BACKGROUND

[0002] There are many services in the service middle platform of a power grid, and some services introduce machine learning and deep learning technologies. Most applications containing machine learning and deep learning need to call machine learning and deep learning, which involves model calling of MachineLearning and DeepLearning.

[0003] Traditional online services are implemented by using tf-servering, and the performance is low. However, the calling of machine learning and deep learning involved in the online services of the power grid has a higher requirement on the performance of the services. At present, the online services of the power grid are becoming more and more complex, one request may contain the calling of multiple machine learning and deep learning models, and the traditional tf-servering cannot meet the use requirement. SUMMARY

[0004] Therefore, it is necessary to provide an online service providing system and method based on go-tensorflow, a computer readable storage medium and a computer program product in view of the above technical problems.

[0005] In a first aspect, the application provides an online service providing system based on go-tensorflow, which comprises an execution engine, a Redis cluster and an HDFS cluster, the execution engine comprising a model building module and a model execution module.

[0006] The execution engine obtains service model data of the online service from the Redis cluster through the model building module, and obtains model identifiers of multiple algorithm models called by the online service and a topology formed by calling paths of the algorithm models by analysis.

[0007] The execution engine obtains algorithm model data of the algorithm models from the HDFS cluster according to the model identifiers through the model building module, and builds model instances in the memory in combination with the topology.

[0008] The execution engine traverses the algorithm models in the model instances through the model execution module, and executes the algorithm models based on go-tensorflow by using a coroutine.

[0009] In one of the embodiments, the system further comprises a configuration module: the configuration module is configured to acquire configuration information of the online service, and synchronize service model data in a predetermined format processed from the configuration information to the Redis cluster; wherein the configuration information of the online service indicates model identifiers of various algorithm models called by the online service in a service providing process and a calling path of the various algorithm models.

[0010] In one of the embodiments, the configuration module is further configured to update the service model data with updated configuration information and synchronize the service model data to the Redis cluster when detecting that the configuration information of the online service has changed.

[0011] In one of the embodiments, the model building module comprises a synchronization unit, an instance storage unit and a loading unit, the synchronization unit is configured to acquire service model data of the online service from the Redis cluster, the loading unit is configured to acquire algorithm model data of various algorithm models from the HDFS cluster according to the model identifiers, and the instance storage unit is configured to store the built model instances; the synchronization unit is configured to detect whether the service model data in the Redis cluster has changed every predetermined time, and acquire updated service model data from the Redis cluster and build updated model instances when the service model data has changed, the instance storage unit stores the model instances currently in use and temporarily stores the updated model instances.

[0012] In one of the embodiments, the model execution module traverses in sequence from the topmost layer to the bottommost layer in the model instances, determines that the algorithm model of the topmost layer satisfies a trigger condition and starts a coroutine to execute the algorithm model of the topmost layer when starting the traversal; in the traversal process, determines that the algorithm model of a child node of the algorithm model that has been executed satisfies the trigger condition after executing each algorithm model, and starts a plurality of coroutines to execute the algorithm models satisfying the trigger condition, and in the traversal process, all the algorithm models satisfying the trigger condition are executed simultaneously by using different coroutines.

[0013] In one of the embodiments, the system maintains a double-linked list in the memory, each node in the double-linked list is a model instance, and when the length of the double-linked list is greater than a threshold, a plurality of model instances on the double-linked list are deleted and stored on the disk.

[0014] In one of the embodiments, the system further comprises a model management module, the model management module is configured to perform a modification operation on the algorithm model data according to a modification instruction, the modification operation comprises adding, updating and deleting; when the model management module receives a modification instruction indicating deletion of the algorithm model data, and determines that the algorithm model data to be deleted is not called by the online service, the algorithm model data is deleted.

[0015] In a second aspect, the application further provides a go-tensorflow-based online service providing method, which comprises:

[0016] obtaining service model data of the online service from the Redis cluster, and parsing to obtain model identifiers of a plurality of algorithm models called by the online service, and a topology formed by calling paths of the algorithm models;

[0017] obtaining algorithm model data of the algorithm models from the HDFS cluster according to the model identifiers, and building model instances in the memory in combination with the topology;

[0018] traversing the algorithm models in the model instances, and executing the algorithm models based on go-tensorflow using coroutines.

[0019] In one of the embodiments, the method further comprises: obtaining configuration information of the online service, and synchronizing the configuration information processed into service model data in a predetermined format to the Redis cluster; wherein the configuration information of the online service indicates model identifiers of the algorithm models called by the online service in the service providing process and calling paths of the algorithm models.

[0020] In one of the embodiments, the method further comprises: when detecting that the configuration information of the online service changes, updating the service model data using the updated configuration information and synchronizing to the Redis cluster.

[0021] In one of the embodiments, the method further comprises: detecting whether the service model data in the Redis cluster changes every predetermined time, and when the service model data changes, obtaining the updated service model data from the Redis cluster and building an updated model instance for temporary storage.

[0022] In one of the embodiments, traversing the algorithm models in the model instances, and executing the algorithm models based on go-tensorflow using coroutines, comprises: traversing in order from the topmost layer to the bottommost layer in the model instances, determining that the algorithm model of the topmost layer satisfies a trigger condition and starting a coroutine to execute the algorithm model of the topmost layer when starting the traversal; in the traversal process, after executing each algorithm model, determining that the algorithm models which are child nodes of the algorithm model of the current execution completion satisfy the trigger condition, and starting a plurality of coroutines to execute the algorithm models satisfying the trigger condition respectively, and in the traversal process, using different coroutines to execute all the algorithm models satisfying the trigger condition simultaneously.

[0023] In one embodiment, when the length of the double-linked list maintained in the memory of the system is greater than a threshold, a number of model instances on the double-linked list are deleted and stored on the disk, each node in the double-linked list being a model instance.

[0024] In one embodiment, the method further includes: performing a modification operation on the algorithm model data according to the modification instruction, the modification operation including adding, updating, and deleting; when the model management module receives a modification instruction indicating deletion of the algorithm model data and determines that the algorithm model data to be deleted is not called by the online service, deleting the algorithm model data.

[0025] In a fourth aspect, the present application further provides a computer readable storage medium. The computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the method of the second aspect.

[0026] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and the computer program is executed by a processor to implement the method of the second aspect.

[0027] The online service providing system, method, computer readable storage medium, and computer program product based on go-tensorflow described above introduce a method for implementing machine learning and deep learning model calling based on go-tensorflow, and use a topological sorting method to perform complex model calling, which greatly improves the performance of online services and saves part of hardware resources, and provides an efficient and feasible solution for online services of a power grid service platform. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 A system framework structure diagram of the online service providing system based on go-tensorflow in one embodiment.

[0029] Figure 2 An information flow direction diagram of the online service provided by the online service providing system in one embodiment.

[0030] Figure 3 An information flow direction diagram of the online service provided by the online service providing system in another embodiment.

[0031] Figure 4 A code schematic diagram of part of the configuration information of the online service in one instance.

[0032] Figure 5 A DAG schematic diagram of the calling path of the algorithm model of the online service in one instance and a schematic of the service model data in a predetermined format.

[0033] Figure 6 A structural diagram of a model building module in an execution engine in an embodiment.

[0034] Figure 7 A flowchart of a method for providing an online service based on go-tensorflow in an embodiment. DETAILED DESCRIPTION

[0035] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0036] The overall architecture of the online service providing system based on go-tensorflow provided by the embodiments of the present application is shown in Figure 1 The system includes an execution engine, a Redis cluster and an HDFS cluster. The execution engine is the core module of the system, including a model building module and a model execution module. The Redis cluster and the HDFS cluster are modules for data storage in the system. Optionally, the system also includes a database such as Mysql for providing data storage functions. In addition, the system also includes a GPU (Graphics Processing Unit) to provide necessary technical support. In addition, the system also includes various APIs (Application Programming Interface) and UIs (User Interface) provided externally. The Redis cluster and the HDFS cluster can be integrated with other modules on the same hardware structure, or can run on separate hardware structures. Therefore, the system can be implemented by a terminal, or by a server or a server cluster composed of multiple servers, or can be implemented through the interaction of a terminal and a server.

[0037] Each module in the system can be implemented in whole or in part by software, hardware and combinations thereof. Each module described above can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0038] The execution engine is the core module of the system. In an embodiment, the process of providing an online service by using the system includes the following processes, please refer to the information flow diagram shown in Figure 3

[0039] ​The execution engine obtains the service model data of the online service from the Redis cluster through the model building module, and parses the model identifiers of the multiple algorithm models called by the online service and the topology formed by the calling paths of the algorithm models. The algorithm model here is a machine learning model or a deep learning model.

[0040] The execution engine obtains the algorithm model data of the algorithm models from the HDFS cluster according to the model identifiers through the model building module, and builds the model instances in the memory in combination with the topology.

[0041] The execution engine traverses the algorithm models in the model instances through the model execution module, and executes the algorithm models based on go-tensorflow using coroutines. Go-tensorflow is a go language version of tensorflow open sourced by Google, and the application uses a feature of go language, i.e., coroutine. Coroutine is a very powerful technology. Coroutine is a technology similar to thread, but it consumes much less resources than thread.

[0042] Based on the above embodiment, the execution engine uses go-tensorflow to execute the algorithm models in the online service, which is high in execution efficiency, and uses the topology sorting method to realize the asynchronous calling and execution of the models, which greatly shortens the latency of the online service.

[0043] I. Configuration of the online service.

[0044] In an embodiment, the system further provides an online configuration function of the online service before the execution engine obtains the service model data and executes. That is, in actual application, the developer can configure the complex model calling process contained in the online service through a Web page. Optionally, as shown in Figure 1 The system further includes a configuration module. As shown in Figure 3 The configuration module is used to obtain the configuration information of the online service, i.e., the configuration information when the user configures the model calling process.

[0045] The configuration information of the online service indicates the model identifiers of the algorithm models called by the online service in the service providing process and the calling paths of the algorithm models, i.e., the flow of the online service. The calling path can be represented by DAG (Directed Acyclic Graph), each algorithm model is represented by a model identifier and implemented as a node in the DAG. When the execution of an algorithm model depends on another algorithm model, the algorithm model is a child node of the algorithm model it depends on. Each algorithm model can have zero or several parent nodes and zero or several child nodes. If an algorithm model has no parent node, it means that it does not depend on other models and can be called a top node.

[0046] Referring to Figure 4 The code of part of the configuration information of the example shown in FIG. 8, the SetDAGCondition function is used to set the configuration state of the service flow. When the service flow is configured, the SetDAGCondition function sets the configuration flag flag to true, and after the service flow is configured, the configuration flag flag is set to false. The modellist is an array used to store data of the Model type. The SelectModel function selects an algorithm model each time, and calls the addFather function to specify the parent node of the algorithm model. The calling path between the algorithm models is configured in turn.

[0047] After the online service is configured, the configuration module processes the configuration information into service model data in a predetermined format, and then synchronizes the service model data to the Redis cluster. The predetermined format includes but is not limited to Yaml, XML and Json. The service model data in the predetermined format records the calling path between the algorithm models, and can restore the DAG configured by the user. For example, in a typical example, the configuration information indicates that the online service configured by the user includes algorithm models A, B, C, D and E, and the DAG of the topological structure formed by the calling path is as shown in FIG. 9. Figure 5 As shown in FIG. 9, the algorithm model A is at the topmost layer and is the parent node of the algorithm models B, C and D, the algorithm models B, C and D are all parent nodes of the algorithm model E, and the algorithm model E is at the bottommost layer. When the configuration information is processed into service model data in the Json format, the Json record field has the Father and Name fields, which can be used to record the parent-child node relationship between the algorithm models, thereby completely recording the calling path.

[0048] In another embodiment, the configuration module is also used to detect whether the configuration information of the online service changes. Typically, the configuration information is detected whether it changes every predetermined time interval. If it is detected that the configuration information of the online service changes, the service model data is updated using the updated configuration information and is synchronized to the Redis cluster. That is, when the online service changes, the system can detect the data change in real time and update it.

[0049] II. Management of algorithm model data

[0050] All online services need to use algorithm models, so the management of algorithm model data is the cornerstone of online services. Therefore, in an embodiment, the system can also include a model management module, as shown in FIG. 10. Figure 1 As shown in FIG. 10, the model management module is used to perform modification operations on the algorithm model data according to modification instructions. The modification operations include adding, updating and deleting, as shown in FIG. 11. Figure 3 As shown in FIG. 11.

[0051] When the model management module receives a modification instruction indicating to increase algorithm model data, the algorithm model is registered in the system, and the algorithm model data is stored in the HDFS cluster and stored corresponding to the model identification of the algorithm model.

[0052] When the model management module receives a modification instruction indicating to update algorithm model data, the algorithm model data already stored in the HDFS cluster is updated, such as when updated to a new version.

[0053] When the model management module receives a modification instruction indicating to delete algorithm model data, and determines that the algorithm model data to be deleted is not called by the online service, the algorithm model data is deleted. If the algorithm model data to be deleted is in the process of being called by the online service, the modification instruction is not responded to, and waiting until the condition is met.

[0054] III. Execution of online services.

[0055] 1. Model instance building.

[0056] Model instance building is the basis of the execution engine, and the system needs to use the model building module to construct the model instance in advance in the memory. In an instance, as shown in Figure 6 , the model building module includes a synchronization unit (Syn_model), an instance storage unit (Model_Map), and a loading unit (ModelLoading).

[0057] The synchronization unit is used to obtain the service model data of the online service from the Redis cluster, which is the data introduced in the first part above that is synchronized to the Redis cluster by the configuration module. The synchronization unit is used to complete the online and offline of the online service. After the synchronization unit obtains the service model data of the online service from the Redis cluster, it uses algorithms such as BFS to parse the service model data, obtains the model identification of the multiple algorithm models called by the online service, and the topology formed by the calling path of each algorithm model, and then loads it into the memory. The topology formed by each model identification is the DAG constructed by the calling path as shown in Figure 5 .

[0058] The loading unit is responsible for loading the algorithm model data. The synchronization unit only loads the topological order between the algorithm models, but the algorithm model data of each algorithm model needs to be used specifically when the online service is executed. Therefore, the algorithm model data of each algorithm model, that is, the data maintained in advance by the model management module as described in the second part above, also needs to be loaded from the HDFS cluster. The algorithm model data and the model identifier of the algorithm model are stored correspondingly in the HDFS cluster, and therefore the algorithm model data can be read by using the model identifier. Thus, the topological structure formed by the calling paths of the algorithm models called by the online service and the algorithm model data of each algorithm model are obtained, so that the model instance can be built in the memory.

[0059] The instance storage unit is used to store the built model instance, which is the model instance being used.

[0060] In an embodiment, as described in the configuration module part above, the calling path of the online service may change during the provision process, and the configuration module will be updated synchronously to the Redis cluster. Correspondingly, the model building module will detect whether the service model data in the Redis cluster changes every predetermined time through the synchronization unit, and when the service model data changes, the updated service model data is obtained from the Redis cluster and the updated model instance is built. When the configuration module changes the service model data of the online service, the execution engine can detect the change of the online service within a maximum of two time intervals. When a new model identifier appears in the updated service model data, the loading unit downloads the algorithm model data corresponding to the new model identifier from the HDFS cluster.

[0061] In another embodiment, as described in the model management module part above, the algorithm model data of the algorithm model called by the online service may also change during the provision process of the online service, and correspondingly, the synchronization unit also detects whether the algorithm model data in the HDFS cluster changes regularly. When it is determined that the algorithm model data of the algorithm model called by the online service changes, the loading unit downloads the algorithm model data corresponding to the new model identifier from the HDFS cluster.

[0062] In an embodiment, the instance storage unit temporarily stores the updated model instance in addition to the model instance being used. When the online service changes, only switching between the two stored model instances is needed to complete the switching.

[0063] 2. Iterative execution of the model instance.

[0064] The model execution module traverses each algorithm model in the model instance and executes each algorithm model based on go-tensorflow using a coroutine. The model execution module traverses in order from the topmost layer to the bottommost layer in the model instance. The algorithm model in the topmost layer is the algorithm model without a parent node, and the algorithm model in the bottommost layer is the algorithm model without a child node.

[0065] When starting the traversal, it is determined that the algorithm model in the topmost layer satisfies the trigger condition and a coroutine is started to execute the algorithm model in the topmost layer. During the traversal, after each algorithm model is executed, it is determined that the algorithm model that is the child node of the algorithm model that is currently executed satisfies the trigger condition, and a number of coroutines are started to execute the algorithm models that satisfy the trigger condition.

[0066] That is, the algorithm model without a parent node dependency relationship is first started to be called and executed, and an algorithm model is executed after the algorithm model on which it depends is executed. After a coroutine executes an algorithm model, the execution state of the child node is triggered. In order to improve the performance of the online service, during the traversal, different coroutines are used to simultaneously execute all algorithm models that satisfy the trigger condition.

[0067] Specifically, an array runninglist is used to store the algorithm models being executed, and an array finished is used to store the algorithm models that are executed. As described above, modellist stores the topological relationship between all algorithm models in the online service. When starting the traversal, the InitModelList function starts the algorithm models without parent nodes in modellist and stores them in runninglist. If modellist is empty, the operation ends. The CheckModelCondition function is used to obtain the algorithm model being executed in runninglist, and if the algorithm model is not null, it is added to finished. The CheckChildModel function detects the child node of the algorithm model, and if the node itself satisfies the running requirement, it is added to runninglist. The operation is repeated until runninglist and modellist are empty, and the function is executed.

[0068] In one embodiment, in order to support more online access of the online service of the application, the model instance is optimized in memory, the execution of the algorithm model of machine learning and deep learning is relatively time-consuming, a certain number of models need to be maintained in an online service, in order to ensure the stable operation of the system and maximize the resource utilization of the node. The implementation of the memory optimization of the execution engine refers to the memory optimization algorithm LRU, that is, the system maintains a doubly linked list in memory, and each node in the doubly linked list is a model instance. Each node in the doubly linked list also maintains the number of online services of the model call and the last scheduling time. When the length of the doubly linked list is greater than the threshold, a number of model instances on the doubly linked list are deleted and stored on the disk.

[0069] Based on the same inventive concept, the embodiment of the application also provides a go-tensorflow-based online service providing method executed by the above-mentioned go-tensorflow-based online service providing system. The method is implemented by the system and is specifically performed by coordinating each module in the system. The implementation scheme for solving the problem provided by the method is similar to the implementation scheme described in the system, so the specific limitations in one or more method embodiments provided below can refer to the limitations of the go-tensorflow-based online service providing system described above, and will not be repeated here.

[0070] In order to verify that the system of the application can indeed improve the performance of the online service, the application tests and experiments from various aspects with the following examples, and the experimental data comes from the data set provided by the power grid service center. The data set source is related to the real business scenario, and part of the data has been encrypted. The size of the data set is about 7G. In addition, the experimental environment of the online service of the application is also important. In this paper, a 16GB, 8-core, 100GB storage server is used. Docker version 18.09 is installed on the server. Redis cluster and Mysql cluster environment are simulated using Docker containers. The tf-servering service of the conventional technology and the service of the application are also built in the form of containers. The test is mainly carried out from three aspects, which are the function test, the precision test and the performance test of the application.

[0071] I. Function test, the focus of this part of the test is to determine whether each module of the system of the application is running normally.

[0072] In order to test whether the function of the online service meets the design requirements, the present application mainly tests the function from three aspects.(1) The function test of the configuration module. The test of the configuration module mainly includes two aspects of model page configuration function and model synchronization function.(2) The function test of the engine. The test of the engine is composed of three parts, the function test of the synchronization unit, the test of the loading unit, and the test of the model execution module.(3) The function test of the model management module. The function test of the model management module includes two parts of the function test of the model adding, updating and deleting, and the function test of the algorithm model data synchronization. In addition, the system also needs to be tested. If the test is passed, the online service runs normally.

[0073] The measured data shows that each part of the function test is passed, which indicates that the design and implementation of each part of the system are very reasonable. Moreover, the system of the present application passes the unit test, which indicates that the entire operation service of the online service of the present application meets the design requirements.

[0074] II. Precision test, the focus of this part of the test is to determine whether the design of the go-tensorflow based on the present application is the same as the conventional tf-Servering method.

[0075] The accuracy of the service of the present application is mainly tested from two aspects, namely single node test (only one algorithm model) and model DAG test (multiple algorithm models constitute a topological order). In order to test the simplicity. Since the model DAG test is more accurate than the single node test, this paper only tests the model DAG. The same input is respectively input to the conventional tf-Servering online service and the input to the online service of the system of the present application, and the test case of the precision test is 1000 inputs. The test results are shown in the following table. The final measured results, the output of the online service of the system of the present application in 1000 test cases is the same as the output of the conventional tf-Servering online service, which indicates that the precision of the service of the present application meets the load requirements. The input is the same as the conventional power grid processing method, and the output obtained is 100% the same, which meets the precision load requirements.

[0076] III. Performance test, the focus of this part of the test is to determine whether the performance of the go-tensorflow based online service of the present application is better than the conventional tf-Servering method.

[0077] The performance of the online service is mainly measured by QPS (Queries Per Second) and average cost (Cost) during allocation. In order to test the performance of the service of the present application, the performance of the present application is verified by using an online service of two algorithm models, an online service of three algorithm models and an online service of four algorithm models. The online service of the present application is built in a Docker container, and the stress test of the present application and the traditional service is carried out during the test. The test results are shown in the following table:

[0078] Conventional approach QPS Cost The present invention QPS Cost 2 nodes 40 38.5 ms 2 nodes 180 13.1 ms 4 nodes 35 44.7 ms 4 nodes 130 15.8 ms 5 nodes 33 48.3 ms 5 nodes 126 16.2 ms

[0079] As shown in the above table, the online service of the present application has made great progress in QPS and average time consumption compared with the traditional online service of power grid. Therefore, the present application provides performance and can meet the design requirements.

[0080] In one embodiment, as shown in Figure 7 a go-tensorflow-based online service providing method is provided, comprising the following steps:

[0081] Step 702, obtaining service model data of the online service from the Redis cluster, and parsing to obtain model identifiers of multiple algorithm models called by the online service, and a topology formed by calling paths of the algorithm models.

[0082] Step 704, obtaining algorithm model data of each algorithm model from the HDFS cluster according to the model identifiers, and combining the topology to build a model instance in the memory.

[0083] Step 706, traversing each algorithm model in the model instance, and executing each algorithm model based on go-tensorflow using a coroutine.

[0084] In another embodiment, the method further comprises: obtaining configuration information of the online service, and processing the configuration information into service model data in a predetermined format and synchronizing to the Redis cluster; wherein the configuration information of the online service indicates model identifiers of each algorithm model called by the online service during service providing and calling paths of the algorithm models.

[0085] In another embodiment, the method further comprises: when it is detected that the configuration information of the online service changes, updating the service model data using the updated configuration information and synchronizing to the Redis cluster.

[0086] In another embodiment, the method further comprises: detecting whether the service model data in the Redis cluster changes every predetermined time, and when the service model data changes, obtaining the updated service model data from the Redis cluster and building the updated model instance for temporary storage.

[0087] In another embodiment, each algorithm model in the model instance is traversed, and each algorithm model is executed based on go-tensorflow using a coroutine, including: traversing in sequence from the top layer to the bottom layer in the model instance, when starting the traversal, determining that the algorithm model of the top layer meets the triggering condition and starting a coroutine to execute the algorithm model of the top layer; in the traversal process, after each algorithm model is executed, determining that the algorithm model which is the child node of the algorithm model of the current execution completion meets the triggering condition, and starting several coroutines to execute the algorithm models meeting the triggering condition respectively, and in the traversal process, all algorithm models meeting the triggering condition are executed simultaneously using different coroutines.

[0088] In another embodiment, when the length of the double-linked list maintained in the memory of the system is greater than a threshold value, a number of model instances on the double-linked list are deleted and stored on the disk, and each node in the double-linked list is a model instance.

[0089] In another embodiment, the method further comprises: performing a modification operation on the algorithm model data according to the modification instruction, the modification operation including adding, updating and deleting; when the model management module receives a modification instruction indicating to delete the algorithm model data, and determines that the algorithm model data to be deleted is not called by the online service, the algorithm model data is deleted.

[0090] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0091] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program is executed by a processor to implement the steps realized by each module in the above system.

[0092] In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps performed by the various modules in the system described above.

[0093] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0094] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0095] Any technical features in the above embodiments can be combined, and for the sake of brevity, not all possible combinations are described above, however, as long as the combinations of technical features do not have contradictions, they shall be considered within the scope of the present disclosure.

[0096] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it shall not be understood as a limitation on the patent scope of the present application. It shall be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these shall be within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A go-tensorflow-based online service providing system, characterized by, The system comprises an execution engine, a Redis cluster and an HDFS cluster, the execution engine comprises a model building module and a model execution module; The execution engine obtains service model data of an online service from the Redis cluster through the model building module, and parses to obtain model identifiers of multiple algorithm models called by online service invocation, and a topology formed by calling paths of each algorithm model; The execution engine obtains algorithm model data of each algorithm model from the HDFS cluster according to the model identifiers through the model building module, and builds a model instance in the memory in combination with the topology; The execution engine traverses each algorithm model in the model instance in order from the top layer to the bottom layer through the model execution module, and executes each algorithm model based on go-tensorflow using coroutines; wherein the algorithm model in the top layer is an algorithm model without a parent node, and the algorithm model in the bottom layer is an algorithm model without a child node; the model execution module traverses in order from the top layer to the bottom layer, determines that the algorithm model in the top layer meets the triggering condition and starts a coroutine to execute the algorithm model in the top layer when starting to traverse; in the traversal process, after executing each algorithm model, it is determined that the algorithm model as the child node of the currently executed algorithm model meets the triggering condition, and a plurality of coroutines are started to execute the algorithm models meeting the triggering condition, and in the traversal process, all algorithm models meeting the triggering condition are executed simultaneously using different coroutines.

2. The system of claim 1, wherein, The system further comprises a configuration module: The configuration module is configured to obtain configuration information of an online service, and process the configuration information into service model data in a predetermined format and synchronize the service model data to the Redis cluster; Wherein, the configuration information of the online service indicates model identifiers of each algorithm model called by the online service in the service providing process and calling paths of each algorithm model.

3. The system of claim 2, wherein: The configuration module is further configured to update the service model data using the updated configuration information and synchronize the service model data to the Redis cluster when detecting that the configuration information of the online service has changed.

4. The system of claim 1, wherein: The model building module comprises a synchronization unit, an instance storage unit and a loading unit, the synchronization unit is configured to obtain service model data of an online service from the Redis cluster, the loading unit is configured to obtain algorithm model data of each algorithm model from the HDFS cluster according to the model identifiers, and the instance storage unit is configured to store the built model instance; and the synchronization unit detects whether the service model data in the Redis cluster has changed every predetermined time, and obtains updated service model data from the Redis cluster and builds an updated model instance when the service model data has changed, the instance storage unit stores the currently used model instance, and temporarily stores the updated model instance.

5. The system of claim 1, wherein, The system maintains a double-linked list in memory, each node in the double-linked list being a model instance, and when the length of the double-linked list is greater than a threshold, a plurality of model instances on the double-linked list are deleted and stored on a disk.

6. The system of claim 1, wherein, The system further comprises a model management module configured to perform a modification operation on the algorithm model data according to a modification instruction, the modification operation including adding, updating, and deleting; When the model management module receives a modification instruction indicating deletion of the algorithm model data and determines that the algorithm model data to be deleted is not called by the online service, the algorithm model data is deleted. 7.A go-tensorflow-based online service providing method, characterized by, The method comprises: obtaining service model data of the online service from a Redis cluster and parsing to obtain model identifiers of a plurality of algorithm models called by the online service and a topology formed by calling paths of the algorithm models; obtaining algorithm model data of the algorithm models from a HDFS cluster according to the model identifiers and building model instances in memory in combination with the topology; traversing the algorithm models in the model instances in order from top to bottom, and executing the algorithm models based on go-tensorflow using coroutines; when starting the traversal, determining that the topmost algorithm model satisfies a trigger condition and starting a coroutine to execute the topmost algorithm model; during the traversal, after executing each algorithm model, determining that an algorithm model serving as a child node of the currently executed algorithm model satisfies the trigger condition and starting a plurality of coroutines to execute the algorithm models satisfying the trigger condition, and during the traversal, using different coroutines to simultaneously execute all the algorithm models satisfying the trigger condition; wherein the topmost algorithm model is an algorithm model without a parent node, and the bottommost algorithm model is an algorithm model without a child node.

8. The method of claim 7, wherein, The method further comprises: obtaining configuration information of the online service and processing the configuration information into service model data in a predetermined format and synchronizing the service model data to the Redis cluster; wherein the configuration information of the online service indicates model identifiers of the algorithm models called by the online service during service provision and calling paths of the algorithm models.

9. The method of claim 8, wherein, The method further comprises: when detecting a change in the configuration information of the online service, updating the service model data using the updated configuration information and synchronizing the service model data to the Redis cluster.

10. The method of claim 7, wherein, The method further comprises: detecting whether the service model data in the Redis cluster changes every predetermined time, and when the service model data changes, obtaining updated service model data from the Redis cluster and building updated model instances for temporary storage.

11. The method of claim 7, wherein, When the length of the double-linked list maintained in the memory of the system is greater than a threshold, a plurality of model instances on the double-linked list are deleted and stored on a disk, each node in the double-linked list being a model instance.

12. The method of claim 7, wherein, The method further comprises: According to the modification instruction, a modification operation is performed on the algorithm model data, and the modification operation includes adding, updating and deleting; when the model management module receives a modification instruction indicating deletion of algorithm model data, and it is determined that the algorithm model data to be deleted is not called by the online service, the algorithm model data is deleted.

Images