A method and device for accelerating model reasoning

Abstract

The present disclosure provides a method and device for accelerating model reasoning, the method comprising: obtaining request information and analyzing the request information, determining a target model service with a dependency relationship corresponding to the request information, wherein the integrated model service system comprises at least one model service; determining a model parameter value, processing the model parameter value in turn through at least one target model service with a dependency relationship and a message queue set therebetween, to obtain a final output model calculation result; wherein the message queue between every two target model services with a dependency relationship is used to receive task data generated by the previous target model service according to its own processing capacity value, and send the task data to the latter target model service according to the processing capacity value of the latter target model service. The present disclosure solves the problem of high latency of the target model service and improves the performance of the entire integrated model service system.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method and device for accelerating model reasoning. Background Art

[0002] In the high-concurrency scenario of risk control, different business scenarios require the integration of multiple models to provide services to meet business needs. The existing solutions focus on optimizing the model itself, optimizing the calculation or compressing the model during the reasoning calculation of a single model, etc.; they do not deeply optimize the model processing link, and cannot reduce the overall delay of the system. In the high-concurrency scenario, the delay will become higher and higher. Summary of the invention

[0003] The present invention provides a method and device for accelerating model reasoning, which solves the problem of high latency in target model services and improves the performance of the entire integrated model service system.

[0004] In a first aspect, the present disclosure provides a method for accelerating model reasoning, for an integrated model service system, the method comprising:

[0005] Obtaining request information and analyzing the request information to determine a target model service with a dependency relationship corresponding to the request information, wherein the integrated model service system includes at least one model service;

[0006] Determine model parameter values, and process the model parameter values ​​in sequence through at least one target model service having a dependency relationship and a message queue set therebetween to obtain a final output model calculation result;

[0007] Among them, the message queue between every two target model services with a dependent relationship is used to receive the task data generated by the previous target model service according to its own processing capability value, and send the task data to the latter target model service according to the processing capability value of the latter target model service.

[0008] According to the method for accelerating model reasoning provided by the present disclosure, for every two target model services having a dependency relationship, the model parameter values ​​are processed sequentially via at least one target model service having a dependency relationship and a message queue set therebetween, including:

[0009] According to the processing capacity value of the previous target model service, the task data processed by the previous target model service is stored in the message queue in batches until the task data corresponding to the previous target model service is completely stored;

[0010] According to the processing capacity value of the next target model service, task data of the corresponding data volume is obtained each time for processing until all the task data in the message queue are obtained, and the task data processed by the next target model service is output.

[0011] According to the method for accelerating model reasoning provided by the present disclosure, the integrated model service system includes a unified configuration center component;

[0012] Before obtaining the request information and analyzing the request information, the method further includes:

[0013] The model is processed into a microservice to obtain a model service, and the model service is registered with a unified configuration center component.

[0014] According to the method for accelerating model reasoning provided by the present disclosure, after outputting the task data processed by the latter target model service, the method further includes:

[0015] Periodically obtaining system information of the target model service, and inputting the system information into a trained processing capacity value regression prediction model, and outputting a new first processing capacity value corresponding to the target model service, wherein the trained processing capacity value regression prediction model is trained based on sample system information;

[0016] The new first processing capability value is synchronized to the unified configuration center component, and the processing capability value of the target model service is updated to the new first processing capability value through the unified configuration center component.

[0017] According to the model reasoning acceleration method provided by the present disclosure, when the trained processing capability value regression prediction model does not output a new first processing capability value corresponding to the target model service, after outputting the task data processed by the latter target model service, the method further includes:

[0018] Determine a new second processing capacity value corresponding to the target model service by a dynamic programming algorithm;

[0019] The new second processing capability value is synchronized to the unified configuration center component, and the processing capability value of the target model service is updated to the new second processing capability value through the unified configuration center component.

[0020] According to the method for accelerating model reasoning provided by the present disclosure, the integrated model service system includes a gateway component;

[0021] The obtaining of request information and analyzing the request information to determine the target model service with dependency relationship corresponding to the request information includes:

[0022] Obtaining request information through the gateway component and analyzing the request information to obtain analysis results;

[0023] The analysis result is sent to the unified configuration intermediate component through the gateway component, and the target model service with a dependency relationship corresponding to the request information is determined through the unified configuration center component.

[0024] According to the method for accelerating model reasoning provided by the present disclosure, before determining the model parameter value, the method includes:

[0025] Perform microservice processing on the feature engineering corresponding to the model to obtain feature engineering services;

[0026] Determine a corresponding target feature engineering service through the target model service;

[0027] Determining the model parameter value includes:

[0028] Obtaining raw data corresponding to the target feature engineering service, and inputting the raw data into the target feature engineering service to extract features;

[0029] The feature is processed by the target feature engineering service to determine the model parameter value.

[0030] In a second aspect, the present disclosure provides a device for accelerating model reasoning, which is arranged in an integrated model service system, and the device includes:

[0031] An acquisition module, used to acquire request information and analyze the request information to determine a target model service with a dependency relationship corresponding to the request information; wherein the integrated model service system includes at least one model service;

[0032] A determination module, used to determine a model parameter value, and process the model parameter value in sequence through at least one target model service having a dependency relationship and a message queue set therebetween to obtain a final output model calculation result;

[0033] Among them, the message queue between every two target model services with a dependent relationship is used to receive the task data generated by the previous target model service according to its own processing capability value, and send the task data to the latter target model service according to the processing capability value of the latter target model service.

[0034] In a third aspect, the present disclosure provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a method for accelerating model reasoning as described in any one of the above items when executing the program.

[0035] In a fourth aspect, the present disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for accelerating model reasoning as described in any one of the above items.

[0036] In a fifth aspect, the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the method for accelerating model reasoning as described in any one of the above items.

[0037] The present disclosure provides a method and device for accelerating model reasoning, which obtains request information and analyzes the request information to determine a target model service with a dependency relationship corresponding to the request information; determines a model parameter value, and processes the model parameter value in turn through at least one target model service with a dependency relationship and a message queue set therebetween to obtain a final output model calculation result, wherein the message queue between every two target model services with a dependency relationship is used to receive task data generated by the previous target model service according to its own processing capability value, and sends the task data to the next target model service according to the processing capability value of the next target model service, so that the processing capability value of the previous target model service can be balanced with the processing capability value of the next target model. When the balance is achieved, the entire integrated model service system can be made smoother, the delay of the target model service can be reduced, and the performance of the entire integrated model service system can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the present disclosure or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 It is a flowchart of a method for accelerating model reasoning provided by the present disclosure;

[0040] Figure 2 It is a block diagram of the message queue provided by the present disclosure for processing;

[0041] Figure 3 It is a block diagram of the method for accelerating model reasoning provided by the present disclosure;

[0042] Figure 4 It is a schematic diagram of the structure of a device for accelerating model reasoning;

[0043] Figure 5 It is a schematic diagram of the structure of an electronic device provided by the present disclosure. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the technical solutions in the present disclosure will be clearly and completely described below in conjunction with the drawings in the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0045] In recent years, the in-depth application of machine learning and deep learning in the field of risk control requires models with different characteristics to complete corresponding prediction or calculation tasks for different scenarios. Models with different algorithms and different goals require different data sources as input parameters. In the current specific business, multiple models are required to cooperate to meet a complex scenario. Risk control, as an indispensable part of the overall business, brings extremely high concurrency requirements to the model service as a whole. This requires optimizing each link in the process of inference calculation provided by the model service to reduce the latency of the overall system.

[0046] The solutions in the prior art do not have a dynamic resource adjustment method to solve the model reasoning service optimization solution in the scenario of multiple model integration. The solutions in the prior art focus on optimizing the model itself, computing optimization or model compression in the reasoning calculation process of a single model, or solutions for the full life cycle management of machine learning models simply manage service online and offline, model status, model monitoring, etc., and do not deeply optimize the model processing link, cannot reduce the overall delay of the system, and the delay will become higher and higher in high concurrency scenarios.

[0047] Reference Figure 1 , which is a flow chart of a method for accelerating model reasoning provided by the present disclosure, and is used for integrating a model service system, specifically comprising:

[0048] 110. Obtain request information and analyze the request information to determine a target model service with a dependency relationship corresponding to the request information, wherein the integrated model service system includes at least one model service.

[0049] In this step, a request based on the HTTP protocol can be initiated by the Client, so that the integrated model service system obtains the request information. Among them, HTTP is an application layer protocol for distributed, collaborative and hypermedia information systems.

[0050] Target model services with dependencies refer to target model services that have upstream and downstream relationships.

[0051] 120, determine the model parameter value, and process the model parameter value in sequence through at least one target model service having a dependency relationship and the message queues set therebetween to obtain a final output model calculation result.

[0052] In this step, the model parameter values ​​are input to the target model service for calculation.

[0053] Since the processing capacity values ​​of the target model services having upstream and downstream relationships are different, message queues are set between the target model services having upstream and downstream relationships to balance the processing capacity values ​​of the target model services.

[0054] Specifically, the message queue between every two target model services with a dependency relationship is used to receive task data generated by the previous target model service according to its own processing capability value, and send the task data to the latter target model service according to the processing capability value of the latter target model service.

[0055] The present invention provides a method for accelerating model reasoning, which obtains request information and analyzes the request information to determine a target model service with a dependency relationship corresponding to the request information; determines a model parameter value, and processes the model parameter value in turn through at least one target model service with a dependency relationship and a message queue set therebetween to obtain a final output model calculation result, wherein the message queue between every two target model services with a dependency relationship is used to receive task data generated by the previous target model service according to its own processing capability value, and send the task data to the next target model service according to the processing capability value of the next target model service, so that the processing capability value of the previous target model service can be equal to the processing capability value of the next target model. When equality is achieved, the entire integrated model service system can be made smoother, the delay of the target model service can be reduced, and the performance of the entire integrated model service system can be improved.

[0056] Based on any of the above embodiments, for every two target model services having a dependency relationship, step 120 specifically includes the following steps 121 to 122:

[0057] Step 121, according to the processing capacity value of the previous target model service, the task data processed by the previous target model service is stored in the message queue in batches until the task data corresponding to the previous target model service is completely stored.

[0058] Step 122, according to the processing capacity value of the next target model service, each time the task data of the corresponding data volume is obtained for processing until all the task data in the message queue are obtained, and the task data processed by the next target model service is output.

[0059] For specific steps 121 to 122, refer to Figure 2 , a block diagram for processing the message queue provided by the present invention.

[0060] The processing capability value is batch, which refers to the ability to process multiple request events simultaneously.

[0061] The previous target model serves A, and the next target model serves B. The processing capacity values ​​of A and B are different. For example, the batch of the previous target model serving A is 3, and the batch of the next target model serving B is 40. (In Figure 2 In order to facilitate drawing, the batch of the latter target model service B is 4 for illustration). The capability values ​​of the two target model services differ by an order of magnitude. If the machines cannot be deployed across orders of magnitude in a high-concurrency scenario, the hardware resources of the overall model service cannot be fully utilized. Therefore, a message queue middleware is set between the two target model services. After processing the task, the former target model service A puts the task into the message queue of the latter target model service B. At this time, the latter target model service B can consume 40 tasks of the batch size at one time for service calculation. This can smoothly solve the batch inequality problem between the former target model service A and the latter target model service B.

[0062] Based on any of the above embodiments, after step 122, the method further includes steps 11 to 12:

[0063] Step 11, regularly obtain system information of the target model service, and input the system information into a trained processing capacity value regression prediction model, and output a new first processing capacity value corresponding to the target model service, wherein the trained processing capacity value regression prediction model is trained based on sample system information.

[0064] Step 12: synchronize the new first processing capability value to the unified configuration center component, and update the processing capability value of the target model service to the new first processing capability value through the unified configuration center component.

[0065] In steps 11 to 12, the system information includes machine information and monitoring indicators. The machine information may include memory, CPU utilization per second, etc., and the monitoring indicators may include response delay, throughput, etc.

[0066] The processing capability value regression prediction model can provide a gradient boosting framework for the Xgboost model.

[0067] Since the request information in different time periods has different distributions, the batch of model services in the integrated model service system is dynamically adjusted over time to make full use of hardware resources to improve the overall throughput of the system. The integrated model service system collects machine information and monitoring indicators of each model service in the entire system, and trains the regression prediction Xgboost model with the collected sample machine information and sample monitoring indicators on a regular basis (which can be every hour). After that, the machine information and monitoring indicators are used as input parameters of the regression prediction Xgboost model within a certain period of time (which can be every hour) to infer the real-time batch of each model service in the current integrated model service system, and the batches of different model services are synchronized and effective through a unified configuration center.

[0068] Based on any of the above embodiments, when the trained processing capability value regression prediction model does not output a new first processing capability value corresponding to the target model service, after step 122, the method further includes steps 13 to 14:

[0069] Step 13: Determine a new second processing capacity value corresponding to the target model service through a dynamic programming algorithm.

[0070] Step 14: synchronize the new second processing capability value to the unified configuration center component, and update the processing capability value of the target model service to the new second processing capability value through the unified configuration center component.

[0071] In steps 13 and 14, the dynamic programming algorithm is a DP dynamic programming algorithm, which refers to an algorithm for solving complex problems by decomposing the original problem into relatively simple sub-problems.

[0072] For the sake of system stability, the DP dynamic programming algorithm is used as a bottom-up solution. If the regression prediction Xgboost model does not predict the result, the DP dynamic programming algorithm is needed to calculate the new second processing capacity value corresponding to the target model service, which can effectively avoid abnormal situations.

[0073] By continuously collecting system information and regularly training the Xgboost regression prediction model, the batch size of each model service is calculated in real time, and the batch size of each model service is dynamically adjusted. At the same time, when the Xgboost regression prediction model cannot make a prediction, a new batch is obtained through the DP dynamic programming algorithm, which improves the stability of the system and makes the entire system more robust.

[0074] Based on any of the above embodiments, the integrated model service system includes a unified configuration center component;

[0075] Before step 110, the method further includes:

[0076] The model is processed into a microservice to obtain a model service, and the model service is registered with a unified configuration center component.

[0077] In this step, Micro Service is a software architecture style that is based on small building blocks that focus on a single responsibility and function. It uses a modular approach to combine complex large applications. Each building block communicates with each other using a language-independent / languageagnostic API set.

[0078] The unified configuration center component is used to provide a registration and discovery mechanism for model services.

[0079] Based on any of the above embodiments, the integrated model service system includes a gateway component;

[0080] The step 110 specifically includes the following steps 111 to 112:

[0081] Step 111: obtain request information through the gateway component and analyze the request information to obtain analysis results.

[0082] In this step, the gateway component refers to the Gateway, which means that when forwarding the request information sent by the client, it can analyze and process the request information just like the source server with its own resources. The gateway component also includes functions such as authentication and current limiting.

[0083] Step 112: Send the analysis result to the unified configuration intermediate component through the gateway component, and determine the target model service with dependency relationship corresponding to the request information through the unified configuration center component.

[0084] In this step, the unified configuration center component will discover the target model service through the obtained analysis results, and determine the network location, port, and other related information of the target model service.

[0085] Based on any of the above embodiments, before step 120, the following steps 1 to 2 are included:

[0086] Step 1: Perform microservice processing on the feature engineering corresponding to the model to obtain feature engineering services.

[0087] In this step, feature engineering refers to using a series of engineering methods to filter out better data features from the original data to improve the training effect of the model.

[0088] Step 2: Determine the corresponding target feature engineering service through the target model service.

[0089] In this step, different model services correspond to different feature engineering services, so the corresponding target feature engineering service is determined by the target model service.

[0090] Determining the model parameter value in step 120 specifically includes the following steps 3 to 4:

[0091] Step 3: Obtain the original data corresponding to the target feature engineering service, and input the original data into the target feature engineering service to extract features.

[0092] In this step, different feature engineering services correspond to different data sources in the peripheral system and obtain raw data from the data sources.

[0093] Step 4: Process the feature through the target feature engineering service to determine the model parameter value.

[0094] In this step, the model parameter values ​​of the current target model service are calculated by the previous feature engineering service.

[0095] Further, for supplementary explanation of this disclosure, refer to Figure 3 , which is a block diagram of the model reasoning acceleration method provided in the present invention, is applied to an integrated model service system, taking the integrated model service system including three models as an example.

[0096] (1) Perform microservice processing on the model and feature engineering in the integrated model service system to obtain model service and feature engineering service. Specifically, in the figure, model 1 corresponds to model service 1, model 2 corresponds to model service 2, model 3 corresponds to model service 3, feature engineering 1 corresponds to feature engineering service 1, and feature engineering 2 corresponds to feature engineering service 2.

[0097] (2) The client initiates a request based on the HTTP protocol, and the integrated model service system obtains the request information through the gateway.

[0098] (3) The analysis result is sent to the unified configuration intermediate component through the gateway component, and the target model service 1, target model service 2 and target model service 3 with dependency relationships corresponding to the request information are determined through the unified configuration center component.

[0099] (4) Feature engineering 1 corresponds to data source 1 in the peripheral system, and feature engineering 2 corresponds to data sources 2 and 3 in the peripheral system. Data source 1 is input into feature engineering service 1 to obtain feature 1, and data sources 2 and 3 are input into feature engineering service 2 to obtain features 2 and 3. Model parameter value 1 of model service 1 is obtained through feature 1, and model parameter value 2 of model service 2 is obtained through features 2 and 3.

[0100] (5) Since the corresponding batches of target model service 1, target model service 2 and target model service 3 are different, message queues are set in target model service 1, target model service 2 and target model service 3 to balance the processing capacity values ​​among the three.

[0101] Specifically, a message queue 1 is set between the target model service 1 and the target model service 2, and the task data generated by the target model service 1 according to its own processing capability value is sent to the message queue 1, and the target model service 2 obtains the corresponding task data from the message queue 1 according to its own processing capability value; a message queue 2 is set between the target model service 2 and the target model service 3, and the task data generated by the target model service 2 according to its own processing capability value is sent to the message queue 2, and the target model service 3 obtains the corresponding task data from the message queue 2 according to its own processing capability value; in this way, the processing capability values ​​of the target model service 1, the target model service 2 and the target model service 3 can be balanced.

[0102] (6) Input model parameter value 1 into model service 1, input model parameter value 2 into model service 2 to obtain a calculation result, and use the calculation result as the model calculation result corresponding to the request information.

[0103] The model reasoning acceleration method provided in the present disclosure solves the delay problem of the integrated model in high-concurrency risk control scenarios and improves the overall throughput of the system.

[0104] The device for accelerating model reasoning provided by the present disclosure is described below. The device for accelerating model reasoning described below and the method for accelerating model reasoning described above can refer to each other.

[0105] Reference Figure 4 , is a schematic diagram of the structure of a device for accelerating model reasoning, which is arranged in an integrated model service system, and the device includes:

[0106] The acquisition module 410 is used to acquire the request information and analyze the request information to determine the target model service with dependency relationship corresponding to the request information; wherein the integrated model service system includes at least one model service;

[0107] A determination module 420 is used to determine a model parameter value, and process the model parameter value in sequence through at least one target model service having a dependency relationship and a message queue set therebetween to obtain a final output model calculation result;

[0108] Among them, the message queue between every two target model services with a dependent relationship is used to receive the task data generated by the previous target model service according to its own processing capability value, and send the task data to the latter target model service according to the processing capability value of the latter target model service.

[0109] The present disclosure provides a device for accelerating model reasoning, which obtains request information and analyzes the request information to determine a target model service with a dependency relationship corresponding to the request information; determines a model parameter value, and processes the model parameter value in turn through at least one target model service with a dependency relationship and a message queue set therebetween to obtain a final output model calculation result, wherein the message queue between every two target model services with a dependency relationship is used to receive task data generated by the previous target model service according to its own processing capability value, and send the task data to the next target model service according to the processing capability value of the next target model service, so that the processing capability value of the previous target model service can be equal to the processing capability value of the next target model. When equality is achieved, the entire integrated model service system can be made smoother, the delay of the target model service can be reduced, and the performance of the entire integrated model service system can be improved.

[0110] Based on any of the above embodiments, for every two target model services having a dependency relationship, the determining module 420 includes:

[0111] A storage module, used for storing the task data processed by the previous target model service in the message queue in batches according to the processing capacity value of the previous target model service, until the task data corresponding to the previous target model service is completely stored;

[0112] The processing module is used to obtain task data of a corresponding amount each time for processing according to the processing capacity value of the next target model service until all the task data in the message queue are obtained, and output the task data processed by the next target model service.

[0113] Based on any of the above embodiments, the integrated model service system includes a unified configuration center component;

[0114] Based on any of the above embodiments, the device further includes:

[0115] The first microservice processing module is used to perform microservice processing on the model, obtain the model service, and register the model service to the unified configuration center component.

[0116] Based on any of the above embodiments, the device further includes:

[0117] an output unit, used to periodically obtain system information of the target model service, input the system information into a trained processing capacity value regression prediction model, and output a new first processing capacity value corresponding to the target model service, wherein the trained processing capacity value regression prediction model is trained based on sample system information;

[0118] The first synchronization unit is used to synchronize the new first processing capability value to the unified configuration center component, and update the processing capability value of the target model service to the new first processing capability value through the unified configuration center component.

[0119] Based on any of the above embodiments, when the trained processing capability value regression prediction model does not output a new first processing capability value corresponding to the target model service, after the processing module, the device further includes:

[0120] A second determining unit, configured to determine a new second processing capability value corresponding to the target model service by a dynamic programming algorithm;

[0121] The second synchronization unit is used to synchronize the new second processing capability value to the unified configuration center component, and update the processing capability value of the target model service to the new second processing capability value through the unified configuration center component.

[0122] Based on any of the above embodiments, the integrated model service system includes a gateway component;

[0123] The acquisition module 410 includes:

[0124] An analysis module, used to obtain request information through the gateway component and analyze the request information to obtain analysis results;

[0125] The sending module is used to send the analysis result to the unified configuration intermediate component through the gateway component, and determine the target model service with dependency relationship corresponding to the request information through the unified configuration center component.

[0126] Based on any of the above embodiments, the device further includes:

[0127] The second microservice processing module is used to perform microservice processing on the feature engineering corresponding to the model to obtain feature engineering services;

[0128] A first determining unit, configured to determine a corresponding target feature engineering service through the target model service;

[0129] The determination module 420 is specifically used for:

[0130] Obtaining raw data corresponding to the target feature engineering service, and inputting the raw data into the target feature engineering service to extract features;

[0131] The feature is processed by the target feature engineering service to determine the model parameter value.

[0132] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 complete mutual communication through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute a method for accelerating model reasoning, which is used for an integrated model service system, and the method includes: obtaining request information and analyzing the request information, determining a target model service with a dependency relationship corresponding to the request information, wherein the integrated model service system includes at least one model service; determining a model parameter value, and processing the model parameter value sequentially through at least one target model service with a dependency relationship and a message queue set therebetween to obtain a model calculation result outputted finally; wherein the message queue between each two target model services with a dependency relationship is used to receive task data generated by the previous target model service according to its own processing capability value, and send the task data to the latter target model service according to the processing capability value of the latter target model service.

[0133] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0134] On the other hand, the present disclosure also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a model reasoning acceleration method provided by the above-mentioned methods, which is used for an integrated model service system. The method includes: obtaining request information and analyzing the request information to determine a target model service with a dependency relationship corresponding to the request information, wherein the integrated model service system includes at least one model service; determining a model parameter value, and processing the model parameter value in turn through at least one target model service with a dependency relationship and a message queue set therebetween to obtain a final output model calculation result; wherein the message queue between every two target model services with a dependency relationship is used to receive task data generated by the previous target model service according to its own processing capability value, and send the task data to the latter target model service according to the processing capability value of the latter target model service.

[0135] On the other hand, the present disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute a model reasoning acceleration method provided by the above-mentioned methods, which is used for an integrated model service system, and the method includes: obtaining request information and analyzing the request information, determining a target model service with a dependency relationship corresponding to the request information, wherein the integrated model service system includes at least one model service; determining model parameter values, and processing the model parameter values ​​in sequence through at least one target model service with a dependency relationship and a message queue set therebetween, to obtain a final output model calculation result; wherein the message queue between every two target model services with a dependency relationship is used to receive task data generated by the previous target model service according to its own processing capability value, and send the task data to the latter target model service according to the processing capability value of the latter target model service.

[0136] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0137] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A method for accelerating model reasoning, characterized in that: For an integrated model service system, the method comprises: Obtaining request information and analyzing the request information to determine a target model service with a dependency relationship corresponding to the request information, wherein the integrated model service system includes at least one model service; Determine model parameter values, and process the model parameter values ​​in sequence through at least one target model service having a dependency relationship and a message queue set therebetween to obtain a final output model calculation result; Among them, the message queue between every two target model services with a dependent relationship is used to receive the task data generated by the previous target model service according to its own processing capability value, and send the task data to the latter target model service according to the processing capability value of the latter target model service.

2. The method for accelerating model reasoning according to claim 1, characterized in that: For every two target model services having a dependency relationship, the model parameter value is processed sequentially via at least one target model service having a dependency relationship and a message queue set therebetween, including: According to the processing capacity value of the previous target model service, the task data processed by the previous target model service is stored in the message queue in batches until the task data corresponding to the previous target model service is completely stored; According to the processing capacity value of the next target model service, task data of the corresponding data volume is obtained each time for processing until all the task data in the message queue are obtained, and the task data processed by the next target model service is output.

3. The method for accelerating model reasoning according to claim 2, characterized in that: The integrated model service system includes a unified configuration center component; Before obtaining the request information and analyzing the request information, the method further includes: The model is processed into a microservice to obtain a model service, and the model service is registered with a unified configuration center component.

4. The method for accelerating model reasoning according to claim 3, characterized in that: After outputting the task data processed by the latter target model service, the method further includes: Periodically obtaining system information of the target model service, and inputting the system information into a trained processing capacity value regression prediction model, and outputting a new first processing capacity value corresponding to the target model service, wherein the trained processing capacity value regression prediction model is trained based on sample system information; The new first processing capability value is synchronized to the unified configuration center component, and the processing capability value of the target model service is updated to the new first processing capability value through the unified configuration center component.

5. The method for accelerating model reasoning according to claim 4, characterized in that: In the case where the trained processing capability value regression prediction model does not output a new first processing capability value corresponding to the target model service, after outputting the task data processed by the next target model service, the method further includes: Determine a new second processing capacity value corresponding to the target model service by a dynamic programming algorithm; The new second processing capability value is synchronized to the unified configuration center component, and the processing capability value of the target model service is updated to the new second processing capability value through the unified configuration center component.

6. The method for accelerating model reasoning according to claim 3, characterized in that: The integrated model service system includes a gateway component; The obtaining of request information and analyzing the request information to determine the target model service with dependency relationship corresponding to the request information includes: Obtaining request information through the gateway component and analyzing the request information to obtain analysis results; The analysis result is sent to the unified configuration center component through the gateway component, and the target model service with a dependency relationship corresponding to the request information is determined through the unified configuration center component.

7. The method for accelerating model reasoning according to claim 1, characterized in that: Before determining the model parameter value, the method includes: Perform microservice processing on the feature engineering corresponding to the model to obtain feature engineering services; Determine a corresponding target feature engineering service through the target model service; Determining the model parameter value includes: Obtaining raw data corresponding to the target feature engineering service, and inputting the raw data into the target feature engineering service to extract features; The feature is processed by the target feature engineering service to determine the model parameter value.

8. A device for accelerating model reasoning, characterized in that: Set in the integrated model service system, the device includes: An acquisition module, used to acquire request information and analyze the request information to determine a target model service with a dependency relationship corresponding to the request information; wherein the integrated model service system includes at least one model service; A determination module, used to determine a model parameter value, and process the model parameter value in sequence through at least one target model service having a dependency relationship and a message queue set therebetween to obtain a final output model calculation result; Among them, the message queue between every two target model services with a dependent relationship is used to receive the task data generated by the previous target model service according to its own processing capability value, and send the task data to the latter target model service according to the processing capability value of the latter target model service.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for accelerating model reasoning as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for accelerating model reasoning as described in any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for accelerating model reasoning as described in any one of claims 1 to 7 is implemented.

Images