Model calling method and device, electronic equipment and program product

Abstract

The present disclosure discloses a model calling method and device, the model calling method comprising: receiving a calling request of a target model by a target customer, the target model being associated with one or more secondary models; in the case of having a model routing configuration for the target customer, obtaining a model routing identification value of the calling request, the model routing configuration comprising respective data segments of the target model and the one or more secondary models, the data segments of the target model and the one or more secondary models being determined according to respective traffic proportions of the target model and the one or more secondary models; and determining an actual running model for the target customer from the target model and the one or more secondary models according to the model routing identification value and the data segments. The model calling method of the present disclosure can realize smooth adjustment of the model, and the progress and risk during model adjustment are controllable.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, specifically to model invocation methods and apparatus, and also to related electronic devices and program products. Background Technology

[0002] With the development of artificial intelligence and service-oriented deployment, AI models have been widely deployed as software-as-a-service (SaaS) products to provide services to customers. Since fluctuations in model performance directly impact subsequent business processing and decision-making, it is desirable for the model's output to possess extremely high stability and accuracy.

[0003] Currently, statistical indicators such as population stability metrics have been proposed to monitor model performance and adjust the model to restore expected performance when significant fluctuations in performance are detected. Known adjustment methods often involve replacing the original model with a new one deployed in the production environment.

[0004] However, this adjustment method has limited adaptability to specific business scenarios, and its process control and risk management capabilities are insufficient, which can easily affect the continuity and reliability of model services.

[0005] Therefore, there is an urgent need to provide a more secure and controllable model adjustment solution to ensure the stability of model services.

[0006] The background description is provided for the purpose of understanding the relevant technologies in this field and is not intended as an admission of prior art. Summary of the Invention

[0007] Accordingly, this disclosure provides a model invocation method and apparatus, electronic device, and program product that at least partially solve the above-mentioned problems.

[0008] In a first aspect, embodiments of this disclosure provide a model invocation method, comprising:

[0009] Receive a request from a target customer to invoke a target model, where the target model is associated with one or more sub-models;

[0010] In the case of having a model routing configuration for the target customer, the model routing identifier value of the call request is obtained, wherein the model routing configuration includes data segments corresponding to the target model and the one or more sub-models respectively, and the data segments of the target model and the one or more sub-models are determined according to the traffic ratio of the target model and the one or more sub-models respectively;

[0011] Based on the model routing identifier value and the data segment, determine the actual operating model for the target customer from the target model and the one or more sub-models.

[0012] In some embodiments, the model invocation method further includes:

[0013] In the absence of a model routing configuration for the target customer, the target model is determined as the actual operating model for the target customer.

[0014] In some embodiments, the model invocation method further includes:

[0015] Obtain the execution result of the actual running model in response to the call request, and return the execution result as the call result of the call request to the target client.

[0016] In some embodiments, the model invocation method further includes:

[0017] Generate model routing configuration update information and store the model routing configuration update information in a cache, wherein the model routing configuration update information includes one or more of the following: model addition information, model replacement information, model deletion information, and traffic ratio adjustment information;

[0018] If the cache contains model routing configuration update information associated with the target model and the target customer, update the model routing configuration according to the model routing configuration update information.

[0019] In some embodiments, updating the model routing configuration according to the model routing configuration update information includes:

[0020] After the model routing configuration is updated by the model routing configuration update information, in the event of an abnormal call result, the model routing configuration update information is generated based on any saved historical model routing configuration.

[0021] In some embodiments, the model invocation method further includes:

[0022] Record the call result and the corresponding target model or sub-model;

[0023] The target model or sub-model corresponding to the call result whose fluctuation exceeds a preset threshold is identified as a fluctuation anomaly model.

[0024] Adjust the traffic ratio between the target model and the sub-model to reduce the traffic ratio corresponding to the fluctuation anomaly model;

[0025] The model routing configuration update information is generated based on the adjusted traffic ratio.

[0026] In some embodiments, obtaining the execution result of the actual running model in response to the call request includes:

[0027] In the event of a database failure on which the actual operating model depends, database failure information is generated;

[0028] The method further includes:

[0029] Based on the database fault information, adjust the traffic ratio between the target model and the sub-model to reduce the traffic ratio of the actual running model;

[0030] The model routing configuration update information is generated based on the adjusted traffic ratio.

[0031] In some embodiments, obtaining the model route identifier value of the invocation request includes:

[0032] Generate hash encoding information based on the input parameter information in the call request;

[0033] The hash code information is moduloed to obtain a random number within a preset range, and the random number is used as the model routing identifier value.

[0034] In some embodiments, determining the actual operating model for the target customer from the target model and the one or more sub-models based on the model routing identifier value and the data segment includes:

[0035] Based on the traffic ratio corresponding to the target model and the preset range, obtain the first data segment within the preset range corresponding to the target model;

[0036] Based on the traffic ratio corresponding to each sub-model and the preset range, obtain the second data segment within the preset range corresponding to each sub-model;

[0037] In the first data segment and the second data segment, the target data segment in which the model routing identifier value is located is determined, and the target model or sub-model corresponding to the target data segment is taken as the actual running model.

[0038] In some embodiments, the target model and the one or more sub-models rely on different datasets at runtime.

[0039] In a second aspect, embodiments of this disclosure provide a model invocation apparatus, comprising:

[0040] The receiving unit is configured to receive a call request from a target client for a target model, wherein the target model is associated with one or more sub-models;

[0041] The acquisition unit is configured to acquire the model route identifier value of the call request when there is a model route configuration for the target customer, wherein the model route configuration includes data segments corresponding to the target model and the one or more sub-models respectively, and the data segments of the target model and the one or more sub-models are determined according to the traffic ratio of the target model and the one or more sub-models respectively;

[0042] The determining unit is configured to determine the actual running model from the target model and the one or more sub-models based on the model routing identifier value and the model routing configuration.

[0043] In a third aspect, embodiments of this disclosure provide an electronic device comprising: a processor and a memory storing a computer program, the processor being configured to implement the method as described in the first aspect when the computer program is executed.

[0044] In a fourth aspect, embodiments of this disclosure provide a program product including a computer program, wherein the computer program, when executed by a processor, implements the method as described in the first aspect.

[0045] The model invocation method provided in this disclosure can receive a target customer's invocation request for a target model, where the target model is associated with one or more sub-models. When a model routing configuration for the target customer is available, the method obtains the model routing identifier value of the invocation request. The model routing configuration includes data segments corresponding to the target model and the one or more sub-models, and the data segments of the target model and the one or more sub-models are determined based on their respective traffic ratios. Based on the model routing identifier value and the data segments, the method determines the actual running model for the target customer from the target model and the one or more sub-models. This model invocation method can achieve customer-level request traffic control during model adjustment, especially during gray-scale model adjustments, overcoming the limitations of traditional upgrade schemes based on direct model replacement. It enables smooth model adjustment and upgrades, and the adjustment progress and risks during model adjustment are controllable. When the new model is ineffective, timely traffic rollback can be achieved, improving the stability and controllability of model upgrades.

[0046] Other optional features and technical effects of the embodiments of this disclosure are described in part below, and in part will be apparent from reading this document. Attached Figure Description

[0047] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings. The elements shown are not limited to the scale shown in the drawings, and the same or similar reference numerals in the drawings denote the same or similar elements, wherein:

[0048] Figure 1 A first flowchart of a model invocation method according to an embodiment of the present disclosure is shown;

[0049] Figure 2 A second flowchart of a model invocation method according to an embodiment of the present disclosure is shown;

[0050] Figure 3 A third flowchart of a model invocation method according to an embodiment of the present disclosure is shown;

[0051] Figure 4 A fourth flowchart of a model invocation method according to an embodiment of the present disclosure is shown;

[0052] Figure 5 A fifth flowchart of a model invocation method according to an embodiment of the present disclosure is shown;

[0053] Figure 6 A sixth flowchart of a model invocation method according to an embodiment of the present disclosure is shown;

[0054] Figure 7 A seventh flowchart of a model invocation method according to an embodiment of the present disclosure is shown;

[0055] Figure 8 An exemplary block diagram of a model invocation apparatus according to embodiments of the present disclosure is shown; and

[0056] Figure 9 A structural diagram of an electronic device for implementing the model invocation method of embodiments of the present disclosure is shown. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this disclosure clearer, the disclosure will be further described in detail below with reference to specific embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this disclosure are used to explain this disclosure, but are not intended to limit this disclosure.

[0058] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0059] The user data, data acquisition, and / or use involved in the embodiments of this disclosure strictly comply with the laws, regulations, and industry standards of relevant countries and regions. The collection, acquisition, processing, forwarding, and use of data involved in the embodiments of this disclosure are all conducted with the user's full knowledge and authorization. In implementing the embodiments of this disclosure, the user is informed and authorized through appropriate means regarding the types of data or information involved, the scope of use, and the usage scenarios. The specific methods of notification and authorization may vary according to actual circumstances, and this disclosure is not limited in this regard. The processing of personal information involved in the embodiments of this disclosure is conducted under legal grounds (such as obtaining the consent of the personal information subject or being necessary for the performance of a contract), and is only processed within the prescribed or agreed scope. If a user refuses to process personal information other than the information necessary for basic functions in the embodiments of this disclosure, it will not affect the use of the basic functions of the embodiments of this disclosure.

[0060] As mentioned earlier, with the service-oriented deployment of AI models, these models can be offered to customers as SaaS (Software as a Service) products. In some business scenarios, customers can invoke the model to score, classify, assess risk levels, or identify abnormal states of business objects. In these scenarios, the model's output often serves as a crucial basis for subsequent business access, resource allocation, manual review, and anomaly handling processes. Therefore, customers in these scenarios have extremely high requirements for the stability and accuracy of the model's output. To explain, in these scenarios, customers often continuously send service requests to the model. If the model's performance fluctuates significantly and is not addressed promptly, it can adversely affect the customer's decisions regarding a large number of business objects.

[0061] Currently, to ensure the stability of model product performance, analysts at model service providers can monitor the effectiveness of model products for clients in real time using statistical indicators such as PSI (Population Stability Index). When statistical indicators such as PSI exceed preset thresholds, it indicates a significant decline in the predictive ability of the model product, requiring immediate model adjustments for that client, such as model upgrades. This is because in these business scenarios, the characteristics of a client's business objects change with market conditions or other factors. Consequently, the data distribution used as input to the model in client requests can also change, and the original evaluation logic of the model product may become inapplicable over time.

[0062] In response, this disclosure acknowledges that known model adjustments primarily employ a direct replacement approach: when the monitoring system detects abnormal model performance, analysts can retrain a new model based on the latest sample data. After offline verification, the new model is directly deployed to the production environment to replace the old model, and relevant customers are notified to update their calling methods. However, this upgrade process is high-risk and irreversible. Once the new model is deployed, it completely replaces the old model, directly impacting all customers related to that model. Furthermore, this approach makes model rollback difficult. When the new model fails to meet expectations in the production environment, redeploying the old model is complex and time-consuming, potentially adversely affecting the businesses of all customers who rely on the model.

[0063] Furthermore, this disclosure recognizes that in SaaS products, the same model may serve multiple customers or multiple business scenarios, and traditional direct replacement methods cannot support customer-level model upgrade control. Specifically, if an old model is directly replaced because it is ineffective for one customer or multiple business scenarios, it will affect other customers served by that model, posing a risk of reducing the stability of ratings obtained by other customers. Therefore, how to improve the flexibility, controllability, and risk management capabilities of the model adjustment process in scenarios where models are provided as SaaS products for external invocation services is a practical problem that urgently needs to be solved in this field.

[0064] To address this, this disclosure provides a model invocation method. This method can be applied to scenarios where a SaaS scoring model service provider offers model services to multiple clients. More specifically, this method can be applied to scenarios such as business object scoring, virtual resource configuration decisions, business access judgment, and risk level assessment. In the above scenarios, the model output results can serve as the basis for subsequent business processing, virtual resource scheduling, anomaly handling, or decision-making. In some embodiments, this method can be implemented by, but is not limited to, a SaaS service provider's model service platform, model management platform, model scheduling system, model service middleware, or distributed model service system.

[0065] In some embodiments of this disclosure, reference is made to Figure 1 The model invocation method may include the following steps 110, 120 and 130.

[0066] 110: Receive the target customer's request to call the target model.

[0067] In some embodiments of this disclosure, the target model may include a scoring model. The following embodiments will describe the model invocation method according to embodiments of this disclosure in detail, in conjunction with a scoring model. However, it will be understood that in other embodiments, the target model may also include, but is not limited to, one or more of a prediction model, a ranking model, a detection model, a recognition model, and a decision model, which fall within the protection scope of embodiments of this disclosure.

[0068] In some embodiments, invocation requests may be received through, but not limited to, an interface gateway. In this disclosure, the term "interface gateway" should be interpreted broadly, encompassing, but not limited to, API gateways, load balancers, proxy servers, routing services, forwarding middleware, or other service components with similar functionalities, as long as the component can receive and forward invocation requests and return results to the client.

[0069] In some embodiments of this disclosure, a target model is associated with one or more sub-models. In these embodiments, one or more model routing configurations can be configured for each pre-deployed (pre-defined) model. In this embodiment, in other words, the model routing configuration corresponding to each model defines one or more other models associated with that model. In some embodiments, the model routing configuration may include client-level model routing configurations, which will be described in detail below.

[0070] In some embodiments, when a model call request is received from a target client, the model directly requested by the call request may be referred to as the target model. In some embodiments, one or more sub-models associated with the target model may be determined based on the model routing configuration corresponding to the target model. It is understood that the terms "target model" and "sub-model" should be understood as relative concepts rather than attributes of the model itself. For example, for model A, when model A is directly pointed to by call request 1, model A may be referred to as the target model (of call request 1); when model B is directly pointed to by a call request and the model routing configuration corresponding to model B includes model A, model A may be referred to as the sub-model (of model B).

[0071] In some embodiments, the target model and one or more sub-models rely on different datasets at runtime. In some embodiments, the training processes of the target model and each sub-model are based on different datasets. In some embodiments, the dataset used for training the sub-models is newer than the dataset used for training the target model. For explanation, in this example, the data samples in the dataset used for training the sub-models were generated at least partially later than the data samples in the dataset used for training the target model. In alternative embodiments, each sub-model can be trained based on the latest data of one or more target customers corresponding to the target model, thereby achieving customer-level sub-model training and deployment. For example, when configuring sub-model C for target model A, the training data for sub-model C mainly includes training data related to target customer 1 and target customer 3 served by target model A.

[0072] In some embodiments of this disclosure, the target customer's request to invoke the target model may include the customer's unique customer code information. In this embodiment, the unique customer code information can be used to subsequently determine whether the customer has configured a corresponding model routing configuration for the target model, thereby deciding whether customer-level model routing processing is required, which will be described in detail below.

[0073] In some embodiments of this disclosure, the target customer's request to invoke the target model may include request parameters for inputting the model to obtain results; these request parameters may also be referred to as input parameters. In some embodiments, the input parameters may include various types of data required for model evaluation, including but not limited to one or more of the following business information: anonymized virtual resource transfer information, business object behavior information, business scenario information, data collection information, and related business context information; this disclosure does not impose any limitations on this.

[0074] 120: If there is a model routing configuration for the target customer, obtain the model routing identifier value of the call request.

[0075] In embodiments of this disclosure, as previously described, the target customer's request to invoke the target model may include the customer's unique customer code information. In some embodiments, each (target) model may be configured with a model routing configuration for one or more customers, the model routing configuration defining the routing links of the (target) model for requests from different customers. In one example, the model routing configuration may accordingly include the unique customer code information of the corresponding target customer or a mapping relationship with the unique customer code information, as described in the embodiments below.

[0076] In some embodiments of this disclosure, multiple (target) models may be deployed, each of which is configured with multiple model routing configurations for multiple different customers. These model routing configurations can be cached for querying. Accordingly, in some embodiments, the latest model routing configuration can also be queried and / or obtained in real time upon receiving a call request from a target customer for a target model.

[0077] In some embodiments of this disclosure, the model routing configuration includes data segments corresponding to the target model and one or more sub-models, and the data segments of the target model and one or more sub-models are determined according to the traffic ratio of the target model and one or more sub-models respectively. In some embodiments, a preset range (data interval) can be divided into the target model and its corresponding sub-models according to the traffic ratio, so that each model corresponds to a data segment, and the first data segment corresponding to the target model and the second data segments corresponding to each sub-model together constitute the above-mentioned preset range (data interval). In some embodiments, the traffic ratio refers to the proportion of requests routed to different models in multiple calls from a client within a certain period of time. In a specific embodiment, when client 001 calls the target scoring model A, the corresponding model routing configuration is: scoring model A: 20%, scoring model B: 30%, scoring model C: 50%. In this regard, the random number interval [1, 100] is divided into different models according to the ratio, where the target model A corresponds to the interval [81, 100]; the sub-model B of the target model A corresponds to the interval [1, 30], and the sub-model C corresponds to the interval [31, 80]. This means that out of 100 requests from customer 001 to target scoring model A, approximately 20 will be actually routed to model A for processing, approximately 30 will be routed to model B, and approximately 50 will be routed to model C.

[0078] In some embodiments of this disclosure, reference is made to Figure 2 The steps to obtain the model route identifier value of the call request may include the following steps 121 and 122.

[0079] 121: Generate hash encoded information based on the input parameters in the call request.

[0080] In some embodiments, the interface gateway can calculate hash code information (hashCode) based on the request parameters of the call request, and then determine the model routing identifier value based on the hash code information. In this embodiment, the description of the request parameters can be found in the descriptions in steps 110 and 120 above, and will not be repeated here. It is understood that the parameters used to calculate the hash code information can be one or more of the request parameters. In some embodiments, a single key parameter can be selected for hash calculation; in other embodiments, multiple parameters can be combined or concatenated before hash calculation, for example, unique customer code information, timestamps, etc., can be concatenated with other parameters into a composite string, and then the composite string can be hashed. This disclosure does not limit the specific selection of parameters.

[0081] 122: Take the remainder of the hash-encoded information to obtain a random number within a preset range, and use the random number as the model routing identifier value.

[0082] As previously mentioned, a preset range (data interval) can be proportionally divided among the target model and its corresponding sub-model. In some embodiments, a random number within the preset range can be obtained by taking the remainder of the hash encoded information, and this random number can be used as the model routing identifier value. It is understood that the divisor of the remainder operation is not limited to 100, but can be determined according to the size of the preset range. In a specific embodiment, when the preset range is [1, 100], a random number within the range [1, 100] can be obtained by taking the remainder of the hash encoded information modulo 100 and then adding 1. In other embodiments, it is also conceivable to set the preset range to [1, 50] and take the remainder of the hash encoded information modulo 50 and then add 1.

[0083] 130: Determine the actual running model from the target model and one or more sub-models based on the model routing identifier value and data segment.

[0084] In some embodiments of this disclosure, the actual running model can be determined based on the model routing identifier value and the traffic proportions corresponding to the target model and each sub-model in the model routing configuration. Specifically, the actual running model can be determined based on the model routing identifier value and the data segments corresponding to the target model and each sub-model in the model routing configuration. In some embodiments, the above steps are performed, for example, by an interface gateway, which determines which model the request should be routed to for actual execution by judging the model corresponding to the data segment to which the model routing identifier value falls.

[0085] In some embodiments of this disclosure, reference is made to Figure 3 The steps of determining the actual running model from the target model and one or more sub-models based on the model routing identifier value and data segment may include the following steps 131, 132 and 133.

[0086] 131: Based on the traffic ratio and preset range corresponding to the target model, obtain the first data segment within the preset range corresponding to the target model.

[0087] As mentioned earlier, a preset range (data interval) can be proportionally divided among the target model and its corresponding sub-model, so that each corresponds to a data range (data segment). In some embodiments, this step may be performed by the interface gateway, for example, determining the first data segment that the target model should occupy within the preset range based on the traffic ratio configured for the target model. In one example, the start and end positions of the first data segment corresponding to the target model can be determined based on the traffic ratio and the preset range, thereby determining the boundary of the first data segment. In a specific example, if the traffic ratio of target model A is 20% and the preset range is [1, 100], then the first data segment corresponding to target model A is determined to be [1, 20].

[0088] 132: Based on the traffic ratio and preset range corresponding to each sub-model, obtain the second data segment within the preset range corresponding to each sub-model.

[0089] In some embodiments, the second data segment corresponding to each sub-model can be determined separately. It is understood that the second data segments of each sub-model do not overlap with each other and do not overlap with the first data segment. In some embodiments, a sequential allocation method can be used, allocating consecutive data segment ranges to each sub-model according to the configuration order or priority of the sub-models. Continuing with the previous specific example, the preset range is [1, 100]. If the traffic ratio of sub-model B is 30%, then the second data segment corresponding to sub-model B is determined to be [21, 50]; if the traffic ratio of sub-model C is 50%, then the second data segment corresponding to sub-model C is determined to be [51, 100].

[0090] 133: Determine the target data segment where the model routing identifier value is located in the first data segment and the second data segment, and use the target model or sub-model corresponding to the target data segment as the actual running model.

[0091] In some embodiments, the data segment in which the model routing identifier value is located can be determined. In one example, the interface gateway can determine the target data segment by comparing the model routing identifier value with the boundary values ​​of each data segment. In a specific example, when the model routing identifier value is 85, it can be determined by boundary comparison that 85∈[41, 100].

[0092] In other embodiments of this disclosure, for clients without model routing configuration, invocation requests from the user can be forwarded to the target model. In some embodiments, refer to... Figure 4 The model invocation method may further include:

[0093] 140: In the absence of model routing configuration for the target customer, determine the target model as the actual running model for the target customer.

[0094] In some embodiments, if the target customer has not configured customer-level model routing, the target model itself can be used by default to provide services to the target customer without the need for routing judgment and model switching.

[0095] In some embodiments of this disclosure, reference is made to Figure 5 The model invocation method may further include:

[0096] 150: Obtain the execution result of the actual running model in response to the call request, and return the execution result as the call result of the call request to the target client.

[0097] In some embodiments, for example, the interface gateway may invoke the actual running model to obtain the running result of the actual running model in response to the invocation request. Specifically, the interface gateway may input the parameters in the invocation request into the actual running model, and the actual running model may retrieve the data required for running from the corresponding database according to the input parameters, and then output the running result.

[0098] In some embodiments, when the actual running model is the target model, its running result can be returned to the client; when the actual running model is a secondary model, the running result of the secondary model can be converted into a format consistent with the target model before being returned to the client, thereby ensuring that the response format received by the client is always consistent with the format when the target model is called. Thus, during the model upgrade process, the client is unaware that the model has been actually called, and there is no need to modify its calling method or result parsing logic, thereby reducing the client integration cost and technical complexity during the model upgrade process.

[0099] The solution in this disclosure, through customer-level traffic ratio control, enables progressive upgrade verification of (secondary) models. When a newly developed secondary model needs to be deployed, it is not necessary to replace the original (target) model all at once. Instead, a small proportion of traffic can be allocated for small-scale verification to observe the performance of the secondary model in real business scenarios. Then, based on the performance of the secondary model, the customer's model routing configuration for the target model can be gradually adjusted, correspondingly increasing / decreasing the traffic ratio of the original (target) model or the secondary model. Therefore, this solution achieves progressive model upgrade verification, avoiding the risks associated with full upgrades. Furthermore, through customer-level traffic control, real-time model performance monitoring, and rapid response, traffic rollback can be quickly completed when the model performance is unsatisfactory. In addition, customers are unaware of model switching and adjustments, ensuring uninterrupted and reliable service even during model upgrades.

[0100] In some embodiments of this disclosure, the aforementioned model routing configuration can be generated and managed through a model management platform, thereby enabling the addition and removal of target models and sub-models, as well as real-time adjustment of traffic ratios. In some embodiments, refer to Figure 6 The model invocation method may further include the following steps 610 and 620.

[0101] 610: Generate model routing configuration update information and store the model routing configuration update information in the cache.

[0102] In some embodiments of this disclosure, the model routing configuration update information may include one or more of the following: model addition information, model replacement information, model deletion information, and traffic ratio adjustment information.

[0103] In some embodiments, the traffic ratio adjustment information includes information on changing the value of the traffic ratio corresponding to at least one of the target model and one or more sub-models in the model routing configuration. In some embodiments, the traffic ratio adjustment information may include the adjustment object and the adjusted traffic ratio value corresponding to the adjustment object. By way of explanation and not limitation, after the model routing configuration is updated according to the traffic ratio adjustment information, the data segments corresponding to each model in the model routing configuration are re-determined based on the adjusted traffic ratio.

[0104] In some embodiments of this disclosure, the model routing configuration update information may also include customer-unique coding information. For example, for a model serving multiple customers, the model management platform can use the model routing configuration update information to precisely adjust the traffic ratio for the target model for each customer.

[0105] In some embodiments of this disclosure, the step of generating model routing configuration update information may include: generating model routing configuration update information based on an upgrade operation. In this embodiment, as previously described, when the predictive ability of the scoring model product declines, a model upgrade is required. In this embodiment, the upgrade operation includes, but is not limited to, one or more of the following operations: adding a sub-model to the target model, increasing the traffic proportion of the sub-model, and decreasing the traffic proportion of the target model. For explanation, when the performance of the sub-model (new model) stabilizes, its traffic proportion can be increased to gradually replace the target model (original model), thereby achieving a smooth model upgrade.

[0106] In some embodiments of this disclosure, the step of generating model routing configuration update information based on the upgrade operation may further include: calling an automated testing platform to verify the model routing configuration after the upgrade operation, and generating model routing configuration update information if the verification passes. In this embodiment, the configuration update information can be verified, including but not limited to: verifying whether the updated traffic ratio totals a preset value (e.g., 100%), verifying whether the updated data segments overlap and whether they total a preset range, verifying whether the unique customer code information is valid, and verifying that all models have been deployed and are available. In some embodiments, it may also be verified whether the traffic is correctly implemented according to the configured traffic ratio. In one example, a preset test dataset corresponding to the target customer for the target model can be used to simulate the target customer's call request to the target model, and the actual number of calls to each model can be counted to verify whether the traffic allocation meets the expected ratio.

[0107] In some embodiments of this disclosure, the call results of each call request from the client can also be monitored, and the traffic ratio can be adjusted based on the call results and a preset threshold. In some embodiments, the adjustment of the traffic ratio includes manual adjustment or automatic adjustment, which will be described in detail below.

[0108] In some embodiments of this disclosure, reference is made to Figure 7 The model invocation method may further include the following steps 611, 612, 613 and 614.

[0109] 611: Record the call results and the corresponding target model or sub-model.

[0110] In some embodiments of this disclosure, for example, the model management platform may record the call information for each call, including but not limited to customer unique code information, call time, input parameters, actual running model, call result, etc.

[0111] 612: The target model or sub-model corresponding to the call result whose fluctuation exceeds the preset threshold is identified as the fluctuation abnormal model.

[0112] In some embodiments of this disclosure, the preset threshold may include one or more statistical indicators, including but not limited to the Population Stability Index (PSI). In one specific example, the PSI threshold is set to 0.1. In some embodiments, the statistical indicators of the call results returned by each model can be monitored, and models with indicators exceeding the preset threshold can be marked as abnormally volatile models.

[0113] 613: Adjust the traffic ratio of the target model and the sub-model to reduce the traffic ratio corresponding to the abnormal fluctuation model.

[0114] In some embodiments of this disclosure, as previously described, a model can serve multiple clients simultaneously, i.e., it can be associated with multiple model routing configurations at the same time. In some embodiments, such as in step 613 above, one or more model routing configurations associated with the fluctuating abnormal model can be determined, and then the traffic ratio of the target model and the sub-model in these model routing configurations can be adjusted to reduce the traffic ratio corresponding to the fluctuating abnormal model. It is understood that the reduction in the model traffic ratio can be determined according to actual needs, and this disclosure does not limit it.

[0115] In some embodiments of this disclosure, the operational status of the database upon which the actual running model depends can also be monitored. In this embodiment, the aforementioned step of obtaining the running result of the actual running model in response to the call request may include: generating database failure information in the event of a database failure on which the actual running model depends. In some embodiments, the model call method may further include the following step: adjusting the traffic ratio of the target model and the sub-model according to the database failure information, so as to reduce the traffic ratio of the actual running model. In some embodiments, for example, the traffic ratio corresponding to the actual running model may be reduced to 0.

[0116] 614: Generate model routing configuration update information based on the adjusted traffic ratio.

[0117] In some embodiments of this disclosure, for each model routing configuration, model routing configuration update information corresponding to these model routing configurations can be generated based on the adjusted traffic ratio.

[0118] 620: If the cache contains model routing configuration update information associated with the target model and the target customer, update the model routing configuration according to the model routing configuration update information.

[0119] In some embodiments of this disclosure, model routing configuration update information in the cache can be monitored in real time. If model routing configuration update information exists, the corresponding model routing configuration can be determined and the model routing configuration can be updated.

[0120] In some embodiments, historical model routing configurations may be cached by the model management platform to support rapid rollback of model routing configurations. In some embodiments of this disclosure, the step of updating the model routing configuration based on model routing configuration update information may include: after updating the model routing configuration based on the model routing configuration update information, in the event of an abnormal call result, generating model routing configuration update information based on any saved historical model routing configuration.

[0121] In some embodiments, model routing configuration update information can be automatically generated based on the most recent model routing configuration in the historical model routing configuration. As an explanation, in cases such as adding a secondary model during an upgrade operation, even though the secondary model has been verified offline, there is still a risk of unstable model performance. The method in the above embodiments of this disclosure overcomes the problems of traditional upgrade methods that require redeployment, are time-consuming, and have a wide impact when the new model performs poorly, by monitoring the call results and enabling rapid rollback of the client-level model routing configuration.

[0122] The model invocation method provided in this embodiment implements a personalized model upgrade scheme. By configuring an update mechanism at the model routing configuration level for different customers and different models, the same model can provide differentiated model upgrade services to multiple customers at the same time. The model upgrades and adjustments between customers do not affect each other, which significantly improves the granularity of the service.

[0123] Furthermore, the model invocation method provided in this disclosure is seamless for customers during model upgrades and requires no downtime. Customers do not need to modify the invocation interface or business logic, which reduces customer coordination costs during model upgrades.

[0124] The model invocation method of this disclosure will be described below with reference to a specific embodiment.

[0125] In one specific embodiment of this disclosure, a scoring model service platform simultaneously provides risk control services to multiple financial institutions. The platform deploys scoring model A, which serves as the target scoring model for financial institution I1 (customer code 0001). During service operation, the scoring model service platform monitoring revealed that the PSI (Probability of Score) of scoring model A for financial institution I1 gradually rose to 0.12, exceeding the preset threshold of 0.1. This indicates a significant decline in the predictive ability of scoring model A for financial institution I1. In response, the scoring model service platform prepared scoring models B and C, trained based on the latest data from financial institution I1. Offline verification showed that both models performed well in terms of PSI performance in simulated customer request data. Therefore, the platform decided to upgrade scoring model A using scoring models B and C.

[0126] To avoid the risks associated with direct replacement, the analyst configured a client-level request routing path for the financial institution's I1 on the model management platform, as follows:

[0127] Table 1. Routing Link Relationship Table for Scoring Model

[0128]

[0129] After the model management platform generates the model routing configuration update information, the automated testing platform of the scoring model service platform automatically verifies it using a simulated dataset and caches the model routing configuration update information after successful verification. At this time, the interface gateway of the scoring model service platform detects an update event in the cache, retrieves the configuration update information cache, and updates the model routing configuration of financial institution I1 for scoring model A to the configuration shown in Table 1.

[0130] After configuration, financial institution I1 sends call request 1 and call request 2 to scoring model A sequentially. For call request 1, the interface gateway receives the request and parses the customer code 0001, request parameters, and request timestamp. The interface gateway then combines the request parameters and timestamp to obtain a combined parameter, calculates its hash value (1234567890), and takes the remainder of this hash value with respect to 100 to obtain a random number as the model routing identifier: randomNumber = Math.abs(1234567890)%100+1 = 91. Subsequently, the interface gateway compares the model routing identifier 91 with the multiple data segments corresponding to the user, determining that 91 ∈ [81, 100]. Therefore, it determines to route to model A, retrieves the required customer feature data from database 1, runs the program, and uses the result scoreA = 750 as the call result of call request 1. For call request 2, the interface gateway receives the request and parses the customer code 0001, request parameters, and request timestamp. The interface gateway then combines the request parameters and the request timestamp to obtain the combined parameters, and calculates their hash value to obtain the hash value = 987654321. Then, it takes the remainder of this hash value divided by 100 to obtain a random number as the model routing identifier value: randomNumber = Math.abs(987654321)%100+1 = 22. Subsequently, the interface gateway compares the model routing identifier value 22 with the multiple data segments corresponding to the user, determining that 22 ∈ [1, 30]. Therefore, it determines to route to model B, retrieves the customer feature data required by model B from database 3, runs the program, and uses the running result scoreB = 680 as the call result of request 2.

[0131] The scoring model service platform monitors the running results of each model in real time and calculates the performance indicators of each model. Among them, scoring model A: PSI=0.15, about 20% of the calls, abnormal effect; scoring model B: PSI=0.08, about 30% of the calls, normal effect; scoring model C: PSI=0.07, about 50% of the calls, normal effect. In response, the analyst gradually reduced the traffic proportion of the abnormal scoring model A and increased the traffic proportion of the sub-models B and C. The first adjustment was as follows: Scoring Model A: 10% (data segment [91, 100]), Scoring Model B: 30% (data segment [1, 30]), and Scoring Model C: 60% (data segment [31, 90]); the second adjustment was as follows: Scoring Model A: 5% (data segment [96, 100]), Scoring Model B: 25% (data segment [1, 25]), and Scoring Model C: 70% (data segment [26, 95]); the third adjustment was as follows: Scoring Model A: 0% (no allocation), Scoring Model B: 20% (data segment [1, 20]), and Scoring Model C: 80% (data segment [21, 100]). This enabled a smooth and stable upgrade of the scoring model for Financial Institution I1, and the calling method of Financial Institution I1 remained unchanged during the upgrade process, requiring no adjustments to the business system. Other customers on the platform (such as financial institutions I2) will remain unaffected and will continue to be served by the original scoring model A.

[0132] In some embodiments of this disclosure, such as Figure 8 As shown, a model invocation device 800 is also provided, which may include: a receiving unit 810, an acquiring unit 820, and a first determining unit 830. Wherein,

[0133] The receiving unit 810 is configured to receive a call request from a target client for a target model, wherein the target model is associated with one or more sub-models;

[0134] The acquisition unit 820 is configured to acquire the model route identifier value of the call request when there is a model route configuration for the target customer, wherein the model route configuration includes data segments corresponding to the target model and the one or more sub-models respectively, and the data segments of the target model and the one or more sub-models are determined according to the traffic ratio of the target model and the one or more sub-models respectively;

[0135] The first determining unit 830 is configured to determine the actual running model from the target model and the one or more sub-models based on the model routing identifier value and the model routing configuration.

[0136] In some embodiments of this disclosure, the model invocation device 800 may further include a second determining unit configured to determine the target model as the actual running model for the target customer in the absence of a model routing configuration for the target customer.

[0137] In some embodiments of this disclosure, the model invocation device 800 may further include a sending unit configured to return the running result as the invocation result of the invocation request to the target client.

[0138] The apparatus, components, modules, units, and features described in the embodiments of this disclosure can be incorporated into the methods of the embodiments of this disclosure in a non-contradictory manner, and the methods, steps, sub-steps, and features described in the embodiments of this disclosure can also be incorporated into the apparatus of the embodiments of this disclosure in a non-contradictory manner.

[0139] In embodiments of this disclosure, an electronic device may also be provided, including: a processor and a memory storing a computer program, the processor being configured to perform the method of any of the embodiments of this disclosure when running the computer program.

[0140] Figure 9 The illustration shows a method or electronic device 900 that can implement embodiments of the present disclosure, and in some embodiments may include more or fewer parts / components than illustrated. In some embodiments, it may be implemented using a single or multiple electronic devices. In some embodiments, it may be implemented using cloud-based or distributed electronic devices.

[0141] like Figure 9 As shown, the electronic device 900 includes a processor 901, which can perform various appropriate operations and processes based on programs and / or data stored in read-only memory (ROM) 902 or programs and / or data loaded from storage portion 908 into random access memory (RAM) 903. The processor 901 can be a multi-core processor or may contain multiple processors. In some embodiments, the processor 901 may include a general-purpose main processor and one or more special coprocessors, such as a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. The RAM 903 also stores various programs and data required for the operation of the electronic device 900. The processor 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0142] The processor and memory described above are used together to execute a program stored in the memory. When the program is executed by a computer, it can implement the steps or functions of the methods described in the above embodiments.

[0143] The following components are connected to I / O interface 905: an input section 906 including a keyboard, mouse, touchscreen, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to I / O interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 910 as needed so that computer programs read from it can be installed into storage section 908 as needed. Figure 9 The diagram only shows a portion of the components and does not imply that the computer system 900 only includes... Figure 9 The components shown.

[0144] The systems, apparatuses, modules, or units described in the above embodiments can be implemented by a computer or its associated components. The computer may be, for example, a mobile terminal, a personal computer, a laptop computer, a tablet computer, an industrial computer, a server, or a combination thereof.

[0145] Although not shown, in embodiments of this disclosure, a program product is provided, the program product comprising a computer program configured to be run to implement the methods of any embodiment of this disclosure.

[0146] Although not shown, in embodiments of this disclosure, a storage medium is provided storing a computer program configured to be executed to implement the methods of any of the embodiments of this disclosure.

[0147] Storage media in embodiments of this disclosure include articles that are permanent and non-permanent, removable and non-removable, capable of storing information by any method or technology. Examples of storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0148] The methods, programs, systems, apparatuses, etc., of the embodiments of this disclosure can be executed or implemented in one or more networked computers, or practiced in a distributed computing environment. In the embodiments of this specification, in these distributed computing environments, tasks can be performed by remote processing devices connected via a communication network.

[0149] Those skilled in the art will understand that the embodiments described in this specification can be provided as methods, systems, or computer program products. Therefore, those skilled in the art will realize that the functional modules / units or controllers and related method steps described in the above embodiments can be implemented in software, hardware, or a combination of both.

[0150] Unless explicitly stated otherwise, the actions or steps of the methods or procedures described in the embodiments of this disclosure do not necessarily have to be performed in a specific order and can still achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0151] This document describes several embodiments of the present disclosure; however, for the sake of brevity, the descriptions of the embodiments are not exhaustive, and identical or similar features or portions between the embodiments may be omitted. In this document, "one embodiment," "some embodiments," "example," "specific example," or "some examples" refers to at least one embodiment or example applicable to the present disclosure, but not all embodiments. The above terms do not necessarily refer to the same embodiment or example. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples.

[0152] The exemplary systems and methods of this disclosure have been specifically shown and described with reference to the foregoing embodiments, and are merely examples of the best mode for implementing the systems and methods. Those skilled in the art will understand that various changes can be made to the embodiments of the systems and methods described herein without departing from the spirit and scope of this disclosure as defined in the appended claims when implementing the systems and / or methods.

Claims

1. A model invocation method, characterized in that, include: Receive a request from a target customer to invoke a target model, where the target model is associated with one or more sub-models; In the case of having a model routing configuration for the target customer, the model routing identifier value of the call request is obtained, wherein the model routing configuration includes data segments corresponding to the target model and the one or more sub-models respectively, and the data segments of the target model and the one or more sub-models are determined according to the traffic ratio of the target model and the one or more sub-models respectively; Based on the model routing identifier value and the data segment, determine the actual operating model for the target customer from the target model and the one or more sub-models.

2. The model invocation method according to claim 1, characterized in that, The method further includes: In the absence of a model routing configuration for the target customer, the target model is determined as the actual operating model for the target customer.

3. The model invocation method according to claim 1 or 2, characterized in that, The method further includes: Obtain the execution result of the actual running model in response to the call request, and return the execution result as the call result of the call request to the target client.

4. The model invocation method according to claim 1 or 2, characterized in that, The method further includes: Generate model routing configuration update information and store the model routing configuration update information in a cache, wherein the model routing configuration update information includes one or more of the following: model addition information, model replacement information, model deletion information, and traffic ratio adjustment information; If the cache contains model routing configuration update information associated with the target model and the target customer, update the model routing configuration according to the model routing configuration update information.

5. The model invocation method according to claim 4, characterized in that, The step of updating the model routing configuration according to the model routing configuration update information includes: After the model routing configuration is updated by the model routing configuration update information, in the event of an abnormal call result, the model routing configuration update information is generated based on any saved historical model routing configuration.

6. The model invocation method according to claim 3, characterized in that, The method further includes: Record the call result and the corresponding target model or sub-model; The target model or sub-model corresponding to the call result whose fluctuation exceeds a preset threshold is identified as a fluctuation anomaly model. Adjust the traffic ratio between the target model and the sub-model to reduce the traffic ratio corresponding to the fluctuation anomaly model; The model routing configuration update information is generated based on the adjusted traffic ratio.

7. The model invocation method according to claim 3, characterized in that, The step of obtaining the execution result of the actual running model in response to the call request includes: In the event of a database failure on which the actual operating model depends, database failure information is generated; The method further includes: Based on the database fault information, adjust the traffic ratio between the target model and the sub-model to reduce the traffic ratio of the actual running model; The model routing configuration update information is generated based on the adjusted traffic ratio.

8. The model invocation method according to claim 1, characterized in that, The step of obtaining the model route identifier value of the call request includes: Generate hash encoding information based on the input parameter information in the call request; The hash code information is moduloed to obtain a random number within a preset range, and the random number is used as the model routing identifier value.

9. The model invocation method according to claim 8, characterized in that, The step of determining the actual running model from the target model and the one or more sub-models based on the model routing identifier value and the model routing configuration includes: Based on the traffic ratio corresponding to the target model and the preset range, obtain the first data segment within the preset range corresponding to the target model; Based on the traffic ratio corresponding to each sub-model and the preset range, obtain the second data segment within the preset range corresponding to each sub-model; In the first data segment and the second data segment, the target data segment in which the model routing identifier value is located is determined, and the target model or sub-model corresponding to the target data segment is taken as the actual running model.

10. The model invocation method according to claim 1, characterized in that, The target model and the one or more sub-models rely on different datasets during runtime.

11. A model invocation device, characterized in that, include: The receiving unit is configured to receive a call request from a target client for a target model, wherein the target model is associated with one or more sub-models; The acquisition unit is configured to acquire the model route identifier value of the call request when there is a model route configuration for the target customer, wherein the model route configuration includes data segments corresponding to the target model and the one or more sub-models respectively, and the data segments of the target model and the one or more sub-models are determined according to the traffic ratio of the target model and the one or more sub-models respectively; The determining unit is configured to determine the actual running model from the target model and the one or more sub-models based on the model routing identifier value and the model routing configuration.

12. An electronic device, characterized in that, include: A processor and a memory storing a computer program, the processor being configured to implement the method as described in any one of claims 1 to 10 when the computer program is executed.

13. A program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 10.

Images