Data offset adaptive model service system and application method thereof

Abstract

The application relates to the field of machine learning, and discloses a data drift adaptive model service system and an application method thereof. The application method comprises the following steps: determining a staging area, performing prediction on received input data through a prediction model to obtain a prediction value, and storing the input data, a real label and the prediction value into the staging area; when a group of input data, real labels and prediction values corresponding to each other are all stored into the staging area, determining that a group of triad sets is complete, then calling a drift detection method of a drift detector, and passing the triad as a parameter to the drift detection method for drift detection; if the drift detection result is that there is drift, a model updating method is called to update the prediction model. The application solves the problem of how the model service self-adaptively copes with data drift.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to a data offset adaptive model service system and its application method. Background Technology

[0002] Data skew can occur during the operation of machine learning models, which can lead to a deterioration in model performance. This requires model deployers to take measures to address the issue.

[0003] Existing technologies include the following methods for addressing data offset:

[0004] 1. By using Seldon Core to construct model services, users only need to provide model files of common machine learning frameworks such as TensorFlow and scikit-learn that meet the requirements of Seldon Core, or implement the code interface defined by Seldon Core for custom models, and then construct the corresponding model service for the given model through the related facilities provided by Seldon Core.

[0005] The main drawbacks of Seldon Core as a machine learning model deployment system that supports model performance monitoring include:

[0006] 1. Seldon Core does not support automatically updating the model when a data offset is detected. It only supports reporting the data offset to the user, who then has to manually update the model. This is not convenient for the user.

[0007] 2. Seldon Core only supports running in Kubernetes clusters. Due to the complexity of Kubernetes, deploying Kubernetes is inconvenient or even impossible for many model deployment environments, which limits the application scope of Seldon Core.

[0008] 3. Seldon Core only supports offset detection algorithms based on input distribution, and does not support offset detection algorithms based on model performance.

[0009] II. River is an open-source online machine learning library written in Python, containing numerous online machine learning algorithms. River has a class called `DriftRetrainingClassifier`. Constructing an instance of this class requires an online machine learning classifier object with `predict_one(x)` and `learn_one(x,y)` methods and an offset detector object. Simultaneously, the instance of this class also has `predict_one(x)` and `learn_one(x,y)` methods, so its usage is exactly the same as a regular online machine learning classifier object. An instance of the `DriftRetrainingClassifier` class can be considered a classifier with adaptive capabilities to data offsets, implicitly including offset detection and model updates.

[0010] The main drawbacks of River's DriftRetrainingClassifier class, as a classifier that adapts to data offsets, in model service scenarios include:

[0011] 1. Robustness issues exist.

[0012] 2. The `predict_one(x)` method of the sub-classifier is called both when the model service receives input data (the `predict_one(x)` method of the `DriftRetrainingClassifier` class simply forwards `x` to the sub-classifier) ​​and when the model service receives the true labels (the `learn_one(x,y)` method of the `DriftRetrainingClassifier` class). This means the sub-classifier's `predict_one(x)` method is called twice for each piece of input data. For some complex machine learning algorithms, doubling the number of predictions results in a significant increase in computational cost.

[0013] 3. The DriftRetrainingClassifier class was not designed with multi-threaded calling scenarios in mind. Simply locking the predict_one(x) and learn_one(x,y) methods can introduce unnecessary blocking in some cases, making the DriftRetrainingClassifier class unsuitable for model service scenarios.

[0014] 4. The DriftRetrainingClassifier class only supports giving online machine learning classifiers the ability to adapt to data offsets, but does not support ordinary machine learning classifiers or non-classification problems such as regression, which limits its application scope. Summary of the Invention

[0015] This invention provides a data offset adaptive model service system and its application method, which solves the problem of how model services can adaptively respond to data offset.

[0016] A data offset adaptive model service system, comprising:

[0017] The storage module is used to store input data, true labels, and predicted values;

[0018] A prediction model is used to execute model prediction methods, make predictions based on input data, and return predicted values.

[0019] Offset detectors are used to detect data offsets based on corresponding input data, ground labels, and predicted values.

[0020] The model update module is used to update the prediction model based on the data offset detection results.

[0021] In one embodiment of the present invention, the data offset adaptive model service system is in the form of a lightweight Python package.

[0022] An application method for a data offset adaptive model service system includes:

[0023] A temporary storage area is determined, and the received input data is predicted using a prediction model to obtain predicted values. The input data, the true labels, and the predicted values ​​are then temporarily stored in the temporary storage area.

[0024] Once a set of corresponding input data, true labels, and predicted values ​​are stored in the temporary storage area, a set of triples is determined to be complete. Then, the offset detection method of the offset detector is called, and the triples are passed as parameters to the offset detection method for offset detection.

[0025] If the offset detection result indicates that an offset exists, the model update method is invoked to update the prediction model.

[0026] In one embodiment of the present invention, temporarily storing the input data and the predicted value in the temporary storage area specifically includes: locking the temporary storage area with a mutex lock, starting a model prediction thread to call a model prediction method, storing the received input data and the model prediction thread in the temporary storage area, and releasing the mutex lock; performing prediction based on the input data using the model prediction method to obtain a predicted value; and after the model prediction thread has finished executing, locking the temporary storage area with a mutex lock, replacing the model prediction thread with the predicted value, and releasing the mutex lock.

[0027] In one embodiment of the present invention, the step of invoking the offset detection method of the offset detector and passing the triplet as a parameter to the offset detection method for offset detection specifically includes: when a set of triplets is ready in the temporary storage area, sending the input data, true label and predicted value included in the set of triplets to the offset detector for offset detection; if there is a data offset, invoking the model update method; if there is no data offset, continuing to wait for the next set of triplets to be collected.

[0028] In one embodiment of the present invention, while waiting for a set of triples to be assembled, the method further includes: if the waiting time for a set of triples to be assembled exceeds a preset threshold, then skip the currently waiting triples and proceed to the operation on the next input data.

[0029] In one embodiment of the present invention, if the offset detection result indicates the existence of an offset, the model update method is invoked to update the prediction model. Specifically, this includes: if the offset detector reports a data offset, the model cloning method is invoked to clone a copy of the current prediction model; the model update method is invoked on the model copy, and the number of labeled samples specified by the offset detector is passed into the model update method; after the model copy is updated, the model copy is used as the new prediction model to replace the original prediction model.

[0030] In one embodiment of the present invention, the input data and its corresponding real label have the same identifier; the corresponding input data, real label and predicted value are identified as a set of triples according to the identifier.

[0031] An application device for a data offset adaptive model service system includes:

[0032] At least one processor; and,

[0033] The memory is communicatively connected to the at least one processor via a bus; wherein,

[0034] The memory stores instructions that can be executed by the at least one processor to implement the method as described in any of the above embodiments.

[0035] A non-volatile storage medium storing computer-executable instructions, which are executed by a processor to implement the method as described in any of the above embodiments.

[0036] This invention provides a data offset adaptive model service system and its application method, which has at least the following beneficial effects:

[0037] 1. It is highly robust and can properly handle special situations such as delayed arrival, out-of-order arrival or even missing real labels, as well as the arrival of new input data during model updates.

[0038] 2. High degree of automation: offset detection and model updates are performed automatically, and no user intervention is required after the model service is started.

[0039] 3. It has a wide range of applications. First, it supports a variety of supervised machine learning problems (classification, regression, etc.) and various offset detection algorithms (based on model performance and based on input distribution). Second, it is a lightweight Python package, which is convenient and flexible to deploy and suitable for scenarios with limited system resources.

[0040] 4. Excellent performance. First, it was designed with multi-threaded calling scenarios in mind from the beginning. When related interfaces are called by multiple threads, it can automatically perform inter-thread synchronization and avoid unnecessary blocking. Second, it does not make unnecessary calls to the user model. Attached Figure Description

[0041] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0042] Figure 1 This is an example diagram of the code for calling objects in the Seldon Core model service based on cURL, provided in an embodiment of the present invention.

[0043] Figure 2 This is a diagram illustrating the composition of the data offset adaptive model service system provided in this embodiment of the invention.

[0044] Figure 3 This is a logical framework diagram of the data offset adaptive model service system provided in an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram illustrating the application method steps of a data offset adaptive model service system provided in an embodiment of the present invention;

[0046] Figure 5 A flowchart illustrating a method for adaptively responding to data offsets in model services provided in this embodiment of the invention;

[0047] Figure 6 This is a schematic diagram of an application device for a data offset adaptive model service system provided in an embodiment of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described clearly and completely below in conjunction with specific embodiments. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0049] It should be noted that those skilled in the art will understand, explicitly and implicitly, that the embodiments described in this invention can be combined with other embodiments without conflict. Unless otherwise defined, the technical or scientific terms used in this invention should be understood in their ordinary sense by those skilled in the art. The terms "a," "an," "an," "the," etc., used in this invention do not indicate quantity limitation and can represent singular or plural. The terms "comprising," "including," "having," and any variations thereof used in this invention are intended to cover non-exclusive inclusion; the terms "first," "second," "third," etc., used in this invention are merely to distinguish similar objects and do not represent a specific ordering of objects.

[0050] The following explains some of the concepts involved in this invention.

[0051] Model service: Deploying trained machine learning models to online environments and enabling them to make predictions on input data from these online environments. In some scenarios, the model service can receive not only input data from external sources but also the true labels of past input data.

[0052] Data offset: The change in the statistical distribution of input data and real labels in the online environment over time.

[0053] Offset detection algorithm: An algorithm for detecting data offset.

[0054] Offset detector: At the code level, it is an instance of the class corresponding to a certain offset detection algorithm. The values ​​of various parameters such as the confidence level of the algorithm are all defined in this instance. This instance can return its judgment on whether a data offset has occurred through its methods.

[0055] Robustness: In this invention, it means that the model service framework can properly cope with various abnormal situations in the online environment and will not cause operational anomalies when abnormal situations occur.

[0056] A common development and application process for machine learning models generally involves developers first designing a machine learning algorithm for a target scenario or directly selecting an existing algorithm. Next, they prepare a certain amount of data representative of the target scenario (i.e., a dataset). Then, they train a well-performing model using the prepared algorithm on the dataset. Finally, they deploy the trained model to an online environment. According to machine learning theory, a prerequisite for a model that performs well offline to also perform well online is that the data the model encounters online follows the same statistical distribution as the data used to train the model. In reality, this prerequisite is not always true in many real-world scenarios. For example, factors such as aging production equipment, changes in production processes, and differences in temperature and humidity can all cause the online data distribution to change over time, deviating from the training set distribution. This phenomenon of online data distribution changing over time is called concept drift.

[0057] Data skew can degrade model performance, requiring developers to take measures. A simple approach is to retrain the model periodically with recently labeled online data, since online data distribution changes over time. This updates the model's training set, or "knowledge," from an outdated offline version to a more recent one. While it can't guarantee perfect consistency with the current data distribution, it's at least expected to be close. The drawback of this "periodic retraining" is that it can't guarantee the retraining schedule aligns with the data skew. Retraining might occur when no data skew has occurred, wasting computational resources, or it might fail to occur when a data skew has happened, negatively impacting model performance.

[0058] To make model retraining timing more reasonable, many offset detection algorithms have been proposed. Offset detection algorithms can be broadly divided into two categories: one is offset detection algorithms based on model performance, which monitor changes in model prediction performance over time and report data offsets when the decline in model performance is statistically significant; the other is offset detection algorithms based on input distribution, which directly monitor changes in the distribution of input data over time. Their advantage over the former is that they can work without requiring real labels on online data (real labels are needed to evaluate model prediction performance), but they also suffer from problems such as high computational cost and excessive sensitivity.

[0059] Seldon Core is an open-source project led by Seldon Technologies Limited in the UK. It's a Kubernetes-based machine learning model deployment system. Users only need to provide model files from common machine learning frameworks such as TensorFlow and scikit-learn that conform to Seldon Core's requirements, or implement the code interface defined by Seldon Core for their custom models. Then, they can use the facilities provided by Seldon Core to construct corresponding model services for a given model. Callers of the model service can invoke it via HTTP or gRPC protocols using tools like cURL. Figure 1 This is a cURL-based call example where the HTTP request sent to the Seldon Core model service contains the model input data, and the response from the Seldon Core model service contains the model predictions.

[0060] Figure 1 In this code snippet, the cURL command sends a POST request to a specified URL, along with JSON data. The target of this request is Seldon's iris-model prediction API. Below is a detailed explanation of this code:

[0061] curl-X POST http: / / <ingress> / seldon / seldon / iris-model / api / v1.0 / predictions: curl is a command-line tool used to send HTTP requests; -X POST specifies that the request should be sent using the POST method; http: / / <ingress> / seldon / seldon / iris-model / api / v1.0 / predictions is the target URL for the request. <ingress>It needs to be replaced with the actual Ingress address.

[0062] -H'Content-Type:application / json': This specifies that the Content-Type in the request header is application / json, indicating that the data being sent is in JSON format.

[0063] -d'{"data":{"ndarray":[[1,2,3,4]]}}': The -d parameter is followed by the JSON data to be sent. The data here is a two-dimensional array named "ndarray" containing one element, which is also an array containing 4 elements: 1,2,3,4.

[0064] The JSON data structure of the response is as follows:

[0065] The value of "meta" is an empty object {}.

[0066] "data" contains two parts: "names" and "ndarray".

[0067] "names" is an array containing three string elements: "t:0", "t:1", and "t:2".

[0068] "ndarray" is an array containing one element, which is also an array. This internal array contains three floating-point numbers: 0.000698519453116284, 0.00366803903943576, and 0.995633441507448.

[0069] In summary, this code sends a POST request to Seldon's iris-model prediction API via the cURL command. The request includes a two-dimensional array named "ndarray" as input data, and receives a response containing some prediction results data.

[0070] Seldon Core supports model performance monitoring. Specifically, users first need to select an input distribution-based offset detection algorithm from the Alibi Detect offset detection algorithm library. Based on this algorithm, they then "train" an offset detector on the model training set that possesses knowledge of the data distribution of the training set. Finally, they construct an offset detection service using the facilities provided by Seldon Core. When the offset detection service detects a data offset, it reports the situation to the Seldon Core user through logs and other means. At this point, the user should update the model through retraining or other methods to maintain good model performance.

[0071] The main drawbacks of Seldon Core as a machine learning model deployment system that supports model performance monitoring include:

[0072] 1. Seldon Core does not support automatically updating the model when a data offset is detected. It only supports reporting the data offset to the user, who then has to manually update the model. This is not convenient for the user.

[0073] 2. Seldon Core only supports running in Kubernetes clusters. Due to the complexity of Kubernetes, deploying Kubernetes is inconvenient or even impossible for many model deployment environments, which limits the application scope of Seldon Core.

[0074] 3. Seldon Core only supports offset detection algorithms based on input distribution, and does not support offset detection algorithms based on model performance.

[0075] River is an open-source online machine learning library written in Python, containing numerous online machine learning algorithms. The biggest difference between online machine learning and regular machine learning is that online machine learning models can learn from labeled samples one by one without needing to review previously learned samples. This allows online machine learning models to learn and make predictions naturally online. Accordingly, River's base class for online machine learning algorithms contains two core methods: `predict_one(x)` and `learn_one(x,y)`, where the former predicts a single sample, and the latter learns from a labeled sample.

[0076] In addition to online machine learning algorithms, River also includes several offset detection algorithms based on model performance, along with numerous supporting online machine learning facilities. River's base class for offset detection algorithms contains two core methods: `update(v)` and `drift_detected()`. The `update(v)` method passes a single prediction performance metric to the offset detector. This metric can be the accuracy of the prediction (for classification problems) or the mean squared error (for regression problems), etc. In River, the offset detector monitors changes in model prediction performance over time by having the user call its `update(v)` method multiple times. The `drift_detected()` method returns the offset detector's judgment on whether a data offset has occurred. It's important to note that unless data offsets occur continuously, a single data offset will only be reported once by the `drift_detected()` method and will not be reported repeatedly.

[0077] River has a `DriftRetrainingClassifier` class. Constructing an instance of this class requires an online machine learning classifier object with `predict_one(x)` and `learn_one(x,y)` methods and an offset detector object. Simultaneously, the instance of this class also has `predict_one(x)` and `learn_one(x,y)` methods, so its usage is exactly the same as a regular online machine learning classifier object. An instance of the `DriftRetrainingClassifier` class can be considered a classifier that adapts to data offsets. Specifically, its `learn_one(x,y)` method includes the following steps, implicitly including offset detection and model updates (simplified for ease of understanding):

[0078] 1. Call the predict_one(x) method of the subclassifier (i.e., the classifier object used to construct an instance of the DriftRetrainingClassifier class), and denote the return value as y_pred.

[0079] 2. If y == y_pred, it means that the prediction result of the first step is correct, and the update(v) method of the offset detector is called, where v == 1; otherwise, update(0) is called.

[0080] 3. Call the drift_detected() method of the offset detector. If it returns True, reset the classifier (forget all learned knowledge).

[0081] The main drawbacks of River's DriftRetrainingClassifier class, as a classifier that adapts to data offsets, in model service scenarios include:

[0082] 1. Robustness issues exist. For example, consider the following scenario: The model service first receives n input data points sequentially, denoted as x1, x2, ..., xn, and then sequentially receives n true labels yn, yn-1, ..., y1 (the true label of xn is received first, and the true label of x1 is received last, i.e., the order of receiving the true labels is the reverse of the order of receiving the input data). When the model service receives xi, predict_one(xi) will be called; when the model service receives yi, learn_one(xi,yi) will be called. If a data shift occurs between x1 and xn, causing the model performance to deteriorate, since the shift detector's update(v) method is called by the learn_one(x,y) method of the DriftRetrainingClassifier class, and the order of calling learn_one(x,y) is the same as the order in which the model service receives the true labels, and the order of the true labels is the reverse of the order of the input data, then the shift detector will receive the prediction performance index of xn first and the prediction performance index of x1 last. The update(v) method will then show an improved model performance. Some offset detection algorithms based on model performance do not detect situations where the model performance is getting better and better (which is unlikely to occur in real-world scenarios). Therefore, the situation discussed in this paragraph may lead to unexpected results for the model service.

[0083] 2. The `predict_one(x)` method of the sub-classifier is called both when the model service receives input data (the `predict_one(x)` method of the `DriftRetrainingClassifier` class simply forwards `x` to the sub-classifier) ​​and when the model service receives the true labels (step 1 of the `learn_one(x,y)` method of the `DriftRetrainingClassifier` class). This means the `predict_one(x)` method of the sub-classifier is called twice for each piece of input data. For some complex machine learning algorithms, doubling the number of predictions results in a significant increase in computational cost.

[0084] 3. The DriftRetrainingClassifier class was not designed with multi-threaded call scenarios in mind. Simply locking the predict_one(x) and learn_one(x,y) methods can introduce unnecessary blocking in some cases. This makes the DriftRetrainingClassifier class unsuitable for model service scenarios (concurrent requests are normal for model services).

[0085] 4. The DriftRetrainingClassifier class only supports giving online machine learning classifiers the ability to adapt to data offsets, but does not support ordinary machine learning classifiers or non-classification problems such as regression, which limits its application scope.

[0086] The purpose of this invention is to address the problem of data offset in production environments leading to the degradation of machine learning model performance. As a framework, it constructs a model service that is adaptive to data offset and has good robustness based on three main elements: the model provided by the model deployer, the offset detector, and the model update code logic when data offset is detected (hereinafter referred to as the model update method). At the same time, it solves as many problems as possible in existing related technologies.

[0087] To address the aforementioned problems, this invention proposes a data offset adaptive model service system and its application method. The details are described below.

[0088] Figure 2 A data offset adaptive model service system architecture diagram provided by the present invention includes:

[0089] Storage module 210 is used to store input data, true labels and predicted values;

[0090] Prediction model 220 is used to execute the model prediction method, make predictions based on the input data, and return the predicted values;

[0091] Offset detector 230 is used to detect data offset based on the corresponding input data, true labels and predicted values;

[0092] The model update module 240 is used to update the prediction model based on the data offset detection results.

[0093] In one embodiment of the present invention, the data offset adaptive model service system is in the form of a lightweight Python package. The system can also be implemented using other programming languages ​​besides Python (such as Java and C++), allowing the present invention to support machine learning ecosystems using other programming languages.

[0094] Specifically, the data offset adaptive model service system provided by this invention is morphologically a Python package, in which the core class is Service, representing the model service. To construct a Service object, the user needs to provide three parameters: model, drift_detector, and model_updator. Figure 3 As shown, where:

[0095] 1. The model needs to be a BaseModel object. The BaseModel class is an abstract base class containing two abstract methods: predict(x), which accepts a single input data point and returns the corresponding model prediction, and clone(), which clones a new model with the exact same structure, weights, and other prediction logic as the original model and returns it. This design of the BaseModel class unifies various supervised machine learning problems, including classification and regression problems. To facilitate users in providing BaseModel objects, this invention provides adapters for common model types such as PyTorch, TensorFlow, and scikit-learn.

[0096] 2. The drift detector `drift_detector` needs to be a `BaseDriftDetector` object. The `BaseDriftDetector` class is an abstract base class containing an abstract method: `step(x,y,y_pred)`. The `step(x,y,y_pred)` method is called in the order the model service receives the input data, one call corresponding to one input data point. `x`, `y`, and `y_pred` represent the input data itself, the corresponding ground truth label, and the model's predicted value, respectively. After each triplet of `x`, `y`, and `y_pred`, the `step(x,y,y_pred)` method should return a judgment on whether a data drift has occurred. If the judgment is true, it should also return the range of data belonging to the new data distribution, i.e., how many recent data points are considered to belong to the new distribution. If the user-provided drift detector is based on model performance, the predicted performance metric can be calculated from the values ​​of `y` and `y_pred` in the `step(x,y,y_pred)` method for drift detection (the type of metric needs to be specified by the user); if it is based on the input distribution, the value of `x` can be directly used for drift detection. The BaseDriftDetector class's design unifies offset detection algorithms based on model performance and those based on input distribution. To facilitate users in providing BaseDriftDetector objects, this invention provides an adapter for offset detection algorithm classes in the River library.

[0097] 3. The model update method `model_updator` needs to be a `BaseModelUpdator` object. The `BaseModelUpdator` class is an abstract base class containing an abstract method: `update(model, x_arr, y_arr)`. The `update(model, x_arr, y_arr)` method is called after the offset detector reports the data offset. `model` is the model to be updated, and `x_arr` and `y_arr` are arrays of the input values ​​and ground truth labels from the most recent samples, respectively. When called, the `update(model, x_arr, y_arr)` method should update the model using the passed-in most recent stripe label samples, through retraining or fine-tuning (for deep learning models).

[0098] The Service class provides two core methods: recv_x(x,id) and recv_y(y,id), which are used to receive input data and ground truth labels, respectively. The id parameter enables the model service to identify which input data a ground truth label corresponds to when it receives a ground truth label.

[0099] Figure 4 This is a schematic diagram illustrating the steps of an application method for a data offset adaptive model service system provided in an embodiment of the present invention. Figure 4 The application method of the data offset adaptive model service system shown can be executed by a data offset adaptive model service system device. This device can be located on a server, such as a physical server containing independent hosts, a virtual server hosted in a host cluster, or a cloud server. This embodiment of the invention does not limit this. The application method of the data offset adaptive model service system may include the following steps:

[0100] S410: Determine the temporary storage area, predict the received input data using the prediction model, obtain the predicted value, and temporarily store the input data, the true label, and the predicted value in the temporary storage area.

[0101] In one embodiment of the present invention, the input data and the predicted value are temporarily stored in a temporary storage area, specifically including: locking a mutex on the temporary storage area, starting a model prediction thread to call a model prediction method, storing the received input data and the model prediction thread in the temporary storage area, and releasing the mutex; performing prediction based on the input data using the model prediction method to obtain the predicted value; and after the model prediction thread has finished executing, locking a mutex on the temporary storage area, replacing the model prediction thread with the predicted value, and releasing the mutex.

[0102] The process of temporarily storing the real label in the temporary storage area includes: setting a mutex lock on the temporary storage area, temporarily storing the real label in the temporary storage area, and releasing the mutex lock.

[0103] Specifically, the temporary storage area can be implemented in various ways, such as memory-based and disk-based.

[0104] Based on the temporary storage area, the following section outlines the general logic of the recv_x(x,id) and recv_y(y,id) methods:

[0105] 1. The recv_x(x, id) method:

[0106] a. Set a mutex lock on the temporary storage area, start a new thread to call the model prediction method (i.e., the predict(x) method of BaseModel), temporarily store the x value and the thread, and release the lock.

[0107] b. Wait for the model prediction thread to finish executing, acquire a mutex lock on the temporary storage area, replace the model prediction thread with the y_pred value in the temporary storage area, and release the lock.

[0108] c. Return the model's predicted values.

[0109] 2. The recv_y(y, id) method:

[0110] a. Apply a mutex lock to the temporary storage area, temporarily store the y value, and then release the lock.

[0111] S420: After a set of corresponding input data, true labels and predicted values ​​are stored in the temporary storage area, a set of triples is determined to be complete. Then, the offset detection method of the offset detector is called, and the triples are passed as parameters to the offset detection method for offset detection.

[0112] In one embodiment of the present invention, the offset detection method of the offset detector is invoked, and the triplet is passed as a parameter to the offset detection method for offset detection. Specifically, the method includes: when a set of triplets is ready in the temporary storage area, the input data, true label and predicted value included in the set of triplets are sent to the offset detector for offset detection; if there is a data offset, the model update method is invoked; if there is no data offset, the system continues to wait for the next set of triplets to be collected.

[0113] In one embodiment of the present invention, the input data and its corresponding real label have the same identifier; the corresponding input data, real label and predicted value are identified as a set of triples according to the identifier.

[0114] In one embodiment of the present invention, while waiting for a set of triples to be assembled, the method further includes: if the waiting time for a set of triples to be assembled exceeds a preset threshold, then skip the currently waiting triples and proceed to the operation on the next input data.

[0115] Specifically, in addition to the temporary storage area, the model service also requires an "offset detection thread" to send the temporarily stored data to the offset detector and perform model updates after the offset detector reports the data offset. It should be noted that the offset detection thread can be replaced by an offset detection process, and there is no essential difference in effect between the two.

[0116] Generally speaking, the thread will perform the following operations on each x value in the order they arrive:

[0117] First, wait for the corresponding y-value and y_pred value to be ready in the temporary storage area, and then send the x-value, y-value, and y_pred value to the step(x,y,y_pred) method of the offset detector. If the waiting time for the y-value and y_pred value is too long, skip the x-value and proceed to the operation on the next x-value (waiting for its corresponding y-value and y_pred value to be collected).

[0118] S430: If the offset detection result indicates that an offset exists, then the model update method is called to update the prediction model.

[0119] In one embodiment of the present invention, if the offset detection result indicates that an offset exists, the model update method is invoked to update the prediction model. Specifically, this includes: if the offset detector reports a data offset, the cloning method of the prediction model is invoked to clone a copy of the current prediction model; the model update method is invoked on the model copy, and the number of labeled samples specified by the offset detector is passed into the model update method; after the model copy is updated, the model copy is used as the new prediction model to replace the original prediction model.

[0120] Specifically, if the offset detector reports a data offset, a copy of the model is cloned using the `BaseModel.clone()` method. Then, the model update method is called on the copy, passing in the number of the most recent labeled samples specified by the offset detector. After the model update is complete, the original model is replaced with the new model.

[0121] When the offset detector reports data offsets, the labeled samples passed to the model update method can be the most recent samples specified by the offset detector, a fixed number of the most recent or future samples, or other selection methods.

[0122] like Figure 5 The illustrated process can be used to summarize the design of the "temporary storage area + offset detection thread" in this invention. Through this design, special cases such as delayed arrival of real labels, out-of-order arrival, or even missing labels, as well as new input data arriving during model updates, are properly handled.

[0123] Through this invention, deployers of machine learning models only need to select a suitable offset detector and model update method for the target scenario to construct a model service capable of adapting to data offsets. This invention saves machine learning model deployers significant time and effort: First, the model service's adaptive mechanism to data offsets is fully automatic; users do not need to monitor the offset detector's reports on data offsets, nor do they need to prepare labeled samples for updating the model when the offset detector reports data offsets. Second, users do not need to act as maintenance personnel to handle various special situations in the online environment that cause the data offset adaptive mechanism to malfunction, because the data offset adaptive mechanism of this invention has been carefully designed to properly handle most special situations. Third, deploying the model service constructed using this invention incurs almost no additional burden, because this invention is simply a Python package, and the constructed model service is simply a Python object. In most environments, as long as a machine learning model in the form of a Python object can be deployed, the model service constructed by this invention can also be deployed.

[0124] This invention also provides an application device for a corresponding data offset adaptive model service system, such as... Figure 6 As shown, it includes:

[0125] The system includes at least one processor 602, a communication interface 604, a memory 606, and a communication bus 608; wherein the processor 602, the communication interface 604, and the memory 606 communicate with each other through the communication bus 608; the processor 602 can call the logical instructions stored in the memory 606 to cause at least one processor 602 to execute the steps of the above embodiments.

[0126] Based on the same idea, some embodiments of the present invention also provide media corresponding to the above methods.

[0127] Some embodiments of the present invention provide a storage medium storing computer-executable instructions, which are executed by a processor to implement the steps of the embodiments described above.

[0128] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the steps of the application method of a data offset adaptive model service system provided in the above embodiments.

[0129] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the device, medium, and program product embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0130] The devices, media, and program products and methods provided in the embodiments of the present invention are one-to-one correspondences. Therefore, the devices, media, and program products also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices, media, and program products will not be repeated here.

[0131] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process method or product that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process method or product. Without further limitation, an element defined by the phrase "comprising one…" does not exclude the presence of other identical elements in the process method or product that includes that element.

[0132] The above are merely embodiments of the present invention and are not intended to limit the invention. Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.< / ingress> < / ingress> < / ingress>

Claims

1. A data offset adaptive model service system, characterized in that, include: The storage module is used to store input data, true labels, and predicted values; A prediction model is used to execute model prediction methods, make predictions based on input data, and return predicted values. Offset detectors are used to detect data offsets based on corresponding input data, ground labels, and predicted values. The model update module is used to update the prediction model based on the data offset detection results; The application methods of the data offset adaptive model service system include: A temporary storage area is defined. The received input data is predicted using a prediction model to obtain predicted values. The input data, ground truth labels, and predicted values ​​are temporarily stored in the temporary storage area. Once a set of corresponding input data, ground truth labels, and predicted values ​​are stored in the temporary storage area, a set of triples is determined. Then, the offset detection method of the offset detector is called, and the triples are passed as parameters to the offset detection method for offset detection. If the offset detection result indicates that an offset exists, the model update method is called to update the prediction model. The process of temporarily storing the input data and the predicted value in the temporary storage area specifically includes: locking the temporary storage area with a mutex lock; starting the model prediction thread and calling the model prediction method; storing the received input data and the model prediction thread in the temporary storage area; and releasing the mutex lock; performing prediction based on the input data using the model prediction method to obtain the predicted value; and after the model prediction thread has finished executing, locking the temporary storage area with a mutex lock; replacing the model prediction thread with the predicted value; and releasing the mutex lock. The step of invoking the offset detection method of the offset detector and passing the triplet as a parameter to the offset detection method for offset detection specifically includes: when a set of triplets is ready in the temporary storage area, sending the input data, true label and predicted value included in the set of triplets to the offset detector for offset detection; if there is a data offset, calling the model update method; if there is no data offset, continuing to wait for the next set of triplets to be collected. If the offset detection result indicates an offset, the model update method is invoked to update the prediction model. Specifically, this includes: if the offset detector reports a data offset, the prediction model cloning method is invoked to clone a copy of the current prediction model; the model update method is invoked on the model copy, and the number of labeled samples specified by the offset detector is passed into the model update method; after the model copy is updated, the model copy is used as the new prediction model to replace the original prediction model.

2. The data offset adaptive model service system according to claim 1, characterized in that, The data offset adaptive model service system is in the form of a lightweight Python package.

3. An application method of the data offset adaptive model service system as described in claim 1, characterized in that, include: A temporary storage area is determined, and the received input data is predicted using a prediction model to obtain predicted values. The input data, the true labels, and the predicted values ​​are then temporarily stored in the temporary storage area. Once a set of corresponding input data, true labels, and predicted values ​​are stored in the temporary storage area, a set of triples is determined to be complete. Then, the offset detection method of the offset detector is called, and the triples are passed as parameters to the offset detection method for offset detection. If the offset detection result indicates that an offset exists, the model update method is invoked to update the prediction model.

4. The application method of the data offset adaptive model service system according to claim 3, characterized in that, The input data and the predicted value are temporarily stored in the temporary storage area, specifically including: Set a mutex lock on the temporary storage area, start the model prediction thread to call the model prediction method, store the received input data and the model prediction thread in the temporary storage area, and release the mutex lock. Based on the model prediction method, a prediction is made according to the input data to obtain a prediction value. After the model prediction thread finishes execution, the mutex lock on the temporary storage area is set, the model prediction thread is replaced with the prediction value, and the mutex lock is released.

5. The application method of the data offset adaptive model service system according to claim 3, characterized in that, The step of invoking the offset detection method of the offset detector and passing the triplet as a parameter to the offset detection method for offset detection specifically includes: Once a set of triples is ready in the temporary storage area, the input data, true label, and predicted value included in the set of triples are sent to the offset detector for offset detection. If a data offset exists, the model update method is invoked; If there is no data offset, continue waiting for the next set of triples to be collected.

6. The application method of the data offset adaptive model service system according to claim 5, characterized in that, While waiting for a set of triples to be collected, the method further includes: If the time required to collect a set of triples exceeds a preset threshold, the currently waiting triples are skipped, and the operation proceeds to the next input data.

7. The application method of the data offset adaptive model service system according to claim 3, characterized in that, If the offset detection result indicates the presence of an offset, then the model update method is invoked to update the prediction model, specifically including: If the offset detector reports a data offset, the cloning method of the prediction model is called to clone a copy of the current prediction model. Call the model update method on the model copy, and pass the number of labeled samples specified by the offset detector into the model update method; Once the model copy is updated, it will replace the original prediction model as the new prediction model.

8. The application method of the data offset adaptive model service system according to claim 3, characterized in that, The input data and its corresponding real label have the same identifier; Based on the identifier, the corresponding input data, true labels, and predicted values ​​are identified as a set of triples.

9. An application device for the data offset adaptive model service system as described in claim 1, characterized in that, include: At least one processor; as well as, The memory is communicatively connected to the at least one processor via a bus; wherein, The memory stores instructions executable by the at least one processor, which are executed to implement the method as described in any one of claims 3-8.

10. A non-volatile storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are executed by a processor to implement the method as described in any one of claims 3-8.

Images