Model updating method and apparatus

Abstract

The application discloses a model updating method and device, and belongs to the AI field. The method comprises the following steps: acquiring abnormal explanatory information, wherein the abnormal explanatory information is explanatory information with abnormal explanation content, and the explanatory information is information used for explaining an output result of a machine learning model; adjusting a training sample set of the machine learning model based on the abnormal explanatory information; and acquiring an updated machine learning model, wherein the machine learning model is obtained by training based on the adjusted training sample set. The application can improve the reliability of the updated machine learning model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence (AI), and in particular to a model updating method and device. BACKGROUND

[0002] Machine learning refers to training a machine learning model based on training samples, so that the machine learning model has a prediction capability (such as a category prediction capability) for data other than the training samples. Machine learning, as an important branch of the AI field, has been widely applied in many fields.

[0003] With changes in the environment or rules, the original machine learning model needs to be updated over time. Currently, the updating of the machine learning model is triggered by human experience or updated periodically, and the updating flexibility is low, resulting in low reliability of the updated machine learning model. SUMMARY

[0004] The embodiments of the present application provide a model updating method and device. The technical solution is as follows:

[0005] In a first aspect, a model updating method is provided, which can be executed by a computer device, and the method comprises:

[0006] Obtaining abnormal explanatory information, the abnormal explanatory information being explanatory information of an abnormal explanation content, the explanatory information being information used to explain an output result of a machine learning model; adjusting a training sample set of the machine learning model based on the abnormal explanatory information; and obtaining an updated machine learning model, the machine learning model being trained based on the adjusted training sample set.

[0007] The model updating method provided by the embodiments of the present application adjusts the training sample set of the machine learning model based on the obtained abnormal explanatory information after obtaining the abnormal explanatory information, and obtains the updated machine learning model by using the training sample set. In this way, the automatic updating of the model based on the abnormal explanatory information is realized, the flexibility of the model updating is effectively improved, and the reliability of the updated machine learning model is improved.

[0008] The explanatory information comprises one or more features and feature values of the features, and the process of adjusting the training sample set of the machine learning model based on the abnormal explanatory information comprises: updating a label corresponding to a target training sample in the training sample set to obtain an adjusted training sample set, wherein the target training sample comprises a target feature same as an abnormal feature in the abnormal explanatory information, and a feature value of the target feature is same as an abnormal feature value of the abnormal feature.

[0009] If a feature of a training sample in the training sample set is the same as the abnormal feature, and the feature value of the feature is the same as the abnormal feature value, the probability of the label of the training sample being incorrect is higher. By updating the labels of the training sample set, the probability of incorrect labels can be reduced, and the reliability of the machine learning model trained using the adjusted training sample set can be improved.

[0010] In the embodiments of the present application, there are various ways to adjust the training sample set of the machine learning model based on the abnormal explanatory information. The following examples are used to illustrate the following several optional ways:

[0011] In the first optional way, the computer device adjusts the training sample set of the machine learning model based on the abnormal feature in the abnormal explanatory information. The process includes: updating the label corresponding to the target training sample in the training sample set to obtain an adjusted training sample set, wherein the target training sample includes a target feature that is the same as the abnormal feature in the abnormal explanatory information, and the feature value of the target feature is the same as the abnormal feature value of the abnormal feature.

[0012] In the first optional implementation, the computer device updates the labels corresponding to all target training samples in the training sample set to obtain an adjusted training sample set, wherein the target training sample includes a target feature that is the same as the abnormal feature in the abnormal explanatory information, and the feature value of the target feature is the same as the abnormal feature value of the abnormal feature.

[0013] In the second optional implementation, the computer device updates the labels corresponding to the target training samples in the training sample set step by step to obtain an adjusted training sample set, wherein the target training sample includes a target feature that is the same as the abnormal feature in the abnormal explanatory information, and the feature value of the target feature is the same as the abnormal feature value of the abnormal feature. The determination process of the target training sample is referred to the first optional implementation. The process of updating the labels corresponding to the target training samples in the training sample set step by step to obtain an adjusted training sample set includes:

[0014] Obtain an initial x value, 1≤x<y, and perform at least two retraining processes based on the initial x value until a preset stopping condition is reached, wherein y is the total number of target training samples in the training sample set; determine the training sample set corresponding to the machine learning model with the optimal validation effect in the machine learning models obtained by the at least two retraining processes as the adjusted training sample set;

[0015] Wherein, the retraining process includes: updating the labels corresponding to x target training samples in the training sample set to opposite labels; training the machine learning model based on the training sample set after label updating to obtain a retrained machine learning model; increasing x to obtain an updated x, and performing the retraining process again.

[0016] In an optional implementation, before the obtaining of the abnormal explanation information, the model updating method further includes: after receiving the output result of the machine learning model, generating the explanation information based on the output result; and displaying the generated explanation information. By displaying the explanation information, the user can understand the reason for the output result based on the explanation information.

[0017] For example, the computer device can display the explanation information after generating the explanation information each time. In an optional example, the computer device can simultaneously display the output result and the explanation information corresponding to the same input information.

[0018] The foregoing process of obtaining the abnormal explanation information can include: receiving determination information of the explanation information, the determination information being used to indicate whether the explanation content of the explanation information is abnormal; and determining the explanation information indicated by the determination information as the abnormal explanation information. For example, the explanation information includes one or more features and feature values of the features, and the process of determining the explanation information indicated by the determination information as the abnormal explanation information includes: when the determination information indicates that any feature in the explanation information is an abnormal feature, determining the explanation information as the abnormal explanation information.

[0019] The foregoing output result is generated based on input information by the machine learning model, and in the embodiments of the present application, the process of generating the explanation information based on the output result can include: obtaining input information corresponding to the output result; determining a target sample in the training sample set that is closest to the input information or has a distance less than a preset distance threshold; and generating the explanation information, the explanation information including features and feature values included in the target sample and pre-obtained weights of each feature in the target sample.

[0020] Since the output result is generated based on input information by the machine learning model, the input information or related information of the input information can be included in the explanation information, so as to enable the user to more comprehensively understand the output result.

[0021] In the embodiments of the present application, there are various ways of generating the explanation information based on the output result, and the following two optional implementation modes are taken as examples for description:

[0022] In a first optional implementation, the computer device obtains input information corresponding to the output result; generates explanatory information based on the input information, the explanatory information including features and feature values included in the input information, and weights of each feature in the input information (also referred to as weight coefficients or weights), the weights reflecting contribution degrees of the corresponding features to the output result. For example, the computer device can obtain the weights of each feature in the input information by using an explanatory algorithm. The explanatory algorithm can be a Local Interpretable Model-Agnostic Explanation (LIME) algorithm.

[0023] In a second optional implementation, the computer device obtains input information corresponding to the output result; determines a target sample in the training sample set that is closest to the input information or has a distance less than a preset distance threshold; and generates explanatory information, the explanatory information including features and feature values included in the target sample, and weights of each feature in the target sample that are obtained in advance, the weights reflecting contribution degrees of the corresponding features to a label of the target sample. The weights of each feature in the target sample that are obtained in advance means that the weights have been obtained before the explanatory information is generated. In this way, the explanatory information obtained by the second optional implementation is close to the explanatory information obtained by the first optional implementation, but since the weights in the explanatory information in the second optional implementation are obtained in advance, there is no need to calculate in real time, thereby effectively saving the calculation delay, reducing the user waiting time, and improving the user experience. For example, the distance between the training sample and the input information is calculated by using the Euclidean formula.

[0024] In an optional example, the computer device can obtain weights of features of all samples in the training sample set in advance, and determine a target sample in all samples in the training sample set that is closest to the input information or has a distance less than a preset distance threshold. For example, the computer device can obtain the weights of each feature of each sample in the training sample set by using the LIME algorithm.

[0025] In another optional example, the computer device can obtain weights of features of part of samples in the training sample set in advance, and determine a target sample in the part of samples in the training sample set that is closest to the input information or has a distance less than a preset distance threshold. In this way, the amount of calculation in the process of obtaining the weights can be reduced, and the calculation cost can be reduced. The part of samples are representative samples in the training sample set. In the embodiments of the present application, each sample in the part of samples is referred to as a reference sample. For example, the computer device can obtain the weights of each feature of each reference sample by using the LIME algorithm.

[0026] In the embodiments of the present application, the explanatory information includes a feature value range in which each of the feature values is located; in this way, the user can more clearly understand the reason for obtaining the output result based on the feature value range in which each of the input information feature values is located.

[0027] In addition, the explanatory information includes a probability of each classification supported by the machine learning model corresponding to the output result. In this way, the user can more clearly understand the reason for obtaining the output result.

[0028] In a second aspect, the example embodiments of the present application provide a model updating apparatus. The apparatus includes one or more modules for implementing any of the model updating methods of the first aspect.

[0029] In a third aspect, the embodiments of the present application provide a computer device, such as a terminal or a server. The computer device includes a processor and a memory. The processor generally includes a CPU. The memory is used to store a computer program; the processor is used to implement any of the model updating methods of the first aspect when executing the computer program stored in the memory.

[0030] In a fourth aspect, the embodiments of the present application provide a computer storage medium, which can be non-volatile. The computer storage medium stores a computer program, which, when executed by a processor, causes the processor to implement any of the model updating methods of the first aspect.

[0031] In a fifth aspect, the embodiments of the present application provide a computer program or a computer program product containing computer readable instructions, which, when the computer program or the computer program product is run on a computer device, causes the computer device to execute any of the model updating methods of the first aspect. The computer program product can include one or more program units for implementing the foregoing method.

[0032] In a sixth aspect, the present application provides a chip, such as a CPU. The chip includes a logic circuit, which can be a programmable logic circuit. The chip is used to implement any of the model updating methods of the first aspect when the chip is running.

[0033] In a seventh aspect, the present application provides a chip, such as a CPU. The chip includes one or more physical cores and a storage medium. The one or more physical cores implement any of the model updating methods of the first aspect after reading computer instructions in the storage medium.

[0034] In an eighth aspect, the embodiments of the present application provide a processing system, which includes a storage device and the model updating apparatus of the second aspect or the computer device of the third aspect.

[0035] To sum up, the model updating method provided in the embodiments of the present application adjusts the training sample set of the machine learning model based on the obtained abnormal explanatory information, and thus obtains an updated machine learning model by using the training sample set. In this way, automatic updating of the model based on the abnormal explanatory information is realized, the flexibility of model updating is effectively improved, and thus the reliability of the updated machine learning model is improved.

[0036] In the embodiments of the present application, the training sample set is updated after the abnormal explanatory information is obtained, and then the model is updated. The abnormal explanatory information can reflect the deterioration of the model to a certain extent, and the model can be updated (for example, the model parameters are adjusted) at an appropriate time (that is, when the model deteriorates) without human triggering, so that the adaptive optimization of the model is realized, the updating time of the machine learning model is more accurate, and thus the prediction accuracy of the updated machine learning model is improved. Since the machine learning model is updated after the abnormal explanatory information, the model can be updated in time when the model deteriorates, the timeliness of model updating is ensured, and thus the user experience is improved.

[0037] In addition, in the embodiments of the present application, the machine learning model is updated based on the abnormal explanatory information, and human intervention is not required for model updating, so that the labor cost can be effectively reduced and the training efficiency can be improved. The model can be updated in time based on the abnormal explanatory information in the embodiments of the present application, and the influence of the change of expert experience on the actual application of the machine learning model can be effectively reduced through automatic verification by the user. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 FIG. 1 is a flowchart of a model updating method provided in the embodiments of the present application;

[0039] Figure 2 FIG. 2 is a schematic diagram of a classification result of training sample set classification using a Mini Batch K-Means algorithm provided in the embodiments of the present application;

[0040] Figure 3 FIG. 3 is a schematic tree structure provided in the embodiments of the present application;

[0041] Figure 4 FIG. 4 is a diagram of a division principle of a tree structure provided in the embodiments of the present application;

[0042] Figure 5 FIG. 5 is a display interface diagram of explanatory information provided in the embodiments of the present application;

[0043] Figure 6 FIG. 6 is another display interface diagram of explanatory information provided in the embodiments of the present application;

[0044] Figure 7 is a display interface schematic diagram of explanatory information provided by an embodiment of the present application;

[0045] Figure 8 is an application environment schematic diagram of a processing system involved in a model updating method provided by an embodiment of the present application;

[0046] Figure 9 is Figure 8 is a schematic diagram of an implementation scenario of the processing system shown in the figure;

[0047] Figure 10 is a block diagram of a model updating device provided by an embodiment of the present application;

[0048] Figure 11 is a block diagram of another model updating device provided by an embodiment of the present application;

[0049] Figure 12 is a block diagram of a generation module provided by an embodiment of the present application;

[0050] Figure 13 is a block diagram of still another model updating device provided by an embodiment of the present application;

[0051] Figure 14 shows a structural schematic diagram of a computer device involved in an image processing method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the principles and technical solutions of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0053] Machine learning algorithms, as an important branch of AI, have been widely applied in many fields. From the perspective of learning methods, machine learning algorithms can be divided into supervised learning algorithms, unsupervised learning algorithms, semi-supervised learning algorithms, and reinforcement learning algorithms. Supervised learning algorithms (also known as supervised learning algorithms) are algorithms that can learn from training data and establish a pattern based on the training data, and then use the algorithm or pattern to predict new instances. Training data, also known as training samples, is composed of input data and expected output. The model of the machine learning algorithm, also known as the machine learning model, has an expected output, which is called a label or target, which can be a predicted classification result (called a classification label). The difference between unsupervised learning algorithms and supervised learning algorithms is that the training samples of unsupervised learning algorithms do not have labels, and the machine learning algorithm model analyzes the training samples to obtain certain results. Semi-supervised learning algorithms have some training samples with labels and some without labels, and the number of unlabeled data is much larger than that of labeled data. Reinforcement learning algorithms try to maximize the expected benefits in the environment through rewards or punishments given by the environment to make choices that can maximize benefits.

[0054] Each training sample includes one or more dimensions of features, and each feature has a feature value (also called value). That is, each training sample includes one or more features and the feature values of the features. For example, in the scenario of predicting the classification result of a reimbursement risk document, each training sample includes one or more of the following features: department reimbursement document average, number of (overdue) reimbursement documents per person per year, accounting subject, approval supervisor's permanent residence, number of times of this reimbursement document flow, department violation rate, number of roles in the process of this reimbursement document flow, number of times of this reimbursement document flow, archiving status, or self-approval by the signatory. And each training sample includes the feature values corresponding to the features. For example, a department reimbursement document average of 3 indicates that the feature value of the feature "department reimbursement document average" is 3; and an approval supervisor's permanent residence of Nanjing indicates that the feature value of the feature "approval supervisor's permanent residence" is Nanjing. The feature value can be a numerical value or other types of strings, or a numerical value converted from a string.

[0055] Further, in the scenario of applying supervised learning algorithms or semi-supervised learning algorithms, the training sample can also include a label. For example, in the aforementioned scenario of predicting the classification result of a reimbursement risk document, assuming that the classification result is used to predict whether the reimbursement risk document is in violation, a training sample also includes a label: "violation" or "compliance". The label can also be represented by a number, for example, 1 represents "violation" and 0 represents "compliance".

[0056] The type of machine learning model can be various, for example, in the network anomaly detection scenario, the aforementioned machine learning model can be a model based on a statistical and data distribution algorithm (such as an N-Sigma algorithm), a model based on a distance / density algorithm (such as a local anomaly factor algorithm), a tree model (such as an isolation forest (Iforest)), or a predictive algorithm model (such as an autoregressive integrated moving average model (ARIMA)), etc.

[0057] In the practical task of machine learning, it is necessary to select a representative sample to form a sample set to construct a machine learning model. In a labeled sample, a sample with strong correlation with a category is usually selected as the sample set. The label is used to identify the sample, such as identifying the category of the sample. The machine learning model constructed in this way is also called a classifier or a classification model.

[0058] Since the update of the current machine learning model is triggered by human experience or updated periodically, the update flexibility is low, resulting in low reliability of the updated machine learning model.

[0059] The embodiment of the present application provides a model updating method, which can automatically update the model, thereby improving the update flexibility and ensuring the reliability of the updated machine learning model. The method can be applied to a computer device for running and updating a machine learning model. The computer device can be a computer, a notebook, a server, a server cluster composed of multiple servers, or a cloud computing center. Figure 1 is a flowchart of a model updating method provided by the embodiment of the present application. As shown in Figure 1 the method comprises:

[0060] S101, obtaining a machine learning model.

[0061] The machine learning model can be used to predict the classification result, for example, it can be a binary classification model. The training device for training the machine learning model can continuously collect training samples to obtain a training sample set (also called a training set), and train (such as offline training or online training) based on the collected training sample set (which can be called a historical training sample set), thereby obtaining the machine learning model. The binary classification model refers to a machine learning model with two classification results.

[0062] The training device can be the same as the computer device or different from the computer device. When the training device is different from the computer device, the training device sends the machine learning model to the computer device after training the machine learning model, and the computer device receives the machine learning model accordingly. When the training device is the same as the computer device, the computer device directly obtains the trained machine learning model.

[0063] S102, after receiving the output result of the machine learning model, generating explanatory information based on the output result.

[0064] In the machine learning model usage stage, the computer device inputs the to-be-predicted information into the machine learning model, and the machine learning model generates an output result based on the input information. The computer device receives the output result accordingly. The input information includes one or more features and feature values of the features. In an optional example, the input information can be regarded as a set of features and feature values. For example, the input information includes features F1 and F2, where the feature value of the feature F1 is x1, and the feature value of the feature F2 is x2. The input information is a set {(F1=x1), (F2=x2)}. In another optional example, the input information can be a sequence of feature values, where the order of each feature value in the sequence identifies a feature. For example, the input information includes features F1 and F2, where the feature value of the feature F1 is x1, and the feature value of the feature F2 is x2. The input information is a sequence of feature values: {x1, x2}, where the feature value in the first position of the sequence identifies the feature F1, and the feature value in the second position of the sequence identifies the feature F2.

[0065] After receiving the output result, the computer device generates explanatory information based on the output result. The explanatory information is information for explaining the output result of the machine learning model. It is information readable by a user (such as a business staff). The computer device can display the explanatory information, so that the user can understand the reason for the output result based on the explanatory information.

[0066] As described above, the output result is generated by the machine learning model based on the input information, and therefore, the input information or related information of the input information can be included in the explanatory information, so as to help the user more comprehensively understand the output result.

[0067] In the embodiments of the present application, there are various ways to generate the explanatory information based on the output result, which will be described below by taking two optional implementation manners as examples.

[0068] In a first optional implementation, the computer device obtains input information corresponding to the output result; and generates the explanatory information based on the input information, the explanatory information including features and feature values included in the input information, and weights of the features in the input information, the weights reflecting contribution degrees of the features to the output result. For example, the computer device can obtain the weights of the features in the input information by using an explanatory algorithm. The explanatory algorithm can be a Local Interpretable Model-Agnostic Explanation (LIME) algorithm. For example, the computer device inputs the input information into the LIME algorithm model, and obtains the weights of the features in the input information from the LIME algorithm model. The LIME algorithm model is configured to: for each input information, first train an easily interpretable linear model by using the information and a set of neighboring data of the information to fit a local boundary of a to-be-interpreted model, and then explain a decision basis of the to-be-interpreted model for the information based on the linear model. The weights of the linear model reflect importance of each dimension feature of the input information in the current decision. The to-be-interpreted model is a machine learning model.

[0069] In a second optional implementation, the computer device obtains input information corresponding to the output result; determines a target sample in the training sample set that is closest to the input information or has a distance less than a preset distance threshold; and generates the explanatory information, the explanatory information including features and feature values included in the target sample, and weights of the features in the target sample, the weights reflecting contribution degrees of the features to a label of the target sample. The weights of the features in the target sample are pre-obtained, which means that the weights are obtained before the explanatory information is generated. In this way, the explanatory information obtained by the second optional implementation is similar to the explanatory information obtained by the first optional implementation, but the weights in the explanatory information obtained by the second optional implementation are pre-obtained, and thus the second optional implementation does not need to calculate the weights in real time, thereby effectively saving the calculation time delay, reducing the user waiting time, and improving the user experience. For example, the distance between the training sample and the input information is calculated by using the Euclidean formula.

[0070] In an optional example, the computer device can pre-obtain the weights of the features of all samples in the training sample set, and determine a target sample in the training sample set that is closest to the input information or has a distance less than a preset distance threshold. For example, the computer device can obtain the weights of the features of each sample in the training sample set by using the LIME algorithm.

[0071] In another optional example, the computer device can pre-acquire the weight of the feature of the part of the samples in the training sample set, and determine the target sample closest to the input information or having a distance less than a preset distance threshold. In this way, the operation amount of the weight acquisition process can be reduced, and the operation cost can be reduced. The part of the samples is a representative sample in the training sample set. The part of the samples is referred to as a reference sample in the embodiments of the present application. For example, the computer device can acquire the weight of each feature of each reference sample by using the LIME algorithm.

[0072] It should be noted that when the computer device determines multiple target samples, for example, there are two samples closest to the input information, the computer device can select one target sample from the multiple target samples for subsequent operation by using a preset algorithm. For example, the computer device can randomly select one target sample from the multiple target samples.

[0073] For example, the computer device can acquire the reference samples by using a clustering algorithm before S101. The process includes:

[0074] A1, the training sample set is divided into multiple classes of training samples by using a clustering algorithm, and one reference sample is determined in each class of training samples.

[0075] For example, the clustering algorithm can be a Mini Batch K-Means algorithm. Each sample in the training sample set is regarded as a sample point, and the Mini Batch K-Means algorithm is performed, so that the training sample set can be divided into multiple classes of training samples. Figure 2 is a schematic diagram of a classification result of classifying the training sample set by using the Mini Batch K-Means algorithm provided by the embodiments of the present application. Figure 2 In the figure, it is assumed that each sample in the training sample set includes two-dimensional features, the horizontal axis represents one-dimensional features, and the vertical axis represents another-dimensional features. Based on the feature values of the features of each sample, the Mini Batch K-Means algorithm can divide the training sample set into multiple regions, and each region represents a class of training samples. One sample point in each region is determined as a reference sample. Figure 2 In the figure, the reference sample is represented by “+”. For example, the reference sample of each class of training samples can be a sample selected in the class of training samples by using a preset algorithm, for example, the reference sample of each class of training samples is a sample randomly selected in the class of training samples; or the reference sample of each class of training samples is a sample located at the center of the region corresponding to the class of training samples. Figure 2For example, the reference sample of each type of training sample is a sample point located at the center of the region corresponding to the type of training sample. It should be noted that when the clustering algorithm is used to cluster the training sample set, the number of clusters can be set to ensure that the reference sample of the center of the region corresponding to each type of training sample obtained is representative.

[0076] A2, obtain the features and feature values of the reference samples in the training sample set, and obtain the weight values of each feature in the reference samples.

[0077] For example, the computer device can use an interpretive algorithm to obtain the weight values of each feature in the reference samples. The interpretive algorithm can be the LIME algorithm. For example, the computer device inputs the reference samples into the LIME algorithm model, and the LIME algorithm model outputs the weight values of each feature in the reference samples. The LIME algorithm model is used to: for each reference sample, first train an easily interpretable linear model using the information and a set of near-neighbor data of the information to fit the local boundary of the to-be-interpreted model, and then explain the decision basis of the to-be-interpreted model for the information based on the linear model. The weight values of the linear model reflect the importance of each dimension feature of the reference sample in the current decision. The to-be-interpreted model is a machine learning model.

[0078] In the embodiments of the present application, the reference samples are obtained by using the clustering algorithm, and the weight values of each feature in the reference samples are obtained. Without obtaining the weight values of all samples in the training sample set, the operation cost is effectively reduced. For example, the training sample set includes 10,000 training samples, and 1,000 reference samples are obtained by using the clustering algorithm, and the computing power is reduced by 10 times.

[0079] As mentioned earlier, machine learning models can be of various types. Some machine learning models predict outcomes based on classification principles. Tree models are a common type of classification-based machine learning model. A tree model consists of multiple related nodes. For ease of understanding, this application provides a brief introduction to tree models in its embodiments. In a tree model, each node includes a node element and several branches pointing to subtrees. The subtree to the left of a node is called the left subtree, and the subtree to the right of a node is called the right subtree. The root of a subtree of a node is called the child node of that node. If a node is a child of another node, then the other node is the parent node of that node. The depth or level of a node refers to the number of edges of the longest simple path from the root node to that node. For example, if the depth (height or level) of the root node is 1, the depth of the root's child nodes is 2, and so on. Leaf nodes, also called terminal nodes, are nodes with a degree of 0. The degree of a node refers to the number of its subtrees. Non-leaf nodes are nodes other than leaf nodes, including the root node and the nodes between the root and leaf nodes. A binary tree is a tree structure in which each node has at most two subtrees, and it is a relatively common tree structure. In this embodiment, the machine learning model can be an ensemble model composed of binary tree models, such as the isolated forest model.

[0080] like Figure 3 As shown, Figure 3 This application provides an illustrative tree structure, comprising nodes P1 to P5, where P1 is the root node, P3, P4, and P5 are leaf nodes, and P1 and P2 are non-leaf nodes. The tree depth is 2. This machine learning model is formed through two node splits at nodes P1 and P3. Node splitting refers to the division of a node's training sample set into at most two subsets at a split point along a certain split dimension. This can be viewed as the node splitting into at most two child nodes, each corresponding to one subset. In other words, the method of dividing a node's training sample set into child nodes is called splitting.

[0081] Figure 4 Assumption Figure 3 The training sample set corresponding to the machine learning model in the example includes feature values ​​in two dimensions (i.e., two types of features), namely dimension x and dimension y. This training sample set includes samples (a1, b1), (a1, b2), and (a2, b1). The first node split has dimension x as the splitting dimension and a3 as the splitting point, dividing the sample space containing the training sample set into two subspaces. Figure 3That is, the left subtree and the right subtree of the P1 node; the second node splitting has a splitting dimension of dimension y and a splitting point of b3, corresponding to Figure 3 That is, the left subtree and the right subtree of the P2 node. Thus, the training sample set {(a1, b1), (a1, b2), (a2, b1)} is divided into three subspaces. When the machine learning model is used, based on the input information, it is determined which subspace the feature value of the feature in the input information falls into, and based on this, the output result is determined. For example, the feature value M of the feature point included in the input information satisfies: a3≤M≤a2, and the feature value N of the dimension y satisfies: b3≤N≤b2.

[0082] Correspondingly, for the machine learning model based on the classification principle, for each feature point input (that is, the input feature and the corresponding feature value), the machine learning model can obtain the feature value range in which the feature value is located. In the embodiments of the present application, the explanatory information can also include the feature value range in which each feature value included in the explanatory information is located. In this way, the user can more clearly understand the reason for obtaining the output result based on the feature value range in which each feature value of the input information is located. The computer device can extract the feature value range in which each feature value of the input information is located from the machine learning model, or determine the feature value range in which each feature value of the input information is located through the same classification principle as the machine learning model.

[0083] It should be noted that the machine learning model is based on probability theory to predict the classification result, that is, to predict the probability of the existence of a plurality of classifications supported by the machine learning model, and to determine the classification with the highest probability as the final classification. For example, the machine learning model is a binary classification model, which supports two classifications. The machine learning model is used to predict the probability of the existence of the two classifications, and to determine the classification with the highest probability. In one optional manner, the classification result output by the machine learning model includes the classification with the highest probability (which is the final classification result) and the probability of the existence of the plurality of classifications; in another optional manner, the classification result output by the machine learning model only includes the classification with the highest probability. For example, the machine learning model based on the binary classification principle selects the classification with a larger probability (such as 0 or 1) as the classification result output.

[0084] Further optionally, the explanatory information can also include the probability of each classification supported by the machine learning model corresponding to the output result (also corresponding to the input information). It should be noted that when the explanatory information includes the probability of each classification supported by the machine learning model, the user can more clearly understand the reason for obtaining the output result. The computer device can extract the probability of each classification supported by the machine learning model from the machine learning model, or determine the probability of each classification supported by the machine learning model through the same classification principle as the machine learning model.

[0085] S103, display the generated explanatory information.

[0086] The computer device can display the explanatory information after generating one explanatory information each time. In an optional example, the computer device can simultaneously display the output result and the explanatory information corresponding to the same input information.

[0087] It should be noted that when the same information is included in the explanatory information and the output result, the two can be displayed alternatively, without the need for repeated display of the same information. For example, when the probability of each classification supported by the machine learning model is included in both the explanatory information and the output result, only the probability of each classification supported by the machine learning model in the output result can be displayed.

[0088] Figure 5 and Figure 6 are respectively schematic diagrams of display interfaces of two kinds of explanatory information provided by the embodiments of the present application. Figure 5 and Figure 6 In, the output result and the explanatory information are simultaneously displayed as an example, but the display timing of the explanatory information is not limited. For example, the output result can be displayed first, and then the explanatory information can be displayed. Figure 5 and Figure 6 As shown in and, the output result includes two types of compliance and violation, and the probability of each type. Figure 5 The output result in further includes: the final classification: violation. Figure 6 The output result in further includes: the final classification: compliance. Figure 5 and Figure 6 In and, the explanatory features respectively include: the feature, the feature value, the weight value of the feature belonging to the classification of “compliance” and the weight value of the feature belonging to the classification of “violation”, and the feature value range in which each feature value is located. For example, in, Figure 5 In, the feature value of the feature “department reimbursement document mean” is 2.46, which belongs to the violation classification, and its weight value is 0.14. The feature value range in which the feature value is located is “> 2.36”. For another example, Figure 6 In, the feature value of the feature “current reimbursement document process role number” is 0.01, which belongs to the compliance classification, and its weight value is 0.01. The feature value range in which the feature value is located is “< 3”.

[0089] The machine learning model usually comprises a multi-level logical structure, and the working manner of the model is difficult for a user to understand. The traditional machine learning model also lacks a way to interact with the user, so that the user cannot effectively interact with the model, cannot build trust in the model, and is not conducive to the promotion and application of the model. In the embodiments of the present application, a way for the model to interact with the user is provided by displaying explanatory information, which can effectively help the user open the black box of the model and understand the working manner of the model. Thus, the user can effectively interact with the model, build trust in the model, and be conducive to the promotion and application of the model.

[0090] Moreover, the traditional explanatory algorithm usually has a large amount of calculation, and if the explanatory information is obtained by the explanatory algorithm when the user uses the machine learning model, a large amount of calculation is required, and the user needs to wait for a long time, so that real-time interaction between the model and the user cannot be realized.

[0091] In the embodiments of the present application, the weight of the feature of all or part of the samples in the training sample set is obtained in advance, so that when the user uses the machine learning model, the explanatory information can be determined by the weight obtained in advance without real-time calculation, the waiting time of the user is effectively reduced, and real-time interaction between the model and the user is realized.

[0092] It should be noted that when the output result and the explanatory information are displayed at the same time, the user can determine whether the output result is correct while determining whether the explanatory information is abnormal, so that the two actions can be performed in the same user interface, and the workload of the user is avoided. Correspondingly, the computer device receives output result determination information, and the output result determination information is used to indicate whether the output result is correct, for example, whether the output classification is correct, whether the proportion of each classification of the output is correct, and the like. Based on the output result determination information, the computer device can adjust the training sample set according to a preset rule, for example, adjust the label in the training sample set.

[0093] S104, obtaining abnormal explanatory information, the abnormal explanatory information being explanatory information with abnormal explanation content.

[0094] In an example, the process of obtaining the abnormal explanatory information comprises the following steps:

[0095] B1, receiving determination information of the explanatory information, the determination information being used to indicate whether the explanation content of the explanatory information is abnormal.

[0096] In the embodiments of the present application, there are various ways to obtain the determination information. The embodiments of the present application are described by taking the following several ways as examples:

[0097] In a first optional example, the computer device can display an indication button of the explanatory information, and the user triggers the generation of the determination information based on the indication button. For example, the determination information generated by performing a target operation once on the indication button indicates that the interpretation content of the explanatory information is normal; the determination information generated by performing the target operation twice in succession or not performing the target operation on the indication button indicates that the interpretation content of the explanatory information is abnormal. Examples of the target operation are single-click, double-click, drag, or swipe operation.

[0098] In a second optional example, the computer device can display an input box corresponding to the explanatory information, and the user inputs the determination information in the input box, and correspondingly, the computer device receives the determination information through the input box. For example, the input content is "abnormal explanatory information" or "abnormal feature F1".

[0099] In a third optional example, as shown in Figure 5 and Figure 6 , the explanatory information includes one or more features and feature values of the features. The determination information includes sub-determination information corresponding to the one or more features one by one, and each sub-determination information is used to indicate whether the corresponding feature is abnormal.

[0100] Figure 7 is a display interface schematic diagram of the explanatory information provided by an embodiment of the present application. Each feature corresponds to a determination button 20, and the sub-determination information generated by performing a target operation once on the determination button 20 is used to indicate that the corresponding feature is normal; the sub-determination information generated by performing the target operation twice in succession or not performing the target operation on the determination button 20 indicates that the corresponding feature is abnormal.

[0101] B2, the explanatory information whose interpretation content is indicated by the determination information as abnormal is determined as abnormal explanatory information.

[0102] Corresponding to the first optional example described above, after receiving the determination information indicating that the interpretation content of the explanatory information is abnormal, the computer device determines the corresponding explanatory information as abnormal explanatory information. After receiving the determination information indicating that the interpretation content of the explanatory information is normal, the computer device determines the corresponding explanatory information as normal explanatory information. For example, the computer device detects that the user performs a target operation once on the indication button, and then determines that the corresponding explanatory information is abnormal explanatory information; the computer device detects that the user performs the target operation twice in succession or does not perform the target operation on the indication button, and then determines that the corresponding explanatory information is normal explanatory information.

[0103] Corresponding to the second optional example described above, the computer device determines the corresponding explanatory information as abnormal explanatory information if the determination information indicates that the explanatory information is abnormal or any feature in the explanatory information is abnormal. If the determination information indicates that the explanatory information is normal or all features in the explanatory information are normal, the corresponding explanatory information is determined as normal explanatory information. For example, if the input content in the input box corresponding to the explanatory information is "abnormal explanatory information" or "feature F1 is abnormal", the corresponding explanatory information is determined as abnormal explanatory information. If the input content in the input box corresponding to the explanatory information is "normal explanatory information" or "all features are normal", the corresponding explanatory information is determined as normal explanatory information.

[0104] Corresponding to the third optional example described above, the explanatory information is determined as abnormal explanatory information when the determination information indicates that any feature in the explanatory information is an abnormal feature. For example, if a certain sub-determination information included in the determination information indicates that the corresponding feature is abnormal, the determination information indicates that the certain feature in the explanatory information is an abnormal feature, and accordingly, the explanatory information is abnormal explanatory information. The explanatory information is determined as normal explanatory information when the determination information indicates that all features in the explanatory information are normal features. For example, if all sub-determination information included in the determination information indicates that the corresponding feature is normal, the determination information indicates that all features in the explanatory information are normal features, and accordingly, the explanatory information is determined as normal explanatory information. For example, the computer device can collect user feedback information for the explanatory information. For example, the model predicts that the output result of a certain input information is A, and the explanatory information includes: feature F1 in the input information is x1, and feature F2 is x2. The user feedback information feeds back whether the explanatory information of F1=x1 and F2=x2 is reasonable.

[0105] It should be noted that, as shown in Figure 7 If each feature corresponds to a determination button, the computer device can determine the feature corresponding to the determination button not operated by the user as a normal feature. In this way, for the normal feature, the user does not need to operate the determination button, reducing the number of user operations and improving user experience.

[0106] S105, based on the abnormal explanatory information, adjusting the training sample set of the machine learning model.

[0107] When the abnormal explanatory information appears, it indicates that the existing machine learning model may have been unable to meet the user's needs as the environment or rules change, and the machine learning model needs to be updated. In the embodiment of the application, the training sample set of the machine learning model is adjusted, and the machine learning model is retrained based on the adjusted training sample set, so as to realize the autonomous update of the machine learning model when the abnormal explanatory information appears. The accuracy and timeliness of the update time of the machine learning model are ensured. The machine learning model is adapted to the new environment or rules.

[0108] In the embodiment of the application, there are various ways to adjust the training sample set of the machine learning model based on the abnormal explanatory information, which will be described below by way of example of the following optional ways:

[0109] In the first optional way, the computer device adjusts the training sample set of the machine learning model based on the abnormal features in the abnormal explanatory information. The process includes: updating the labels corresponding to the target training samples in the training sample set to obtain the adjusted training sample set, wherein the target training samples include target features same as the abnormal features in the abnormal explanatory information, and the feature values of the target features are same as the abnormal feature values of the abnormal features.

[0110] In the first optional implementation, the computer device updates the labels corresponding to all target training samples in the training sample set to obtain the adjusted training sample set, wherein the target training samples include target features same as the abnormal features in the abnormal explanatory information, and the feature values of the target features are same as the abnormal feature values of the abnormal features. For example, according to the foregoing user feedback information or determination information, the abnormal features and their abnormal feature values are known, and then the computer device can establish a set of abnormal features and abnormal feature values. When a certain feature F of any training sample K is same as a certain feature G in the set, and the feature value of the certain feature F of any training sample K is same as the abnormal feature value of the certain feature G in the set, the any training sample K is a target training sample.

[0111] For example, the abnormal features are F1 and F3, and their feature values are x1 and x3, respectively. The computer device can establish a set of abnormal features and abnormal feature values {(F1=x1), (F3=x3),...}. If a certain training sample includes F1=x1 or F3=x3, the certain training sample is a target training sample.

[0112] In a second optional implementation, the computer device updates the labels corresponding to the target training samples in the training sample set step by step to obtain an adjusted training sample set, wherein the target training sample includes a target feature same as the abnormal feature in the abnormal explanatory information, and the feature value of the target feature is same as the abnormal feature value of the abnormal feature, and the determination process of the target training sample refers to the first optional implementation. The process of updating the labels corresponding to the target training samples in the training sample set step by step to obtain the adjusted training sample set includes:

[0113] C1, obtaining an initial x value, 1≤x

[0114] The retraining process includes:

[0115] C11, updating the labels corresponding to the x target training samples in the training sample set to opposite labels.

[0116] In an example, the opposite label can be obtained by performing an inverse operation on the label. For example, the label is "Zhang San", and the opposite label obtained by performing the inverse operation is "not Zhang San"; for example, the label is "0", and the opposite label obtained by performing the inverse operation is "1".

[0117] In an optional example, the machine learning model is a binary classification model, and the training sample set of the machine learning model has two labels, and the two labels are opposite labels. For example, the labels include "0" and "1"; or "normal" and "abnormal", etc. For a label, the inverse operation is to update the label to a different label.

[0118] In another optional example, the machine learning model can be converted into multiple binary classification models, and the training sample set thereof has multiple groups of labels corresponding to the multiple binary classification models one by one, each group of labels including two opposite labels, such as "Zhang San" and "not Zhang San"; or "cat" and "not cat". For a label, the inverse operation is to update the label to a different label in the same group as the label.

[0119] C12, training the machine learning model based on the training sample set after the label update to obtain a retrained machine learning model.

[0120] Based on the updated training sample set with labels, training the machine learning model is a process of fitting the machine learning model on the training sample set. The fitting process can include both feature selection and parameter estimation. For example, for a supervised learning algorithm, the training sample set is used to fit parameters (e.g., weights of links between neurons in an artificial neural network).

[0121] During the training process, the machine learning model makes predictions on samples in the updated training sample set with labels, and compares the predictions with the labels. Based on the comparison, the computer device updates the model parameters of the machine learning model. The model parameters of the machine learning model can be updated by a validation dataset, which provides an unbiased evaluation of the machine learning model fitted on the training sample set. The computer device inputs samples in the validation dataset into the machine learning model to obtain validation results. The computer device repeats the sample prediction and parameter update process until the error of the validation results of the machine learning model after updating the parameters increases (i.e., the error increases compared with the previous validation results, which indicates that the machine learning model has overfitting on the training sample set), and stops training the machine learning model. The machine learning model obtained by the previous training is used as the retrained machine learning model. Since the machine learning model obtained by the current training has an increased error of the validation results compared with the machine learning model obtained by the previous training, which indicates overfitting, the machine learning model obtained by the previous training is more reliable than the machine learning model obtained by the current training, and thus the machine learning model obtained by the previous training is used as the retrained machine learning model.

[0122] C13, increasing x to obtain an updated x, and performing the retraining process again.

[0123] There are various ways to increase x. For example, x can be increased in an arithmetic sequence, i.e., the difference between the updated x and the original x is a fixed value. The difference can be calculated based on the total number of target training samples y and the initial x value. For example, assuming the initial x value is x0, the difference is (y-x0) / m, where m can be a preset value or a value determined according to a preset rule (e.g., m is an integer, and m is the largest integer that can be divided by (y-x0)).

[0124] For example, x can also be increased in a geometric progression, i.e., the ratio of the updated x to the original x is a fixed ratio, which can be calculated based on the total number of target training samples y and the initial x value. For example, assuming the initial x value is x0, the ratio q satisfies: y = x0 * q n ; where n is the number of times x is updated, i.e., the number of times the retraining process is performed. q or n can be a preset value or a value determined according to a preset rule (for example, n is an integer). For example, the initial x value is 10, y is 1000, and n = 2, so the ratio is:

[0125] By increasing x in a geometric progression or an arithmetic progression, x can be increased at a stable step, so that the target training samples are updated stably and uniformly.

[0126] Optionally, the step of increasing x is positively correlated with the size of the total number of target training samples y. That is, the larger the total number of target training samples y, the larger the step; the smaller the total number of target training samples y, the smaller the step. In this way, the size of the step can be flexibly adjusted according to the total number of target training samples, so that the training sample set can be flexibly adjusted.

[0127] In actual implementation, the relationship between y and the initial x value may not satisfy the requirement of a geometric progression or an arithmetic progression, so the way of increasing x can be set to be approximately a geometric progression or an arithmetic progression, for example, the way of increasing x in the first few times is a geometric progression or an arithmetic progression, and x is updated to y in the last time. There are other ways of actually increasing x, and the embodiments of the present application are only illustrative and are not limited in this regard.

[0128] For example, the foregoing preset stopping condition can be various. In one example, the preset stopping condition includes: the value of the updated x is greater than the total number of target training samples y; in another example, the preset stopping condition includes: the number of times the retraining process is performed is equal to a set number of times.

[0129] C2, determining the training sample set corresponding to the machine learning model with the optimal validation effect among the machine learning models obtained through the at least two retraining processes as the adjusted training sample set.

[0130] After the at least two machine learning models are obtained through the at least two retraining processes, the retrained machine learning models can be verified through a validation dataset. During the verification, the at least two machine learning models make predictions on the same samples in the validation dataset. According to the prediction results, the computer device selects a machine learning model with the best verification effect from the at least two machine learning models, and determines the training sample set corresponding to the selected machine learning model as the adjusted training sample set. For example, the computer device determines the verification effects of the at least two machine learning models based on the prediction results corresponding to the at least two machine learning models by using a metric function. For example, the metric function can be a loss function or a cost function. For example, when the metric function is a loss function, the loss value is calculated by using the loss function based on the prediction results corresponding to the at least two machine learning models, and the machine learning model with the smallest loss value is determined as the machine learning model with the best verification effect.

[0131] Compared with the first optional implementation manner, the second optional implementation manner avoids overcorrection of the label update by gradually updating the label multiple times, and determines the training sample set corresponding to the machine learning model with the best verification effect as the adjusted training sample set, which can improve the prediction accuracy of the finally obtained machine learning model.

[0132] In the second optional manner, the computer device adjusts the training sample set of the machine learning model based on the anomaly explanatory information. The process includes: displaying the anomaly explanatory information, receiving an abnormal feature selected from the displayed anomaly explanatory information; and updating the label corresponding to a target training sample in the training sample set to obtain an adjusted training sample set, wherein the target training sample includes a target feature same as the abnormal feature in the anomaly explanatory information, and the feature value of the target feature is same as the abnormal feature value of the abnormal feature.

[0133] In the optional mode, the computer device can display the obtained one or more pieces of abnormal explanatory information, and the user can select the abnormal features. According to the foregoing A2, the user can determine the whole of one piece of explanatory information as the abnormal explanatory information, or determine the abnormal explanatory information by pointing out an abnormal feature in the one piece of explanatory information. In the embodiment of the present application, it is generally required to determine the accurate abnormal features, so as to adjust the training sample set based on the abnormal features. If the abnormal features in the explanatory information are not determined, the determination of the abnormal features can be realized by the user selecting the abnormal features. If the abnormal features in the explanatory information are determined, the secondary confirmation can be realized by the user selecting the abnormal features, so as to filter out the previously selected incorrect abnormal features, confirm the previously selected correct abnormal features, and ensure the accuracy of the finally obtained abnormal features. After the abnormal features are obtained, the label corresponding to the target training sample in the training sample set is updated to obtain the adjusted training sample set. The process can be referred to the foregoing first optional mode, and the embodiment of the present application will not be described herein.

[0134] In the foregoing first and second optional modes, the computer device can collect multiple pairs of abnormal features and abnormal feature values. In an optional example, the computer device can respectively use each pair of abnormal features and abnormal feature values to screen the target training sample in the training sample set. In another optional example, the computer device can simultaneously use each pair of abnormal features and abnormal feature values to screen the target training sample in the training sample set. In yet another optional example, the computer device can select one or more pairs of abnormal features and abnormal feature values from the multiple pairs of abnormal features and abnormal feature values, and screen the target training sample in the training sample set based on the selected one or more pairs of abnormal features and abnormal feature values.

[0135] In the related art, the new training sample is generally collected by an expert or a designated device, or the original training sample is adjusted one by one to serve as the adjusted training sample set. Thus, the workload is large, and the acquisition cost of the training sample set is high.

[0136] The embodiment of the present application updates the label corresponding to the target training sample in the training sample set to obtain the adjusted training sample set. The adjustment mode of the training sample set is simple and fast, the workload is small, and the acquisition cost of the training sample set can be effectively reduced.

[0137] S106, obtaining an updated machine learning model, which is trained based on the adjusted training sample set.

[0138] In an optional implementation mode, the computer device can train the machine learning model based on the obtained adjusted training sample to obtain the updated machine learning model.

[0139] In another optional implementation, C1 and C2 are combined, and at least two machine learning models have been obtained in the retraining process, wherein the machine learning model with the optimal performance is obtained by training the adjusted training sample set, and thus the machine learning model corresponding to the adjusted training sample set is directly obtained from the at least two machine learning models to obtain the updated machine learning model.

[0140] It should be noted that the foregoing embodiments are described by taking LIME as an example of an explanatory algorithm, and other explanatory algorithms can also be used in actual implementation of the embodiments of the present application. For example, a SHapley Additive exPlanations (SHAP) model explanation method.

[0141] In summary, the model updating method provided in the embodiments of the present application adjusts the training sample set of the machine learning model based on the obtained abnormal explanatory information, and then obtains an updated machine learning model using the training sample set. In this way, automatic updating of the model based on abnormal explanatory information is achieved, the flexibility of model updating is effectively improved, and the reliability of the updated machine learning model is improved.

[0142] In related technologies, if the updating of the machine learning model is triggered by human experience, the timing of triggering is not appropriate, which leads to low prediction accuracy of the updated machine learning model. If the machine learning model is periodically updated, the model cannot be updated in time when the model deteriorates, which affects the user experience.

[0143] In the embodiments of the present application, the training sample set is updated after the abnormal explanatory information is obtained, and then the model is updated. The abnormal explanatory information can reflect the deterioration of the model to some extent, and the model can be updated (e.g., the model parameters are adjusted) at an appropriate time (i.e., when the model deteriorates) without human triggering. The model is self-adaptively optimized, the timing of updating the machine learning model is more accurate, and the prediction accuracy of the updated machine learning model is improved. The machine learning model is updated after the explanatory information is abnormal, and the abnormal explanatory information can reflect the deterioration of the model to some extent. In this way, the model can be updated in time when the model deteriorates, the timeliness of model updating is ensured, and the user experience is improved.

[0144] In related technologies, the updating of the machine learning model is usually performed manually, for example, the labels or training samples are manually adjusted. In this way, the labor cost is high, and the training efficiency is low.

[0145] In the embodiment of the present application, the updating of the machine learning model based on the abnormal explanatory information does not require manual model updating, thereby effectively reducing the labor cost and improving the training efficiency.

[0146] In the related art, the labels of the training sample set are determined based on expert experience. Changes in the expert experience result in changes in the label setting standard of the machine learning model, which sometimes cannot be perceived by the experts, but has a great influence on the prediction effect when the machine learning model is actually applied. The embodiment of the present application can update the model in time based on the abnormal explanatory information, and the influence of the changes in the expert experience on the actual application of the machine learning model can be effectively reduced through automatic verification by the user.

[0147] To facilitate the understanding of the reader, the model updating method involved in the embodiment of the present application is explained below in combination with the actual structure of the computer device. Figure 8 FIG. 1 is a schematic diagram of an application environment of a processing system involved in a model updating method provided by an embodiment of the present application. The processing system includes a computer device 20 and a storage device 30.

[0148] The computer device 20 can be a personal computer, a server, or a cloud computing center, etc. The computer device 20 includes a machine learning model 201, a model explanatory module (also referred to as an interpreter) 202, and a model adjustment module 203. The machine learning model 201 is used to receive input information and make a prediction based on the input information to obtain an output result (such as the classification result described above), for example, whether there is a risk or a problem. The model explanatory module 202 is used to generate explanatory information based on the output result (the process is referred to as S102 described above). The explanatory information can be of various types, for example, assuming that the content of the explanatory information is: the reimbursement amount of the reimbursement form is greater than the average amount of the same type of reimbursement form, therefore, the reimbursement form is determined as a violation of the reimbursement form. The content of the explanatory information can be represented in the form of graphics, text, and / or tables, etc. The model adjustment module 203 is used to adjust the training sample set of the machine learning model based on the abnormal explanatory information (the process is referred to as S103 described above). For example, Figure 8 The scenario in which the computer device displays the output result and the explanatory information at the same time is shown in FIG. 2. In this way, the determination information of whether the explanatory information is abnormal and the determination information of whether the output result is correct can be obtained.

[0149] The storage device 30 is used to store data, for example, to store the training sample set and / or the data to be predicted when the machine learning model is actually used. For example, the storage device 30 can be a database management system (DBMS).

[0150] In the embodiments of the present application, the model explanation module 202 is configured to provide an explanation for the output result of the machine learning model. The model explanation module 202 can be a model explanation model, and the process of constructing the model explanation model also includes two steps of training and prediction. When training, the working process of the model explanation model can include the following steps:

[0151] D1, an initial model of the model explanation model is constructed, and the initial model is trained by using a training sample set. The training sample set can be the same as the training sample set of the machine learning model.

[0152] D2, reference samples are determined by using a clustering algorithm, and features and feature values of the reference samples in the training sample set are obtained, and weights of the features in the reference samples are obtained. The process can refer to A1 and A2. In this way, the complex operation is moved from the application link of the machine learning model to the training link, and the representative nodes of the training sample set are found by using the clustering algorithm, so that the nearest neighbor reference sample is found in the subsequent application of the model explanation model, and the explanation information of the output result is quickly given, which can effectively save the time delay of the application link and improve the user experience. For example, the running time of the LIME algorithm for calculating the explanation information corresponding to the input information of a machine learning model is 5 minutes (min), and the running time of the model explanation model provided by the embodiments of the present application for calculating the explanation information corresponding to the input information is less than 1 second (s). In this way, the user waiting time is greatly reduced.

[0153] The foregoing step D2 is only used as an example to illustrate the training process of obtaining the features and feature values of the reference samples in the training sample set by using the model explanation model, and obtaining the weights of the features in the reference samples. In actual implementation, step D2 can not be performed.

[0154] Figure 9 is Figure 8 a schematic diagram of an implementation scenario of the processing system. Figure 9 Suppose the machine learning model 201 is a single document scoring model 201a, the input information of which is reimbursement single data, the reimbursement single data including one or more features and corresponding feature values; the output result of the single document scoring model 201a is used to indicate whether the reimbursement single is abnormal, and the output result includes: “normal reimbursement single” or “abnormal reimbursement single”. The corresponding model updating method can include:

[0155] E1, in the training phase, a single document scoring model is trained by using a training sample set. The single document scoring model can be an XGBoost model. Each training sample in the training sample set can include: reimbursement single data and a label thereof.

[0156] E2, in the verification phase or use phase, the actual report of the bill is predicted by using the bill scoring model 201a to obtain a prediction result of a normal reimbursement bill or a prediction result of an abnormal reimbursement bill.

[0157] E3, the output result of the bill scoring model 201a is predicted by using the model explanation module 202 to obtain an explanatory result of the reimbursement bill data.

[0158] E4, the computer device 20 obtains the determination information of the explanatory information input by the user (such as a business expert) whether it is abnormal, and the determination information of whether the output result is correct. For the same reimbursement bill data, the user can input the two kinds of determination information in the same user interface. This does not increase much of the user's workload, on the contrary, the explanatory information can assist the user to judge whether the output result is correct.

[0159] E5, the model adjustment module 203 is used to adjust the training sample set of the machine learning model based on the abnormal explanatory information (the process is referred to as S103).

[0160] Figure 9 Only the application scenario of the model updating method provided by the embodiment of the application is schematically described, and the machine learning model involved in the model updating method is not limited. For example, the machine learning model can also be a face recognition model, and the input information is a face image including one or more face features and corresponding feature values; for another example, the machine learning model can also be a network anomaly recognition model, and the input information includes one or more network parameters (i.e. the feature is a network parameter) and corresponding parameter values.

[0161] In actual implementation, Figure 8 And Figure 9 The model explanation module 202 and the model adjustment module 203 in the above embodiment can be respectively deployed in different computer devices, and the storage device 30 can be integrated in the computer device.

[0162] The steps of the model updating method provided by the embodiment of the application can be adjusted appropriately, and the steps can be increased or decreased according to the situation. Any person skilled in the art can easily think of the method within the technical range disclosed by the application, which should be covered by the protection scope of the application, therefore, it is not described again.

[0163] Figure 10 is a block diagram of a model updating device 40 provided by the embodiment of the application, the device 40 comprises:

[0164] The first acquisition module 401 is used to acquire abnormal explanatory information, the abnormal explanatory information is explanatory information with abnormal explanation content, and the explanatory information is information used to explain the output result of the machine learning model.

[0165] The adjusting module 402 is configured to adjust the training sample set of the machine learning model based on the abnormal explanatory information.

[0166] The second obtaining module 403 is configured to obtain an updated machine learning model, which is trained based on the adjusted training sample set.

[0167] The model updating apparatus provided by the embodiments of the present application adjusts the training sample set of the machine learning model based on the obtained abnormal explanatory information after the first obtaining module obtains the abnormal explanatory information, so that the second obtaining module obtains the updated machine learning model by using the training sample set. In this way, the automatic updating of the model based on the abnormal explanatory information is realized, the flexibility of the model updating is effectively improved, and the reliability of the updated machine learning model is improved.

[0168] In an optional implementation, the explanatory information includes one or more features and feature values of the features, and the adjusting module is configured to: update labels corresponding to target training samples in the training sample set to obtain the adjusted training sample set, wherein the target training samples include target features same as the abnormal features in the abnormal explanatory information, and the feature values of the target features are same as the abnormal feature values of the abnormal features.

[0169] In an optional implementation, the adjusting module 402 is configured to: obtain an initial x value, 1≤x<y, and perform at least two retraining processes based on the initial x value until a preset stop condition is reached, the y being the total number of target training samples in the training sample set; and determine the training sample set corresponding to a machine learning model with the optimal validation effect in the machine learning models obtained through the at least two retraining processes as the adjusted training sample set.

[0170] The retraining process includes: updating labels corresponding to x target training samples in the training sample set to opposite labels; training the machine learning model based on the training sample set after the label updating to obtain a retrained machine learning model; increasing the x to obtain an updated x, and performing the retraining process again.

[0171] Figure 11 is a block diagram of another model updating apparatus 40 provided by the embodiments of the present application, and the apparatus 40 further includes:

[0172] The generating module 404 is configured to generate explanatory information based on an output result of the machine learning model before the abnormal explanatory information is obtained; and the display module 405 is configured to display the generated explanatory information.

[0173] In an example, the first obtaining module 401 is configured to: receive determination information of the interpretative information, the determination information being used to indicate whether the interpretation content of the interpretative information is abnormal; and determine the interpretative information indicated by the determination information as abnormal interpretative information.

[0174] Optionally, the output result is generated based on input information of the machine learning model. Figure 12 is a block diagram of a generating module 404 provided by an embodiment of the present application. The generating module 404 comprises: an obtaining sub-module 4041 configured to obtain input information corresponding to the output result; a determining sub-module 4042 configured to determine, in a training sample set, a target sample having a distance less than a preset distance threshold or being closest to the input information; and a generating sub-module 4043 configured to generate interpretative information, the interpretative information comprising features and feature values included in the target sample, and weights of each feature in the target sample obtained in advance.

[0175] Figure 13 is a block diagram of another model updating device 40 provided by an embodiment of the present application. The device 40 further comprises:

[0176] a clustering module 406 configured to divide the training sample set into multiple classes of training samples by using a clustering algorithm before obtaining the abnormal interpretative information, and determine a reference sample in each class of training samples; and a third obtaining module 407 configured to obtain features and feature values of the reference samples in the training sample set, and obtain weights of each feature in the reference samples.

[0177] Correspondingly, the determining sub-module 4052 is configured to determine, in the reference samples included in the training sample set, a target sample having a distance less than a preset distance threshold or being closest to the input information.

[0178] Optionally, the third obtaining module 407 is configured to: obtain the weights of each feature in the reference samples by using a local interpretative model algorithm LIME.

[0179] In an optional example, the interpretative information comprises one or more features and feature values of the features, and the first obtaining module 401 is configured to: when the determination information indicates that any feature in the interpretative information is an abnormal feature, determine the interpretative information as abnormal interpretative information.

[0180] In an example, the interpretative information comprises a feature value range in which each feature value is located and / or a probability of each classification supported by the machine learning model corresponding to the output result.

[0181] Figure 14A structural diagram of a computer device 500 involved in the image processing method provided by the embodiments of the present application is shown. The computer device 500 can be, but is not limited to, a laptop computer, a desktop computer, a mobile phone, a smartphone, a tablet computer, a multimedia player, an e-book reader, a smart in-vehicle device, a smart home appliance (such as a smart television), an artificial intelligence device, a wearable device, an Internet of Things device, or a virtual reality / augmented reality / mixed reality device, etc. For example, the computer device 500 can include the aforementioned Figure 1 The structure of the model updating apparatus shown.

[0182] The computer device 500 can include a processor 510, an external memory interface 520, an internal memory 521, a universal serial bus (USB) interface 530, a charge management module 540, a power management module 541, a battery 542, an antenna 4, an antenna 2, a mobile communication module 550, a wireless communication module 560, an audio module 570, a loudspeaker 570A, a receiver 570B, a microphone 570C, a headset interface 570D, a sensor module 580, a key 590, a motor 591, an indicator 592, a camera 593, a display screen 594, and a subscriber identification module (SIM) card interface 595, etc. The sensor module 580 can include one or more of a pressure sensor 580A, a gyroscope sensor 580B, a barometric pressure sensor 580C, a magnetic sensor 580D, an acceleration sensor 580E, a distance sensor 580F, a proximity light sensor 580G, a fingerprint sensor 580H, a temperature sensor 580J, a touch sensor 580K, an ambient light sensor 580L, a bone conduction sensor 580M, etc.

[0183] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the computer device 500. In other embodiments of the present application, the computer device 500 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0184] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation on the computer device 500. In other embodiments of the present application, the computer device 500 can also use different interface connection modes (such as bus connection mode) in the above embodiments, or a combination of multiple interface connection modes.

[0185] The processor 510 can include one or more processing units, such as a central processing unit (CPU) (e.g., an application processor (AP)), a graphics processing unit (GPU), a modem processor, an image signal processor (ISP), a neural-network processing unit (NPU), a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent components or integrated in one or more processors.

[0186] The processor 510 can further include a memory for storing instructions and data. In some embodiments, the memory in the processor 510 is a cache memory. The memory can hold instructions or data that the processor 510 has just used or is using repeatedly. If the processor 510 needs to use the instructions or data again, it can directly call them from the memory. This avoids repeated access and reduces the waiting time of the processor 510, thus improving the efficiency of the system.

[0187] In some embodiments, the processor 510 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0188] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 510 can contain multiple sets of I2C bus. The processor 510 can be coupled to the touch sensor 580K, the charger, the flash, the camera 593, etc. through different I2C bus interfaces respectively. For example, the processor 510 can be coupled to the touch sensor 580K through an I2C interface, so that the processor 510 and the touch sensor 580K communicate through the I2C bus interface, and the touch function of the computer device 500 is realized.

[0189] The I2S interface can be used for audio communication. In some embodiments, the processor 510 can contain multiple sets of I2S bus. The processor 510 can be coupled to the audio module 570 through the I2S bus, and communication between the processor 510 and the audio module 570 is realized. In some embodiments, the audio module 570 can deliver audio signals to the wireless communication module 560 through the I2S interface, and the function of answering a phone through a Bluetooth headset is realized.

[0190] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 570 and the wireless communication module 560 can be coupled through the PCM bus interface. In some embodiments, the audio module 570 can also deliver audio signals to the wireless communication module 560 through the PCM interface, and the function of answering a phone through a Bluetooth headset is realized. The I2S interface and the PCM interface can both be used for audio communication.

[0191] The UART interface is a universal serial data bus, which is used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is usually used to connect the processor 510 and the wireless communication module 560. For example, the processor 510 communicates with the Bluetooth module in the wireless communication module 560 through the UART interface, and the Bluetooth function is realized. In some embodiments, the audio module 570 can deliver audio signals to the wireless communication module 560 through the UART interface, and the function of playing music through a Bluetooth headset is realized.

[0192] The MIPI interface can be used to connect the processor 510 and the display screen 594, the camera 593 and other peripheral devices. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), and the like. In some embodiments, the processor 510 and the camera 593 communicate through the CSI interface to implement the shooting function of the computer device 500. The processor 510 and the display screen 594 communicate through the DSI interface to implement the display function of the computer device 500.

[0193] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 510 and the camera 593, the display screen 594, the wireless communication module 560, the audio module 570, the sensor module 580, and the like. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, and the like.

[0194] The USB interface 530 is an interface that meets the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface 530 can be used to connect a charger to charge the computer device 500, and can also be used to transmit data between the computer device 500 and peripheral devices. It can also be used to connect earphones to play audio through the earphones. The interface can also be used to connect other computer devices, such as AR devices and the like.

[0195] The charging management module 540 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 540 can receive charging input from a wired charger through the USB interface 530. In some wireless charging embodiments, the charging management module 540 can receive wireless charging input through the wireless charging coil of the computer device 500. The charging management module 540 can charge the battery 542 while also providing power to the computer device through the power management module 541.

[0196] The power management module 541 is configured to connect the battery 542 and the charging management module 540 to the processor 510. The power management module 541 receives input from the battery 542 and / or the charging management module 540 to power the processor 510, the internal memory 521, the display screen 594, the camera 593, the wireless communication module 560, and the like. The power management module 541 can also be configured to monitor parameters such as the battery capacity, the number of battery cycles, the state of health of the battery (leakage, impedance), and the like. In some embodiments, the power management module 541 can also be disposed in the processor 510. In some embodiments, the power management module 541 and the charging management module 540 can also be disposed in the same device.

[0197] Optionally, the wireless communication function of the computer device 500 can be implemented by the antenna 4, the antenna 2, the mobile communication module 550, the wireless communication module 560, the modem processor, and the baseband processor, and the like.

[0198] The antenna 4 and the antenna 2 are configured to transmit and receive electromagnetic wave signals. Each antenna in the computer device 500 can be configured to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example, the antenna 4 can be multiplexed as a diversity antenna for a wireless local area network. In some embodiments, the antennas can be used in combination with a tuning switch.

[0199] The mobile communication module 550 can provide a solution for wireless communication including 2G / 3G / 4G / 5G and the like applied to the computer device 500. The mobile communication module 550 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), and the like. The mobile communication module 550 can receive electromagnetic waves from the antenna 4, filter, amplify, and the like the received electromagnetic waves, and transmit the processed signals to the modem processor for demodulation. The mobile communication module 550 can also amplify signals modulated by the modem processor and radiate the signals as electromagnetic waves through the antenna 4. In some embodiments, at least part of the function modules of the mobile communication module 550 can be disposed in the processor 510. In some embodiments, at least part of the function modules of the mobile communication module 550 and at least part of the modules of the processor 510 can be disposed in the same device.

[0200] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 570A, a microphone 570B, etc.), or displays an image or a video through the display screen 594. In some embodiments, the modem processor can be a separate device. In other embodiments, the modem processor can be independent of the processor 510, and can be disposed in the same device as the mobile communication module 550 or other functional modules.

[0201] The wireless communication module 560 can provide a wireless communication solution including wireless local area networks (WLAN) (such as a wireless fidelity (Wi-Fi) network), Bluetooth (BT), a global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, and the like, which are applied to the computer device 500. The wireless communication module 560 can be one or more devices that integrate at least one communication processing module. The wireless communication module 560 receives an electromagnetic wave via an antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signal, and transmits the processed signal to the processor 510. The wireless communication module 560 can also receive a signal to be transmitted from the processor 510, perform frequency modulation and amplification on the signal, and radiate the signal as an electromagnetic wave via the antenna 2.

[0202] In some embodiments, the antenna 4 and the mobile communication module 550 of the computer device 500 are coupled, and the antenna 2 and the wireless communication module 560 are coupled, so that the computer device 500 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS can include global positioning system (GPS), global navigation satellite system (GLONASS), beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).

[0203] The computer device 500 implements a display function through a GPU, a display screen 594, and an application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 594 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 510 can include one or more GPUs that execute program instructions to generate or change display information.

[0204] The display screen 594 is configured to display images, videos, and the like. The display screen 594 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diodes (QLED), or the like. In some embodiments, the computer device 500 can include 4 or N display screens 594, where N is a positive integer greater than 4.

[0205] The computer device 500 can implement a photographing function through an ISP, the camera 593, a video codec, a GPU, the display screen 594, and an application processor, and the like.

[0206] The ISP is configured to process data fed back by the camera 593. For example, when taking a photo, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing to convert it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be disposed in the camera 593.

[0207] The camera 593 is configured to capture still images or videos. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or the like format. In some embodiments, the computer device 500 can include 4 or N cameras 593, where N is a positive integer greater than 4.

[0208] The digital signal processor is used to process digital signals, in addition to being able to process digital image signals, it can also process other digital signals. For example, when the computer device 500 is in frequency point selection, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0209] The video codec is used to compress or decompress digital video. The computer device 500 can support one or more video codecs. In this way, the computer device 500 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 4, MPEG2, MPEG3, MPEG4, etc.

[0210] The NPU is a neural-network (NN) calculation processor, which can quickly process input information by drawing on the structure of a biological neural network, such as drawing on the transmission mode between human brain neurons, and can also constantly self-learn. Through the NPU, intelligent cognitive applications of the computer device 500 can be realized, such as: image recognition, face recognition, speech recognition, text understanding, etc.

[0211] The external memory interface 520 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the computer device 500. The external storage card communicates with the processor 510 through the external memory interface 520 to realize data storage functions. For example, music, video, etc. Files are saved in the external storage card.

[0212] The internal memory 521 can be used to store computer executable program codes, which include instructions. The internal memory 521 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc. The data storage area can store data created during the use of the computer device 500 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 521 can include a high-speed random access memory, such as a double data rate synchronous dynamic random access memory (DDR), and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 510 executes various function applications and data processing of the computer device 500 by running instructions stored in the internal memory 521 and / or instructions stored in a memory disposed in the processor.

[0213] The computer device 500 can realize audio functions through the audio module 570, the speaker 570A, the receiver 570B, the microphone 570C, the earphone interface 570D, and the application processor, etc. For example, music playing, recording, etc.

[0214] The audio module 570 is used to convert digital audio information into analog audio signals, and is also used to convert analog audio input into digital audio signals. The audio module 570 can also be used to encode and decode audio signals. In some embodiments, the audio module 570 can be arranged in the processor 510, or part of the functions of the audio module 570 can be arranged in the processor 510.

[0215] The speaker 570A, also known as a “loudspeaker”, is used to convert audio electrical signals into sound signals. The computer device 500 can listen to music or listen to hands-free calls through the speaker 570A.

[0216] The receiver 570B, also known as a “earpiece”, is used to convert audio electrical signals into sound signals. When the computer device 500 answers a call or a voice message, the receiver 570B can be placed close to the ear to listen to the voice.

[0217] The microphone 570C, also known as a “microphone”, “sound collector”, is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak into the microphone 570C close to the mouth to input the sound signal into the microphone 570C. The computer device 500 can be provided with at least one microphone 570C. In other embodiments, the computer device 500 can be provided with two microphones 570C, in addition to collecting sound signals, it can also realize the noise reduction function. In other embodiments, the computer device 500 can also be provided with three, four or more microphones 570C, in addition to collecting sound signals, noise reduction, it can also identify the source of the sound, realize the function of directional recording, etc.

[0218] The earphone interface 570D is used to connect a wired earphone. The earphone interface 570D can be a USB interface 530, or a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0219] The pressure sensor 580A is configured to sense a pressure signal and convert the pressure signal to an electrical signal. In some embodiments, the pressure sensor 580A can be disposed on the display 594. The pressure sensor 580A can be of various types, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, etc. The capacitive pressure sensor can include at least two parallel plates of conductive material. When a force is applied to the pressure sensor 580A, the capacitance between the electrodes changes. The computer device 500 determines the intensity of the force based on the change in capacitance. When a touch operation is applied to the display 594, the computer device 500 determines the intensity of the touch operation based on the pressure sensor 580A. The computer device 500 can also determine the location of the touch based on the detection signal of the pressure sensor 580A. In some embodiments, touch operations applied to the same touch location but with different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than a first pressure threshold is applied to a short message application icon, an instruction to view short messages is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold is applied to the short message application icon, an instruction to create a new short message is executed.

[0220] The gyroscope sensor 580B can be configured to determine the motion attitude of the computer device 500. In some embodiments, the angular velocity of the computer device 500 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 580B. The gyroscope sensor 580B can be used for anti-shake. For example, when a shutter is pressed, the gyroscope sensor 580B detects the angle of shaking of the computer device 500, calculates the distance that the lens module needs to compensate based on the angle, and lets the lens offset the shaking of the computer device 500 by reverse movement to achieve anti-shake. The gyroscope sensor 580B can also be used for navigation and motion sensing game scenarios.

[0221] The barometric pressure sensor 580C is configured to measure air pressure. In some embodiments, the computer device 500 calculates altitude, assists positioning and navigation based on the air pressure value measured by the barometric pressure sensor 580C.

[0222] The magnetic sensor 580D includes a Hall sensor. The computer device 500 can use the magnetic sensor 580D to detect the opening and closing of a flip cover. In some embodiments, when the computer device 500 is a flip phone, the computer device 500 can detect the opening and closing of the flip cover based on the magnetic sensor 580D. In turn, based on the detected opening and closing state of the cover or the flip cover, the computer device 500 can set features such as automatic unlocking of the flip cover.

[0223] The accelerometer 580E can detect the magnitude of the acceleration of the computer device 500 in various directions (typically three axes). The magnitude and direction of gravity can be detected when the computer device 500 is stationary. It can also be used to identify the computer device's orientation, for example, for switching between landscape and portrait modes, pedometer applications, etc.

[0224] The distance sensor 580F can be used to measure distance. The computer device 500 can measure distance by infrared or laser. In some embodiments, the computer device 500 can use the distance sensor 580F to measure distance to implement fast focus when taking pictures.

[0225] The proximity light sensor 580G can include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode can be an infrared light emitting diode. The computer device 500 emits infrared light outwardly through the light emitting diode. The computer device 500 detects infrared reflected light from nearby objects using the photodiode. When sufficient reflected light is detected, it can be determined that there is an object near the computer device 500. When insufficient reflected light is detected, the computer device 500 can determine that there is no object near the computer device 500. The computer device 500 can use the proximity light sensor 580G to detect when a user is holding the computer device 500 close to their ear for a phone call, so that the screen can be automatically turned off to save power. The proximity light sensor 580G can also be used for automatic unlocking and locking of the screen in a holster mode or a pocket mode.

[0226] The ambient light sensor 580L can be used to sense the brightness of ambient light. The computer device 500 can adaptively adjust the brightness of the display 594 based on the sensed brightness of ambient light. The ambient light sensor 580L can also be used to automatically adjust white balance when taking pictures. The ambient light sensor 580L can also work with the proximity light sensor 580G to detect whether the computer device 500 is in a pocket to prevent accidental touch.

[0227] The fingerprint sensor 580H can be used to capture a fingerprint. The computer device 500 can use the captured fingerprint characteristics to implement fingerprint unlocking, access application lock, fingerprint picture taking, fingerprint call answering, etc.

[0228] The temperature sensor 580J is configured to detect temperature. In some embodiments, the computer device 500 performs temperature handling strategies based on the temperature detected by the temperature sensor 580J. For example, when the temperature reported by the temperature sensor 580J exceeds a threshold, the computer device 500 reduces the performance of a processor located near the temperature sensor 580J to reduce power consumption and implement thermal protection. In some other embodiments, when the temperature is below another threshold, the computer device 500 heats the battery 542 to avoid abnormal shutdown of the computer device 500 caused by low temperature. In some other embodiments, when the temperature is below yet another threshold, the computer device 500 boosts the output voltage of the battery 542 to avoid abnormal shutdown caused by low temperature.

[0229] The touch sensor 580K is also referred to as a "touch device". The touch sensor 580K can be disposed on the display screen 594, and the touch sensor 580K and the display screen 594 form a touch screen, also referred to as a "touch panel". The touch sensor 580K is configured to detect a touch operation acting on or near the touch sensor 580K. The touch sensor 580K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 594. In some other embodiments, the touch sensor 580K can also be disposed on the surface of the computer device 500, which is different from the position where the display screen 594 is located.

[0230] The bone conduction sensor 580M can obtain a vibration signal. In some embodiments, the bone conduction sensor 580M can obtain a vibration signal of a human body sound vibration bone block. The bone conduction sensor 580M can also contact a human body pulse to receive a blood pressure pulsation signal. In some embodiments, the bone conduction sensor 580M can also be disposed in a headset to form a bone conduction headset. The audio module 570 can analyze a voice signal based on the vibration signal of the sound vibration bone block obtained by the bone conduction sensor 580M to implement a voice function. The application processor can analyze heart rate information based on the blood pressure pulsation signal obtained by the bone conduction sensor 580M to implement a heart rate detection function.

[0231] In some other embodiments of the present application, the computer device 500 can also use different interface connection modes of the above-described embodiments, for example, some or all of the above-described sensors are connected to the MCU, and the MCU is connected to the AP.

[0232] The keys 590 include a power-on key, a volume key, and the like. The keys 590 can be mechanical keys. The keys 590 can also be touch keys. The computer device 500 can receive key inputs and generate key signal inputs related to user settings and function control of the computer device 500.

[0233] The motor 591 can generate a vibration prompt. The motor 591 can be used for incoming call vibration prompt, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. The touch operation acting on different regions of the display screen 594 can also correspond to different vibration feedback effects of the motor 591. Different application scenarios (such as time reminders, received messages, alarms, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0234] The indicator 592 can be an indicator light, which can be used to indicate the charging state, the power change, and can also be used to indicate messages, missed calls, notifications, etc.

[0235] The SIM card interface 595 is used to connect the SIM card. The SIM card can be inserted into or pulled out of the SIM card interface 595 to realize contact and separation with the computer device 500. The computer device 500 can support four or N SIM card interfaces, N being a positive integer greater than 4. The SIM card interface 595 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. The same SIM card interface 595 can simultaneously insert multiple cards. The types of the multiple cards can be the same or different. The SIM card interface 595 can also be compatible with different types of SIM cards. The SIM card interface 595 can also be compatible with external storage cards. The computer device 500 interacts with the network through the SIM card to realize functions such as calling and data communication. In some embodiments, the computer device 500 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the computer device 500 and cannot be separated from the computer device 500.

[0236] The software system of the computer device 500 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present application takes an Android system with a layered architecture as an example to exemplarily illustrate the software structure of the computer device 500.

[0237] It should be noted that the model updating apparatus provided in the above embodiments is only exemplarily described by dividing the above functional modules when the model updating method is executed. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above. In addition, the model updating apparatus and the model updating method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0238] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or can be instructed by a program to complete the related hardware, and the program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0239] The above only describes optional embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A model updating method characterized by, The method comprises: After receiving an output result generated by a machine learning model based on input information, generating and displaying explanatory information based on the output result, the explanatory information being information for explaining the output result, the representation of the explanatory information including at least one of graphics, text or tables, the explanatory information including one or more features and feature values of the features; Receiving decision information of the explanatory information, the decision information being generated based on user operations on the displayed explanatory information, the decision information being used to indicate whether the explanation content of the explanatory information is abnormal, and determining explanatory information whose decision information indicates that the explanation content is abnormal as abnormal explanatory information; Updating a label corresponding to a target training sample in a training sample set of the machine learning model to obtain an adjusted training sample set, wherein the target training sample includes a target feature identical to an abnormal feature in the abnormal explanatory information, and the feature value of the target feature is identical to an abnormal feature value of the abnormal feature; Obtaining an updated machine learning model trained based on the adjusted training sample set.

2. The method of claim 1, wherein, The method further comprises: Obtaining an initial x value, 1≤x Determining a training sample set corresponding to a machine learning model with the optimal validation effect in the machine learning models obtained through the at least two retraining processes as the adjusted training sample set; The retraining process comprises: Updating labels corresponding to x target training samples in the training sample set to opposite labels; Training the machine learning model based on the training sample set after the label update to obtain a retrained machine learning model; Increasing the x value to obtain an updated x value and executing the retraining process again.

3. The method of claim 1, wherein, The method further comprises: Obtaining input information corresponding to the output result; Determining a target sample in the training sample set that is closest to the input information or has a distance less than a preset distance threshold to the input information; Generating the explanatory information, which includes features and feature values included in the target sample, and weights of each feature in the target sample obtained in advance.

4. The method of claim 3, wherein, The method further comprises: Dividing the training sample set into multiple classes of training samples using a clustering algorithm, and determining a reference sample in each class of training samples; Obtaining features and feature values of the reference samples in the training sample set, and obtaining weights of each feature in the reference samples; The method further comprises: Determining a target sample in the reference samples included in the training sample set that is closest to the input information or has a distance less than a preset distance threshold to the input information.

5. The method of claim 4, wherein, The acquiring the weight of each feature in the reference sample comprises: The weight of each feature in the reference sample is acquired by using a local interpretability model algorithm LIME.

6. The method of claim 1, wherein, The determining the interpretability information indicating that the explanation content is abnormal as the abnormal interpretability information comprises: When the determination information indicates that any feature in the interpretability information is an abnormal feature, the interpretability information is determined as abnormal interpretability information.

7. The method according to any one of claims 1 to 6, characterized in that, The interpretability information comprises a feature value range in which each feature value is located and / or a probability of each classification supported by the machine learning model corresponding to the output result.

8. A model update apparatus characterized by comprising: The device comprises: A generation module configured to generate interpretability information based on an output result generated by a machine learning model based on input information after receiving the output result, the interpretability information being information for explaining the output result, a representation manner of the interpretability information comprising at least one of a graph, a text or a table, the interpretability information comprising one or more features and feature values of the features; A display module configured to display the interpretability information; A first acquisition module configured to receive determination information of the interpretability information, the determination information being generated based on an operation of a user on the displayed interpretability information, the determination information being used to indicate whether an explanation content of the interpretability information is abnormal, and determining interpretability information indicated by the determination information as abnormal interpretability information; An adjustment module configured to update a label corresponding to a target training sample in a training sample set to obtain an adjusted training sample set, wherein the target training sample comprises a target feature same as an abnormal feature in the abnormal interpretability information, and a feature value of the target feature is same as an abnormal feature value of the abnormal feature; A second acquisition module configured to acquire an updated machine learning model, the machine learning model being trained based on the adjusted training sample set.

9. The apparatus of claim 8, wherein, The adjustment module is configured to: Acquire an initial x value, 1≤x The retraining process comprises: Updating labels corresponding to x target training samples in the training sample set to opposite labels; Training the machine learning model based on the training sample set after the label update to obtain a retrained machine learning model; Increasing the x value to obtain an updated x value, and executing the retraining process again. The generation module comprises:

10. The apparatus of claim 8, wherein, An acquisition submodule configured to acquire input information corresponding to the output result; A determination submodule configured to determine a target sample in the training sample set closest to the input information or having a distance less than a preset distance threshold. ​ The generating sub-module is configured to generate the interpretive information, and the interpretive information includes features and feature values included in the target sample and pre-acquired weights of each feature in the target sample.

11. The apparatus of claim 10, wherein, The device further includes: The clustering module is configured to divide the training sample set into multiple classes of training samples by using a clustering algorithm, and determine a reference sample in each class of training samples. The third obtaining module is configured to obtain features and feature values of the reference samples in the training sample set, and obtain weights of each feature in the reference samples. The determining sub-module is configured to determine a target sample in the reference samples included in the training sample set, which has a distance less than a preset distance threshold or is closest to the input information.

12. The apparatus of claim 11, wherein, The third obtaining module is configured to obtain the weights of each feature in the reference samples by using a local interpretability model algorithm (LIME).

13. The apparatus of claim 8, wherein, The first obtaining module is configured to determine that the interpretive information is abnormal interpretive information when the determination information indicates that any feature in the interpretive information is an abnormal feature.

14. The apparatus of any one of claims 8 to 13, wherein, The interpretive information includes a feature value range in which each feature value is located and / or a probability of each classification supported by the machine learning model corresponding to the output result.

15. A computer device, comprising: It includes: A processor and a memory; The memory is configured to store a computer program, and the computer program includes a computer program; The processor is configured to implement the model updating method according to any one of claims 1 to 7 when executing the computer program stored in the memory.

16. A computer storage medium, comprising, The computer storage medium stores a computer program, and the computer program causes the processor to implement the model updating method according to any one of claims 1 to 7 when being executed by the processor.

Images