A model reasoning consistency verification method

Abstract

The present invention relates to the technical field of model services, and specifically discloses a model reasoning consistency verification method, comprising: deploying a model management service and a model publishing service; importing a model in a model center and selecting appropriate parameter configuration to build an initial model; configuring training environment information for the initial model, and triggering the initial model to perform a prediction task in the training environment according to a basic data set, obtaining a training task execution result, and judging whether the initial model runs normally in the training environment, and if so, retaining the training environment; calling a batch service operator and an online service operator to the initial model of the reasoning environment at the same time and inputting the same test data to obtain a comparison output result; based on the comparison output result and the model publishing service, publishing it into a batch service or an online service according to an application scenario; the prediction task can be automatically triggered by the model management service, saving time and labor costs in the process of model consistency verification.

Description

Technical Field

[0001] The present invention relates to the field of model service technology, and in particular to a model reasoning consistency verification method. Background Art

[0002] In the field of machine learning, the accuracy verification of model results after being published as a service has always been a key challenge. In recent years, with the rapid development of machine learning model service technology, the accuracy verification technology of model service results has received widespread attention. These technologies cover methods such as model batch service publishing, model real-time service publishing, prediction result export, and result comparison, which enable the model to perform consistency verification in the prediction task and ensure the stability of the results in various applications after the actual launch. These innovations not only ensure the stability of the model, but also improve the intelligence and automation level of the machine learning system, thereby meeting the evolving needs. Traditional machine learning models can be published as batch inference services for batch prediction of data, and published as real-time inference services for real-time prediction of single data. The actual model application environment is divided into development environment, test environment, and production environment. The development environment is used for development, the test environment is used for service verification, and the production environment is used for actual application of services. Machine learning model services often face performance challenges in a constantly changing server environment. Therefore, when the model service actually transitions from the test environment stage to the production environment stage, there is a lack of systematic methods to ensure the consistency of model behavior. The performance of the model may be affected by internal issues such as the server environment where the service is located and the environmental dependencies required by the model. Due to inconsistent prediction scenarios, the published application logic and architecture often differ. Therefore, it is required to verify the consistency of the results to ensure the consistency of the model prediction results and the stability of the model application. At present, the result verification of many machine learning applications relies on manual intervention, that is, manually performing a batch prediction and exporting the results, manually performing real-time service predictions on the same data, and manually comparing the two results after exporting them.

[0003] However, model publishing of traditional machine learning models usually requires expensive human and time resources. Professional data scientists and engineers must manually intervene to build services, publish services, verify service results, take services offline, and redeploy services. This process is not only time-consuming and labor-intensive, but can also lead to human errors. Therefore, researchers have been seeking automated and intelligent methods to automatically perform consistency checks on service results and provide differentiated result analysis without human intervention.

[0004] Therefore, the present invention proposes a model reasoning consistency verification method. Summary of the invention

[0005] The present invention provides a model reasoning consistency verification method, which is used to automatically trigger a prediction task through a model management service, initiate a prediction task using existing model data, and obtain a comparison output result by simultaneously inputting the same test data into an initial model in a training environment and an initial model in an inference environment, thereby realizing consistency verification of model reasoning and saving time and labor costs in the process of model consistency verification.

[0006] The present invention provides a model reasoning consistency verification method, comprising:

[0007] S1: Deploy model management service and model publishing service;

[0008] S2: Import the model in the model center, select appropriate parameter configuration, and build the initial model;

[0009] S3: Configure the training environment information for the initial model, and trigger the initial model to perform the prediction task in the training environment according to the basic data set, obtain the training task execution result, and determine whether the initial model runs normally in the training environment based on the training task execution result. If so, retain the training environment and generate the initial model image and push it to the image warehouse;

[0010] S4: Call the batch service operator and the online service operator simultaneously to the initial model of the reasoning environment and input the same test data to obtain the comparison output result;

[0011] S5: Publish services based on the comparison output results and models, and publish them into batch services or online services according to application scenarios.

[0012] Preferably, the parameter configuration includes: model type, training data set, and initial hyperparameters.

[0013] Preferably, after the initial model is constructed, the method includes: freezing and saving the initial model.

[0014] Preferably, configuring training environment information for the initial model includes:

[0015] Use model management services and containers to build the dependency packages and algorithm packages required for the initial model operation.

[0016] Preferably, the batch service operator and the online service operator are simultaneously called to the initial model of the reasoning environment and the same test data is input to obtain a comparison output result, including:

[0017] Use the model publishing service to call the batch service operator and the real-time service operator to the initial model of the reasoning environment at the same time, and execute the model prediction task based on the same input test data to obtain the task execution results of the batch service operator and the task execution results of the real-time service operator;

[0018] The task execution results of the batch service operator are compared with the task execution results of the real-time service operator to obtain a comparison output result.

[0019] Preferably, S5: publishing services based on the comparison output results and the model, publishing them as batch services or online services according to application scenarios, including:

[0020] When the comparison output results are consistent, the model publishing service is used to publish the initial model as a batch service or online service according to the application scenario;

[0021] When the comparison output result is inconsistent, the operation logic troubleshooting or data processing is performed based on the comparison output result to obtain the trained model, and the comparison output result of the trained model is obtained. When the comparison output result of the trained model is consistent, the model publishing service is used to publish the trained model into batch services or online services according to the application scenario.

[0022] Preferably, the operation logic check or data processing is performed based on the comparison output result to obtain a trained model, including:

[0023] Obtain the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to each set of test data in the comparison output results;

[0024] Evaluate the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to each set of test data, including:

[0025]

[0026] In the formula, It is the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to a single set of test data. The task execution result of the batch service operator corresponding to a single set of test data. The value obtained by quantizing the task execution results of the batch service operator corresponding to a single set of test data according to the preset quantization mapping function. It is the task execution result of the real-time service operator corresponding to a single set of test data. The value obtained by quantizing the task execution result of the real-time service operator corresponding to a single set of test data according to the preset quantization mapping function;

[0027] Calculate the generalization degree of deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data;

[0028] When the generalization degree of deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data is not less than the generalization threshold, the operation logic of the initial model is checked and corrected to obtain a trained model.

[0029] Preferably, calculating the generalization degree of the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data includes:

[0030]

[0031] In the formula, The generalization degree of the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data, is the total number of test data sets input to both the batch service operator and the real-time service operator. For the The deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to the group test data, is the model reliability corresponding to the batch service operator, is the model reliability corresponding to the real-time service operator, The model corresponding to the batch service operator completes the first The number of model iterations completed when performing model prediction tasks on the test data set. The model corresponding to the real-time service operator completes the first The number of model iterations completed when performing the model prediction task on the test data set.

[0032] Preferably, performing operation logic troubleshooting or data processing based on the comparison output result to obtain a trained model also includes:

[0033] When the generalization degree of the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data is less than the generalization threshold, the test data is processed to obtain new test data;

[0034] The model publishing service is used to call the batch service operator and the real-time service operator respectively, and the model prediction task is executed based on the same new test data input to obtain the new task execution results of the batch service operator and the new task execution results of the real-time service operator.

[0035] The new task execution results of the batch service operator and the new task execution results of the real-time service operator are regarded as new comparison data results;

[0036] Based on the new comparison output results, the initial model is run logically checked or the new test data is processed until a trained model is obtained.

[0037] Preferably, based on the new comparison output result, the initial model is run logically checked or the new test data is processed until a trained model is obtained, including:

[0038] When the new comparison outputs a consistent result, the initial model is considered as the trained model;

[0039] When the new comparison output result is inconsistent, the deviation between the new task execution result of the batch service operator and the new task execution result of the real-time service operator corresponding to each set of new test data is evaluated;

[0040] Calculate the generalization degree of the deviation between the new task execution results of the batch service operator and the new task execution results of the real-time service operator corresponding to all groups of test data;

[0041] When the generalization of the deviation between the new task execution results of the batch service operator and the new task execution results of the real-time service operator corresponding to all the new test data is not less than the generalization threshold, the operation logic of the initial model is checked and corrected to obtain a trained model;

[0042] When the generalization of the deviation between the new task execution results of the batch service operator and the new task execution results of the real-time service operator corresponding to all groups of new test data is less than the generalization threshold, the new test data continues to be processed until a trained model is obtained.

[0043] The beneficial effects of the present invention compared with the prior art are as follows:

[0044] The result consistency check of machine learning model service is realized, and there is no need to manually initiate the inference service and then compare the results. The model management service can automatically trigger the prediction task, and call the batch service operator and the online service operator to compare the results, display detailed information, and accelerate the model verification process. For the published inference service, part of the training data will also be used to initiate the prediction task, and end-to-end consistency check will be performed to ensure that the prediction capabilities of the training model and the inference model are consistent.

[0045] Through container technology, the model management service can build the required operating environment for each model separately and perform verification tasks on the model operation to ensure the stability of the model service operation and the consistency of the model service when the model transitions from the test environment to the production environment.

[0046] The present invention provides a universal solution that is applicable to various machine learning tasks and fields, which means that it is no longer necessary to redevelop and optimize the iterative process for each new problem, thereby improving portability.

[0047] Through scheduling technology, the model management service can effectively plan and allocate computing resources to meet the needs of modeling tasks. This includes allocating computing resources, data storage, and other key resources. This maximum utilization of resources not only improves efficiency, but also reduces resource waste, helping to optimize the cost-effectiveness of model improvement.

[0048] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in this application document.

[0049] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0051] Figure 1 is a flow chart of a model reasoning consistency verification method in an embodiment of the present invention;

[0052] Figure 2 It is an execution logic flow diagram of the model management service and the model publishing service in the embodiment of the present invention;

[0053] Figure 3 The figure is a logical flow diagram of publishing the comparison output results as batch services or online services in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0055] Example 1

[0056] The present invention provides a model reasoning consistency verification method, referring to Figure 1 and Figure 2 ,include:

[0057] S1: Deploy model management service and model publishing service;

[0058] S2: Import the model in the model center and select appropriate parameter configuration (these parameters may include model type, training data set, initial hyperparameters, etc.) to build the initial model;

[0059] S3: Configure the training environment information for the initial model, and trigger the initial model to perform the prediction task in the training environment according to the basic data set, obtain the training task execution result, and determine whether the initial model runs normally in the training environment based on the training task execution result. If so, retain the training environment and generate the initial model image and push it to the image warehouse;

[0060] S4: Call the batch service operator and the online service operator simultaneously to the initial model of the reasoning environment and input the same test data to obtain the comparison output result;

[0061] S5: Based on the comparison output results and model publishing services, they are published as batch services or online services according to the application scenarios to ensure that the model behaves the same in the inference environment and the training environment.

[0062] In this embodiment, the model management service and the model publishing service are the core of the entire technology. The model management service is mainly responsible for monitoring and managing machine learning models, while the model publishing service builds an operating environment for the model to be published or publishes the model into batch inference services and online services.

[0063] In this embodiment, after completing the consistency check of the prediction results, it can be ensured that the results of the model in batch service and online service are completely consistent, ensuring the stability of the results. The model can be used to publish services, which can be published as batch services or online services according to application scenarios, to provide service tools for actual production activities. After that, end-to-end testing is performed in an inference environment, and a portion of the training data is used as input to compare the inference results with the prediction results during training. Ensure that the predictions during training and the predictions during inference are consistent. Quantitatively compare the prediction results during training and inference, and output a consistency report.

[0064] In this embodiment, throughout the process, model management and model publishing services effectively plan and allocate computing resources to meet the needs of modeling tasks and model services. This includes resource allocation, data storage, and other key resources to ensure efficient execution of tasks and maximize utilization.

[0065] The technical method of this embodiment is universal and can be applied to various machine learning tasks and fields, which means that it can be applied to a variety of application scenarios, reducing the repeated development and optimization work for different problems.

[0066] The beneficial effects of the above technology are: the consistency check of the results of the machine learning model service is realized, and there is no need to manually initiate the inference service and then compare the results. The model management service can automatically trigger the prediction task, and call the batch service operator and the online service operator to compare the results, display detailed information, and accelerate the model verification process. For the published inference service, part of the training data will also be used to initiate the prediction task, and end-to-end consistency check will be performed to ensure that the prediction capabilities of the training model and the inference model are consistent.

[0067] Through container technology, the model management service can build the required operating environment for each model separately and perform verification tasks on the model operation to ensure the stability of the model service operation and the consistency of the model service when the model transitions from the test environment to the production environment.

[0068] The present invention provides a universal solution that is applicable to various machine learning tasks and fields, which means that it is no longer necessary to redevelop and optimize the iterative process for each new problem, thereby improving portability.

[0069] Through scheduling technology, the model management service can effectively plan and allocate computing resources to meet the needs of modeling tasks. This includes allocating computing resources, data storage, and other key resources. This maximum utilization of resources not only improves efficiency, but also reduces resource waste, helping to optimize the cost-effectiveness of model improvement.

[0070] Example 2

[0071] Based on Example 1, parameter configuration includes: model type, training data set, and initial hyperparameters.

[0072] The beneficial effect of the above technology is that the model parameters that need to be configured during the initial model construction process are clarified.

[0073] Example 3

[0074] On the basis of Example 1, after the initial model is constructed, the following steps include: freezing and saving the initial model.

[0075] The beneficial effect of the above technology is to ensure that the parameters and files of the model itself will not change throughout the entire process from training to inference. This step is the starting point of the technology and usually requires human participation.

[0076] Example 4

[0077] Based on Example 1, the training environment information is configured for the initial model, including:

[0078] Use model management services and containers to build the dependency packages and algorithm packages required for the initial model operation.

[0079] The beneficial effects of the above technologies are: the model management service will use container technology to configure the environment such as dependency packages and algorithm packages required for model construction and operation, and clarify the specific items that need to be configured in the environment configuration.

[0080] Example 5

[0081] Based on Example 1, the batch service operator and the online service operator are simultaneously called to the initial model of the reasoning environment and the same test data is input to obtain the comparison output results, including:

[0082] Use the model publishing service to call the batch service operator and the real-time service operator to the initial model of the reasoning environment at the same time, and execute the model prediction task based on the same input test data to obtain the task execution results of the batch service operator and the task execution results of the real-time service operator;

[0083] The task execution results of the batch service operator are compared with the task execution results of the real-time service operator to obtain a comparison output result.

[0084] The beneficial effects of the above technology are: the model management service can automatically trigger the prediction task, call the batch service operator and the online service operator to compare the results, display detailed information, and accelerate the model verification process.

[0085] Example 6

[0086] Based on Example 1, S5: Based on the comparison output results and model publishing services, publish them into batch services or online services according to the application scenarios, refer to Figure 3 ,include:

[0087] When the comparison output results are consistent, the model publishing service is used to publish the initial model as a batch service or online service according to the application scenario;

[0088] When the comparison output result is inconsistent, the operation logic troubleshooting or data processing is performed based on the comparison output result to obtain the trained model, and the comparison output result of the trained model is obtained. When the comparison output result of the trained model is consistent, the model publishing service is used to publish the trained model into batch services or online services according to the application scenario.

[0089] In this embodiment, the model publishing service calls the batch service operator and the real-time service operator respectively to perform the model prediction task, compares the results of the two prediction tasks, and outputs the comparison results. If the results are completely consistent, the service can be directly published for application. If the results are inconsistent, the report contains detailed comparison information of the two results, and it is necessary to check the operating logic such as missing value filling and other data processing.

[0090] The beneficial effects of the above technology are: by comparing the output results, the model verification process is accelerated to ensure that the model behaves the same in the inference environment and the training environment.

[0091] Example 7

[0092] On the basis of Example 6, the operation logic is checked or data is processed based on the comparison output result to obtain a trained model, including:

[0093] Obtain the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to each set of test data in the comparison output results;

[0094] Evaluate the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to each set of test data, including:

[0095]

[0096] In the formula, It is the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to a single set of test data. The task execution result of the batch service operator corresponding to a single set of test data. The value obtained by quantizing the task execution results of the batch service operator corresponding to a single set of test data according to the preset quantization mapping function. It is the task execution result of the real-time service operator corresponding to a single set of test data. The value obtained by quantizing the task execution result of the real-time service operator corresponding to a single set of test data according to the preset quantization mapping function;

[0097] Calculate the generalization degree of deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data;

[0098] When the generalization degree of deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data is not less than the generalization threshold, the operation logic of the initial model is checked and corrected to obtain a trained model.

[0099] In this embodiment, since the batch service operator and the real-time service operator need to use multiple sets of test data to perform multiple result consistency comparisons to ensure that the judgment result that the model behaves the same in the inference environment and the training environment has generalization performance, there are multiple sets of test data, and each task execution requires a set of test data.

[0100] In this embodiment, It is a preset quantitative mapping function, that is, a mapping function that quantifies the output results of the corresponding model of the batch service operator or the real-time service operator (that is, the task execution result) into a specific numerical value.

[0101] In this embodiment, the generalization threshold is preset to distinguish whether to check and correct the operating logic of the initial model or to process the test data to obtain a new trained model.

[0102] The beneficial effects of the above technology are: by evaluating the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to each group of test data, and calculating the generalization of the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data, the generalization of the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data is used as the judgment basis to determine whether logic troubleshooting or data processing is needed in the future, so as to realize the correction and iteration of the model.

[0103] Example 8

[0104] On the basis of Example 7, the generalization degree of the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data is calculated, including:

[0105]

[0106] In the formula, The generalization degree of the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data, is the total number of test data sets input to both the batch service operator and the real-time service operator. For the The deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to the group test data, is the model reliability corresponding to the batch service operator, is the model reliability corresponding to the real-time service operator, The model corresponding to the batch service operator completes the first The number of model iterations completed when performing model prediction tasks on the test data set. The model corresponding to the real-time service operator completes the first The number of model iterations completed when performing the model prediction task on the test data set.

[0107] In this embodiment, since the reliability of the model corresponding to the batch service operator or the real-time service operator increases as the number of times the model is applied increases, the model corresponding to the batch service operator or the real-time service operator is used to complete the first The number of model iterations completed when the group test data executes the model prediction task, retrieves the preset number of iterations-model reliability list, and determines the model reliability corresponding to the batch service operator or the real-time service operator.

[0108] In this embodiment, the model corresponding to the batch service operator or the real-time service operator completes the first The number of model iterations completed when executing the model prediction task on the test data set refers to the number of model iterations completed by the model corresponding to the batch service operator or the real-time service operator based on the input. The number of completed model applications when executing model prediction tasks on the test data set (also the total number of completed prediction task executions).

[0109] The beneficial effects of the above technology are: by introducing the model reliability corresponding to the batch service operator, the model reliability corresponding to the real-time service operator, and the model corresponding to the batch service operator, the model reliability corresponding to the real-time service operator is completed based on the input. The number of model iterations completed when executing the model prediction task on the test data set, and the number of model iterations completed when the model corresponding to the real-time service operator completes the first The number of model iterations completed when executing the model prediction task on the group test data enables an accurate assessment of the generalization of the deviation between the task execution results of the batch service operators and the task execution results of the real-time service operators corresponding to all the group test data.

[0110] Example 9

[0111] On the basis of Example 7, the operation logic is checked or data is processed based on the comparison output result to obtain a trained model, which further includes:

[0112] When the generalization degree of the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data is less than the generalization threshold, the test data is processed to obtain new test data;

[0113] The model publishing service is used to call the batch service operator and the real-time service operator respectively, and the model prediction task is executed based on the same new test data input to obtain the new task execution results of the batch service operator and the new task execution results of the real-time service operator.

[0114] The new task execution results of the batch service operator and the new task execution results of the real-time service operator are regarded as new comparison data results;

[0115] Based on the new comparison output results, the initial model is run logically checked or the new test data is processed until a trained model is obtained.

[0116] In this embodiment, data processing refers to partially correcting or replacing the test data to obtain new test data.

[0117] The beneficial effect of the above technology is: when the generalization of the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data is less than the generalization threshold, the trained model is obtained by processing the test data to obtain new test data.

[0118] Embodiment 10:

[0119] On the basis of Example 9, based on the new comparison output result, the initial model is run logically checked or the new test data is processed until a trained model is obtained, including:

[0120] When the new comparison outputs a consistent result, the initial model is considered as the trained model;

[0121] When the new comparison output result is inconsistent, the deviation between the new task execution result of the batch service operator and the new task execution result of the real-time service operator corresponding to each set of new test data is evaluated;

[0122] Calculate the generalization degree of the deviation between the new task execution results of the batch service operator and the new task execution results of the real-time service operator corresponding to all groups of test data;

[0123] When the generalization of the deviation between the new task execution results of the batch service operator and the new task execution results of the real-time service operator corresponding to all the new test data is not less than the generalization threshold, the operation logic of the initial model is checked and corrected to obtain a trained model;

[0124] When the generalization of the deviation between the new task execution results of the batch service operator and the new task execution results of the real-time service operator corresponding to all groups of new test data is less than the generalization threshold, the new test data continues to be processed until a trained model is obtained.

[0125] Based on the newly obtained comparison data results, the initial model is run logically or the new test data is processed to obtain a trained model.

[0126] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A model reasoning consistency verification method, characterized in that: include: S1: Deploy model management service and model publishing service; S2: Import the model in the model center, select appropriate parameter configuration, and build the initial model; S3: Configure the training environment information for the initial model, and trigger the initial model to perform the prediction task in the training environment according to the basic data set, obtain the training task execution result, and determine whether the initial model runs normally in the training environment based on the training task execution result. If so, retain the training environment and generate the initial model image and push it to the image warehouse; S4: calling the batch service operator and the real-time service operator simultaneously to the initial model of the reasoning environment and inputting the same test data to obtain the comparison output result; S5: Based on the comparison output results and model publishing services, they can be published as batch services or real-time services according to the application scenarios; Among them, the training environment information is configured for the initial model, including: Use model management services and containers to build dependency packages and algorithm packages required for the initial model. The batch service operator and the real-time service operator are called simultaneously to the initial model of the reasoning environment and the same test data is input to obtain the comparison output results, including: Use the model publishing service to call the batch service operator and the real-time service operator to the initial model of the reasoning environment at the same time, and execute the model prediction task based on the same input test data to obtain the task execution results of the batch service operator and the task execution results of the real-time service operator; Compare the task execution results of the batch service operator with the task execution results of the real-time service operator to obtain the comparison output results; Wherein, step S5: based on the comparison output results and model publishing services, publishing them into batch services or real-time services according to application scenarios, including: When the comparison output results are consistent, the model publishing service is used to publish the initial model as a batch service or real-time service according to the application scenario; When the comparison output result is inconsistent, the operation logic is checked or data is processed based on the comparison output result to obtain the trained model, and the comparison output result of the trained model is obtained. When the comparison output result of the trained model is consistent, the trained model is published as a batch service or a real-time service according to the application scenario using the model publishing service; Among them, the operation logic is checked or data is processed based on the comparison output results to obtain the trained model, including: Obtain the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to each set of test data in the comparison output results; Evaluate the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to each set of test data, including: In the formula, It is the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to a single set of test data. The task execution result of the batch service operator corresponding to a single set of test data. The value obtained by quantizing the task execution results of the batch service operator corresponding to a single set of test data according to the preset quantization mapping function. It is the task execution result of the real-time service operator corresponding to a single set of test data. The value obtained by quantizing the task execution result of the real-time service operator corresponding to a single set of test data according to the preset quantization mapping function; Calculate the generalization degree of deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data; When the generalization of the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data is not less than the generalization threshold, the operation logic of the initial model is checked and corrected to obtain a trained model; The generalization degree of the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data is calculated, including: In the formula, The generalization degree of the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data, is the total number of test data sets input to both the batch service operator and the real-time service operator. For the The deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to the group test data, is the model reliability corresponding to the batch service operator, is the model reliability corresponding to the real-time service operator, The model corresponding to the batch service operator completes the first The number of model iterations completed when performing model prediction tasks on the test data set. The model corresponding to the real-time service operator completes the first The number of model iterations completed when performing the model prediction task on the test data set.

2. The model reasoning consistency verification method according to claim 1 is characterized in that: Parameter configuration includes: model type, training data set, and initial hyperparameters.

3. The model reasoning consistency verification method according to claim 1 is characterized in that: After the initial model is constructed, the following steps are performed: freezing and saving the initial model.

4. The model reasoning consistency verification method according to claim 1, characterized in that: Based on the comparison output results, the operation logic is checked or data is processed to obtain the trained model, which also includes: When the generalization degree of the deviation between the task execution results of the batch service operator and the task execution results of the real-time service operator corresponding to all groups of test data is less than the generalization threshold, the test data is processed to obtain new test data; The model publishing service is used to call the batch service operator and the real-time service operator respectively, and the model prediction task is executed based on the same new test data input to obtain the new task execution results of the batch service operator and the new task execution results of the real-time service operator. The new task execution results of the batch service operator and the new task execution results of the real-time service operator are regarded as new comparison data results; Based on the new comparison output results, the initial model is run logically checked or the new test data is processed until a trained model is obtained.

5. The model reasoning consistency verification method according to claim 4 is characterized in that: Based on the new comparison output results, the initial model is run logically or the new test data is processed until a trained model is obtained, including: When the new comparison outputs a consistent result, the initial model is considered as the trained model; When the new comparison output result is inconsistent, the deviation between the new task execution result of the batch service operator and the new task execution result of the real-time service operator corresponding to each set of new test data is evaluated; Calculate the generalization degree of the deviation between the new task execution results of the batch service operator and the new task execution results of the real-time service operator corresponding to all groups of test data; When the generalization of the deviation between the new task execution results of the batch service operator and the new task execution results of the real-time service operator corresponding to all the new test data is not less than the generalization threshold, the operation logic of the initial model is checked and corrected to obtain a trained model; When the generalization of the deviation between the new task execution results of the batch service operator and the new task execution results of the real-time service operator corresponding to all groups of new test data is less than the generalization threshold, the new test data continues to be processed until a trained model is obtained.

Images