Model data processing method and device, storage medium and computer program product

Abstract

The invention discloses a model data processing method and device, a storage medium and a computer program product, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining a full life cycle management process of a machine learning model, and determining a current process node where a target model is located from all process nodes of the full life cycle management process; determining a current node data processing tool corresponding to the current process node from a preset tool set; determining current node data corresponding to the current process node, and calling the current node data processing tool to carry out data processing on the current node data to obtain node output data; and outputting the node output data according to a preset data transmission path corresponding to the current process node. According to the invention, the efficiency of full-life-cycle management is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model data processing method, device, storage medium and computer program product. Background Art

[0002] With the rapid development of artificial intelligence technology, machine learning models are increasingly used in all walks of life, providing strong support for the intelligence of various fields. The full life cycle management of machine learning models is the key to ensuring that the model can operate efficiently, stably and in compliance with regulations, including requirements writing, model development, testing, deployment, monitoring, backchecking and exiting. At present, each link in the full life cycle management of machine learning models relies on manual operation. For example, manual design of test cases is required during the testing phase, and testers also need to manually analyze the accuracy and stability of the model based on the test results; after the model is deployed, manual monitoring of the model's operating status and performance indicators is required, and regular manual backchecking of the model is required, resulting in inefficient cycle management and difficulty in meeting business needs.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a model data processing method, device, storage medium and computer program product, aiming to solve the technical problem of low efficiency in model full life cycle management.

[0005] To achieve the above purpose, the present application proposes a model data processing method, which comprises:

[0006] Obtaining the full life cycle management process of the machine learning model, and determining the current process node where the target model is currently located from each process node of the full life cycle management process;

[0007] Determine a current node data processing tool corresponding to the current process node from a preset tool set;

[0008] Determine the current node data corresponding to the current process node, and call the current node data processing tool to process the current node data to obtain node output data;

[0009] The node output data is output according to a preset data transmission path corresponding to the current process node.

[0010] In one embodiment, the current process node is a model development node; the step of determining the current node data corresponding to the current process node and calling the current node data processing tool to process the current node data to obtain node output data includes:

[0011] Obtaining machine learning platform buried data and a preset model requirement document, and using the model requirement document and the machine learning platform buried data as current node data;

[0012] The document generation tool is called to process the current node data to obtain node output data.

[0013] In one embodiment, the document generation tool includes a template matching module and a large language model;

[0014] The step of calling the document generation tool to process the current node data to obtain node output data includes:

[0015] Calling the template matching module to determine the target buried point data corresponding to the current process node from the buried point data of the machine learning platform;

[0016] Filling the non-evaluation data in the target buried point data into a preset text template to obtain a description text through the template matching module, and converting the evaluation data in the target buried point data into an evaluation picture through the template matching module;

[0017] Inputting the model requirement document into the large language model to obtain semantic text;

[0018] The semantic text, the description text and the evaluation picture are used as node output data.

[0019] In one embodiment, the current process node is a deployment approval node; the step of determining the current node data corresponding to the current process node and calling the current node data processing tool to process the current node data to obtain node output data includes:

[0020] Obtain a model process document, and use the model process document as current node data, wherein the model process document is a document recording the target model development process and test process;

[0021] A document scanning tool is called to process the current node data to obtain a scanning result, and node output data is generated based on the scanning result.

[0022] In one embodiment, the step of calling a document scanning tool to process the current node data to obtain a scanning result includes:

[0023] Calling a document scanning tool to extract chapter contents according to each chapter of the model process document;

[0024] Performing integrity check on the model process document based on the chapter content to obtain an integrity check result;

[0025] Performing risk detection on the model process document based on the chapter content to obtain a risk detection result;

[0026] The integrity detection result and the risk detection result are used as scanning results.

[0027] In one embodiment, the current process node is a model monitoring node; the step of determining the current node data corresponding to the current process node and calling the current node data processing tool to process the current node data to obtain node output data includes:

[0028] Acquire model operation data collected by a model operation monitoring tool, and use the model operation data as current node data;

[0029] The model operation alarm tool is called to process the current node data based on the current node data to obtain node output data.

[0030] In one embodiment, the step of calling the model alarm tool to process the current node data based on the current node data to obtain node output data includes:

[0031] Calling a model alarm tool to extract operation status data from the model operation data, and performing an operation status detection on the target model based on the operation status data to obtain an operation status detection result;

[0032] Calling the model alarm tool to extract model output data from the model operation data, and performing quality inspection on the target model based on the model output data to obtain a quality inspection result;

[0033] Calling the model alarm tool to extract model check data from the model operation data, and performing application effect detection on the target model based on the model check data to obtain a model application detection result, wherein the model check data is used to evaluate the model application effect;

[0034] The model alarm tool makes an alarm decision based on the operating status detection result, the quality detection result and the model application detection result.

[0035] In addition, to achieve the above-mentioned purpose, the present application also proposes a model data processing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the model data processing method described above.

[0036] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the model data processing method described above are implemented.

[0037] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the model data processing method described above are implemented.

[0038] In the present application, by acquiring the full life cycle management process of the machine learning model, the current process node where the target model is currently located is determined from each process node of the full life cycle management process; the current node data processing tool corresponding to the current process node is determined from a preset tool set; the current node data corresponding to the current process node is determined, and the current node data processing tool is called to perform data processing on the current node data to obtain node output data; the node output data is output according to the preset data transmission path corresponding to the current process node.

[0039] In this application, automated tools can be used to process and analyze the current node data of each process node based on preset rules and algorithms, reducing reliance on manual operations, thereby speeding up the entire process from model development to deployment to monitoring, significantly improving the efficiency of full life cycle management, enabling the model to respond more quickly to changes in business needs, and reducing the possibility of human errors, significantly improving the accuracy and stability of model management. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A flowchart of the first embodiment of the model data processing method of the present application is provided;

[0043] Figure 2 A flow chart of the second embodiment of the model data processing method of the present application;

[0044] Figure 3A schematic diagram of a brief flow chart of a model data processing method provided in one embodiment of the present application;

[0045] Figure 4 A brief flowchart of the full life cycle management provided by an embodiment of the present application

[0046] Figure 5 A simplified flowchart of a standardized approval process provided for an embodiment of the present application;

[0047] Figure 6 A schematic diagram of a brief process for generating a model development report provided in an embodiment of the present application;

[0048] Figure 7 A schematic diagram of a simplified architecture of a model alarm provided in an embodiment of the present application;

[0049] Figure 8 A schematic diagram of a brief process of model monitoring provided in an embodiment of the present application;

[0050] Fig. 9 Schematic diagram of the device structure of the hardware operating environment involved in the model data processing method in the embodiment of the present application.

[0051] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0053] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0054] The main solution of the embodiment of the present application is: by obtaining the full life cycle management process of the machine learning model, determine the current process node where the target model is currently located from each process node of the full life cycle management process; determine the current node data processing tool corresponding to the current process node from the preset tool set; determine the current node data corresponding to the current process node, and call the current node data processing tool to perform data processing on the current node data to obtain node output data; output the node output data according to the preset data transmission path corresponding to the current process node.

[0055] In this embodiment, for the convenience of description, the following description is made with the model data processing device as the execution subject.

[0056] With the rapid development of artificial intelligence technology, machine learning models are increasingly used in all walks of life. However, the full life cycle management of machine learning models, including requirements writing, model development, testing, deployment, monitoring, back-checking and exit, faces many challenges. The existing machine learning model development and management system mainly focuses on the rapid construction and online iteration of models, providing a series of modeling algorithms, data processing, feature analysis and other tool sets, which greatly reduces the connection cost from model creation to model deployment and ensures the rapid use of model services. At present, each link in the full life cycle management of machine learning models relies on manual operation. For example, manual design of test cases is required in the testing phase, and testers also need to manually analyze the accuracy and stability of the model based on the test results; after the model is deployed, manual monitoring of the model's operating status and performance indicators is required, and regular manual back-checking of the model is required, resulting in low cycle management efficiency and difficulty in meeting business needs.

[0057] The present application provides a solution that uses automated tools to process and analyze the current node data of each process node based on preset rules and algorithms, reducing reliance on manual operations, thereby speeding up the entire process from model development to deployment to monitoring, significantly improving the efficiency of full life cycle management, enabling the model to respond more quickly to changes in business needs, and reducing the possibility of human errors, significantly improving the accuracy and stability of model management.

[0058] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a model data processing device, etc. The following takes the model data processing device as an example to illustrate this embodiment and the following embodiments.

[0059] Based on this, the present application embodiment provides a model data processing method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the model data processing method of the present application.

[0060] In this embodiment, the model data processing method includes steps S10 to S40:

[0061] Step S10, obtaining the full life cycle management process of the machine learning model, and determining the current process node where the target model is currently located from each process node of the full life cycle management process;

[0062] Full life cycle management is a comprehensive and systematic management of machine learning models from requirements writing, model development, testing, deployment, monitoring, backchecking, exit and other aspects to ensure that the model can run efficiently, stably and in compliance with regulations. The full life cycle management process is a series of orderly and coherent steps or stages set to achieve full life cycle management. Each step or stage corresponds to a specific link in the model life cycle.

[0063] The target model is the machine learning model that is currently undergoing full life cycle management. The process node where the target model is currently located is called the current process node. Specifically, the current process node where the target model is currently located can be determined by analyzing the logs, metadata and other information of the machine learning platform. There is no restriction here and it can be set according to actual needs.

[0064] Step S20, determining a current node data processing tool corresponding to the current process node from a preset tool set;

[0065] The preset toolset is a series of tools that are prepared in advance to support the full life cycle management of machine learning models. These tools can provide data processing, document generation, compliance checking and other functions for different process nodes. The node data processing tool is a tool in the preset toolset that is designed for a specific process node and is used to process the node data.

[0066] According to the process node where the target model is currently located, select the corresponding node data processing tool from the preset tool set. For example, if the target model is currently in the model development stage, select a tool that can assist in generating a model development report; if it is in the compliance check stage, select a compliance document scanning tool.

[0067] In a feasible implementation manner, the process of determining the current node data processing tool corresponding to the current process node from the preset tool set may be:

[0068] S201, determining a node type of the current process node, wherein the node type is used to characterize an operation content corresponding to the current process node;

[0069] Node type is the result of classifying nodes according to their tasks or operation contents in the compliance management process of the machine learning model throughout its life cycle. Node type reflects the function and role of the process node.

[0070] According to the definition of the compliance management process of the machine learning model's entire life cycle, each node has its specific tasks or operations, such as writing reports, approving requirements, monitoring model status, etc. By analyzing the tasks or operation content of the current node, it can be classified into reporting nodes, approval nodes, monitoring nodes, or composite nodes, etc. It should be noted that reporting nodes refer to process nodes responsible for generating reports or documents. In the compliance management process of the machine learning model's entire life cycle, reporting nodes are usually nodes that record key information such as model requirements; approval nodes refer to process nodes responsible for reviewing and approving or rejecting a request. In the compliance management process of the machine learning model's entire life cycle, approval nodes are usually used to conduct compliance reviews of model requirements, deployment plans, etc.; monitoring nodes refer to process nodes responsible for monitoring the model's execution status, data quality, or application effects. In the compliance management process of the machine learning model's entire life cycle, monitoring nodes are crucial for timely discovery and handling of potential model problems. Composite node: refers to a process node that contains multiple types of operations or tasks. In the full life cycle compliance management process of machine learning models, composite nodes may involve the use of multiple stages or multiple tools. For example, development requires two actions: development and generation of development reports. Therefore, it is necessary to select appropriate tools from the preset tool set according to the specific task.

[0071] S202, determining a tool corresponding to the node type from various tools in a preset tool set, and using the tool corresponding to the node type as a current node data processing tool.

[0072] After determining the node type of the current node, the purpose of this step is to find a tool that matches the node type from the preset tool set. If the node type is a report node, such as a requirement writing node or a model exit node, the tool corresponding to the node type is a document generation tool as the tool corresponding to the node type; if the node type is an approval node, such as a requirement approval node or a deployment approval node, the document scanning tool in the preset tool set is used as the data processing tool for the current node of the model; if the node type is a monitoring node, the model monitoring tool and the document in the preset tool set are used as the data processing tool for the current node of the model; if the node type is a composite node, such as a model development node, a model testing node, a model monitoring node, and a model checkback node, the target tool corresponding to the node stage is determined from the preset tool set, and the target tool is used as the data processing tool for the current node of the model. For example, when the node stage of the current process node is the approval stage, the document scanning tool corresponding to the approval stage is called; when the node stage of the process node is the report generation stage, the document generation tool is called.

[0073] Step S30, determining the current node data corresponding to the current process node, and calling the current node data processing tool to process the current node data to obtain node output data;

[0074] Current node data refers to data related to the machine learning model generated in the current process node, such as model code, data sets, test reports, etc. Node output data refers to output data related to the current process node obtained after being processed by the node data processing tool, such as model development reports, compliance inspection reports, etc.

[0075] Determine the current node data corresponding to the current process node. This data may be directly obtained from the machine learning platform, or obtained from the output data of the process node before the current process node, or obtained from a third-party platform. There is no restriction here. Call the corresponding node data processing tool to process the current node data to obtain the node output data.

[0076] Step S40: outputting the node output data according to the preset data transmission path corresponding to the current process node.

[0077] The data transmission path refers to the path that transmits the node output data from the current process node to the next process node or the final output location.

[0078] It can be understood that the process of outputting data from an output node can be: determining the output method of the node output data based on the characteristics of the current process node and the preset data transmission path. For example, if the current process node is the model development stage and the next process node is the model testing stage, the model development report may be directly transmitted to the testing team; if the current process node is the compliance inspection stage and the inspection passes, the compliance inspection report may be submitted to the relevant department as the final output; according to the preset data transmission path, the node output data is output to the specified location or the next process node.

[0079] By providing an efficient and systematic method for processing data throughout the entire life cycle of machine learning models, it can automatically select the corresponding data processing tools for processing according to the needs of different process nodes, and output node output data that meets the requirements, thereby supporting the full life cycle management of machine learning models.

[0080] In this embodiment, the automation tool can process and analyze the current node data of each process node based on preset rules and algorithms, reducing the reliance on manual operations, thereby speeding up the entire process from model development to deployment to monitoring, significantly improving the efficiency of full life cycle management, enabling the model to respond more quickly to changes in business needs, and reducing the possibility of human errors, significantly improving the accuracy and stability of model management.

[0081] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , the current process node is a model development node; the step S30: determining the current node data corresponding to the current process node, and calling the current node data processing tool to process the current node data to obtain node output data, including:

[0082] Step S301, obtaining machine learning platform buried point data and a preset model requirement document, and using the model requirement document and the machine learning platform buried point data as current node data;

[0083] Model development is the bridge between requirement writing and model testing. After determining that the current process node is model development, the specific tasks that need to be performed later can be clarified, such as data preparation, model training, model evaluation, etc.

[0084] Machine learning platform buried data refers to the data collection points that are pre-set in the machine learning platform to monitor and record various information in the model development process. These data points can capture various key information in the model development process, such as data sources, data processing logic, feature engineering configuration, algorithm model hyperparameter settings, etc. This information is crucial for the development, testing, deployment and monitoring of subsequent models. The preset model requirements document is a detailed document written by relevant personnel based on business needs and technical requirements before the model is developed. The document describes in detail the background, goals, input data, output requirements, performance indicators and other key information of the model development.

[0085] After determining the current process node for model development, relevant data and information need to be collected to support subsequent development work.

[0086] Step S302: calling a document generation tool to process the current node data to obtain node output data.

[0087] Use document generation tools to process the collected machine learning platform embedded data and preset model requirement documents to generate various documents required in the model development process. These documents may include model development reports, data processing logic diagrams, feature engineering configuration instructions, etc.

[0088] A document generation tool is a tool that can automatically generate various types of documents. It can generate the required documents based on the input data and information, according to preset templates and formats. In the compliance management process of the machine learning model throughout its life cycle, the document generation tool can improve the efficiency and accuracy of document generation.

[0089] Using document generation tools can improve the efficiency and accuracy of document generation and reduce the cost and risk of manual document writing. At the same time, the generated documents can also serve as an important reference in the subsequent development, testing, deployment and monitoring processes.

[0090] In a feasible implementation manner, the document generation tool includes a template matching module and a large language model; the step S302: calling the document generation tool to process the current node data to obtain node output data, includes:

[0091] Step S3021, calling the template matching module to determine the target buried point data corresponding to the current process node from the buried point data of the machine learning platform;

[0092] During the development of the machine learning model, the platform will record a large amount of embedded data. This data contains various information in the model development process, such as data source information, data processing logic, feature engineering configuration, algorithm model hyperparameter settings, etc. From these data, the data related to the current process node (ie, model development node) is selected as the target embedded data for subsequent processing.

[0093] According to preset rules or configurations, data related to model development nodes are filtered out from the machine learning platform's embedded data. These data may include key information such as the data source, data preprocessing logic, feature selection, etc. used in the model development process.

[0094] Step S3022, filling the non-evaluation data in the target buried point data into a preset text template to obtain a description text through the template matching module, and converting the evaluation data in the target buried point data into an evaluation picture through the template matching module;

[0095] After obtaining the target tracking data, the target tracking data is divided into two parts: non-evaluation data and evaluation data for processing. Non-evaluation data is used to describe, record or configure non-quantitative or difficult to directly evaluate information in the model development process, such as data source information, data processing logic, feature engineering configuration, etc. This information is not directly used to evaluate model performance. Evaluation data is data that can be directly used to evaluate model performance or effect, such as model accuracy, recall rate, etc. Evaluation data can be converted into visual forms (such as charts, images) to more intuitively display model performance data. For non-evaluation data, the system fills the relevant data into the template according to the preset text template (such as the model development report template) to generate a descriptive text. For evaluation data, the system may use charts, curves, etc. to intuitively display data changes and trends. These charts or curves will be converted into picture format, that is, evaluation pictures.

[0096] It can be understood that the description text refers to the text obtained by filling the non-evaluation data in the target tracking data into the preset text template through the template matching module, which contains various detailed information in the model development process, such as data source, data processing logic, feature engineering configuration, etc.

[0097] Step S3023, inputting the model requirement document into the large language model to obtain semantic text;

[0098] Semantic text refers to text generated by semantically understanding and analyzing the input text through natural language processing technology (such as a large language model). Semantic text is obtained by inputting the model requirement document into a large language model. It contains the key information and logic in the model requirement document and presents it in a more structured and clear manner.

[0099] The model requirement document is used as input and passed to the large language model. The large language model performs semantic understanding and analysis on the input document and extracts key information, logical relationships, and constraints in the document. Based on the extracted information and logical relationships, the large language model generates structured semantic text.

[0100] Through the semantic understanding and analysis capabilities of large language models, the utilization and readability of model requirement documents are improved, allowing developers to understand model requirements more accurately and reducing the difficulty and cost of manually interpreting and understanding model requirement documents.

[0101] Step S3024: Use the semantic text, the description text and the evaluation picture as node output data.

[0102] The generated semantic text, description text and evaluation image are used as the output data of the current process node. These output data will be used as the input of subsequent process nodes to guide the further development and optimization of the model.

[0103] Exemplarily, in a feasible implementation manner, the process of calling the document generation tool to process the current node data to obtain the node output data may be:

[0104] Step 1, system rule information extraction: extract data source information, data processing logic, feature engineering configuration, algorithm model hyperparameter settings, modeling environment resource specification selection, algorithm model fitting process, algorithm model interpretability, task operation log, algorithm evaluation results, etc. from the machine learning platform data, generate descriptive text through template matching, and convert the evaluation data into images. Step 2, the large language model extracts the background summary content of the requirement document, the key logical parameters of the algorithm code and other information, and inputs them into the large language model to assist in semantic understanding and key information extraction to generate text. Please refer to Figure 3, the specific process of information extraction by the large language model can be: first, clarify the input parameters, and determine that the input parameters are flowId (a unique identifier for identifying the process), requirementId (a unique identifier for the requirement), requirementName (the name of the requirement), and requirementType (the type of requirement); associate the requirements according to the input parameters, and store the associated data in a table in the database. For example, the associated data can be placed in the mdpdevdb.requirement_relation (table name in the database) table; retrieve the corresponding requirement document from the database according to requirementId and requirementType, and process the retrieved requirement document, that is, process the requirement document title, and assemble the title into a List <catalog>In the form of; traversing List <catalog>For each chapter in the , parse the corresponding chapter content of each chapter. In the process of parsing the chapter content, detect whether the requirement type is a model algorithm requirement. If it is a model algorithm requirement, obtain the second-level title of the chapter, such as the overview and the content under the related topic measurement indicator chapter. If it is not a model algorithm requirement, obtain the first-level title of the chapter, such as the feature overview and the content under the related topic measurement indicator chapter; then assemble the obtained content into a List <modeldocllmquerydataentity>In the form of and stored in the database, call the DTO (Data Transfer Object) of the large model according to the assembled data assembly, and then call the query interface of the large model in document-service, passing DTO as the input parameter; the large model returns the content and stores it in the database. Step 3, combine the above two parts of content, output the model document, and use the output document as the node output data.

[0105] In a feasible implementation manner, the current process node is a deployment approval node; the step S30: determining the current node data corresponding to the current process node, and calling the current node data processing tool to process the current node data to obtain node output data, includes:

[0106] Step S303, obtaining a model process document, and using the model process document as current node data, wherein the model process document is a document recording the target model development process and test process;

[0107] At present, modelers' understanding and recognition of model compliance management requirements vary from person to person, and there is a lack of relatively standardized and consistent compliance management processes, which greatly reduces the effectiveness of implementing model compliance requirements and reduces the compliance rate. This implementation method uses tools to review compliance and improve the standardization and efficiency of model compliance reviews.

[0108] Model process documentation refers to the documentation that records the entire process of machine learning models from requirement writing, development, testing to deployment, usually including key content such as model requirement documents, development reports, and test reports. It is understandable that by providing a complete model development process and test records as node data for the deployment approval node, it helps the approvers to fully understand the model and make more accurate and comprehensive approval decisions.

[0109] Step S304: calling a document scanning tool to process the current node data to obtain a scanning result, and generating node output data based on the scanning result.

[0110] Deployment approval needs to ensure the integrity and compliance of the model process documents, so the documents need to be scanned and checked comprehensively. By integrating compliance checklists and rule templates, document scanning tools can automatically identify and mark potential risk items in documents, thereby improving the efficiency and accuracy of approval.

[0111] The automated scanning and inspection process reduces the workload of manual review and improves approval efficiency. By identifying and marking potential risk items, it helps approvers quickly locate and handle problems, improves the security and stability of model deployment, and reduces business risks caused by model defects or compliance issues.

[0112] In a feasible implementation manner, the step S304: the step of invoking a document scanning tool to process the current node data to obtain a scanning result, includes:

[0113] Step S3041, calling a document scanning tool to extract chapter contents according to each chapter of the model process document;

[0114] The document scanning tool accurately extracts the content of each chapter from the model process document. Specifically, the document scanning tool will parse the structure of the document and identify the boundaries and hierarchical relationships of each chapter; content extraction: Based on the parsing results, the tool will extract the content chapter by chapter and store it in an appropriate data structure for subsequent processing.

[0115] By extracting content in a structured manner, the possibility of missing or incorrect information can be reduced. In addition, the automated extraction process can significantly shorten the time for document processing and improve overall work efficiency.

[0116] Step S3042, performing integrity check on the model process document based on the chapter content to obtain an integrity check result;

[0117] After extracting the contents of each chapter, the document scanning tool will perform a completeness check to ensure that all necessary chapters in the model process document exist and that the contents of each chapter meet the predetermined requirements. Exemplarily, the specific detection process can be: obtaining a set of predefined rules to determine which chapters are necessary and what key information each chapter should contain; the tool matches the extracted chapter content with the predefined rules to check whether all necessary chapters exist; for each necessary chapter, the tool will further verify whether its content meets the requirements, such as whether it contains all key information; the tool will output the completeness test results, which at least include missing chapters or non-compliant chapters.

[0118] Integrity testing can ensure that model process documentation meets relevant compliance requirements and reduce compliance risks.

[0119] Step S3043, performing risk detection on the model process document based on the chapter content to obtain a risk detection result;

[0120] The document scanning tool also performs risk detection to identify possible compliance risk points in the model process documents, such as data risk, model defect risk, ethical risk, etc. For example, the specific detection process can be: obtaining a pre-set risk list, which includes compliance risk points and their corresponding judgment criteria; the tool will conduct an in-depth analysis of the extracted chapter content and determine whether there is a compliance risk in combination with the risk list; for the identified risk points, the tool will record and classify them in detail; the tool will output the risk detection results, indicating the specific risk points in the model process documents and their possible consequences.

[0121] Risk detection can identify potential risk points in the early stages of model process documentation, providing sufficient time for subsequent risk management and response. By timely discovering and addressing risk points, the compliance risk of the model throughout its life cycle can be significantly reduced.

[0122] Step S3044: Use the integrity detection result and the risk detection result as scanning results.

[0123] The integrity test results and risk test results are summarized to form a comprehensive scan result report, which can then be presented in a preset format (such as electronic reports, charts, etc.) so that relevant personnel can clearly understand the compliance status of the model process documents. The scan results will serve as the basis for subsequent decisions and actions, helping relevant personnel to formulate targeted compliance management measures.

[0124] By summarizing the integrity test results and risk test results, a comprehensive and objective comprehensive assessment of the compliance of the model process documentation can be conducted. The scanning results provide a strong basis for subsequent decisions and actions, helping relevant personnel make more informed and compliant decisions.

[0125] Exemplarily, in a feasible implementation, the process of calling the current node data processing tool to process the current node data to obtain node output data may be: Step 1, input the model process document, and extract the specific content according to the chapter directory. Step 2, integrity check, identify whether each chapter of the document actually exists to ensure the integrity of the filling. Step 3, high-risk item identification, integrate the compliance checklist into the tool, identify high-risk items and clarify them, and combine them with the approval process to remind relevant parties to pay special attention. Each question in the compliance checklist will provide multiple options, and some of the options have compliance risks. If the user selects the corresponding option, it will quickly identify and extract the detailed description filled in by the user, and place it in the approval process as a highlighted reminder.

[0126] In a feasible implementation manner, the current process node is a model monitoring node; the step S30: determining the current node data corresponding to the current process node, and calling the current node data processing tool to process the current node data to obtain node output data, comprises:

[0127] Step S305, obtaining model operation data collected by the model operation monitoring tool, and using the model operation data as current node data;

[0128] At present, in addition to model creation and deployment, modelers need to invest more time and energy to complete the actions required by model compliance management, such as: model documentation writing and management, pre-deployment review process, model effect monitoring, value review, etc. In the full life cycle management of machine learning models, model monitoring is a crucial link. The main task of the model operation monitoring tool is to collect various data of the model in real time or regularly during operation, including but not limited to the execution status of the model, data quality indicators, and the application effect of the model. Therefore, this implementation method proposes an automated monitoring tool for monitoring to improve the efficiency of full life cycle management.

[0129] When the process advances to the model monitoring node, the model operation monitoring tool monitors the model operation data for subsequent processing and analysis. By real-time monitoring of the model operation status, abnormal conditions in the model operation can be discovered and handled in a timely manner to reduce potential risks. The collected data provides a reliable analysis basis for subsequent steps and supports more accurate decision-making.

[0130] Step S306: calling the model operation alarm tool to process the current node data based on the current node data to obtain node output data.

[0131] After obtaining the model operation data, call the model operation alarm tool to process this data. The specific process can be: according to the preset alarm rules, check and analyze the current node data. Once any abnormal situation is found or the data indicator exceeds the threshold, the alarm mechanism will be triggered to generate node output data (alarm information). These alarm information usually include the type of abnormality, the specific time when the abnormality occurred, related model operation data and other information, so that relevant personnel can handle it in time.

[0132] By promptly discovering and handling abnormal situations in model operation, the stable operation of the model can be ensured. Through alarm information, relevant personnel can quickly locate problems and take targeted measures to improve the efficiency and accuracy of model operation management and reduce potential losses caused by model abnormalities.

[0133] In a feasible implementation manner, the step S306: calling the model alarm tool to process the current node data based on the current node data to obtain node output data, includes:

[0134] Step S3061, calling a model alarm tool to extract operation status data from the model operation data, and performing an operation status detection on the target model based on the operation status data to obtain an operation status detection result;

[0135] The model alarm tool will first extract data related to the target model's operating status from the model operation data, such as job scheduling status, operating time, success rate and other key indicators, and perform real-time operating status detection on the target model based on these operating status data to determine whether the model is in a normal working state.

[0136] For example, the specific operation status detection process can be: obtaining a series of preset operation status detection rules or thresholds, such as abnormal situations such as job scheduling failure and operation timeout, and comparing these rules or thresholds with the extracted operation status data to obtain the operation status detection results. By monitoring the operation status of the model in real time, potential model operation anomalies can be discovered and handled in a timely manner to ensure that the model can provide services continuously and stably.

[0137] Step S3062, calling the model alarm tool to extract model output data from the model operation data, and performing quality inspection on the target model based on the model output data to obtain a quality inspection result;

[0138] The model alarm tool extracts the model's output data from the model running data. This data usually includes the model's prediction results, classification labels, etc. Based on this output data, the target model is quality checked to evaluate the model's prediction accuracy, stability and other performance indicators.

[0139] For example, the specific process of quality inspection may be: obtaining a series of preset quality inspection rules or standards, such as evaluation indicators such as accuracy and recall. The tool will compare these rules or standards with the extracted model output data to obtain the quality inspection results.

[0140] Step S3063, calling the model alarm tool to extract model check data from the model operation data, and performing application effect detection on the target model based on the model check data to obtain a model application detection result, wherein the model check data is used to evaluate the model application effect;

[0141] The model alarm tool extracts model backtest data from the model operation data. These data are usually used to evaluate the performance of the model in actual application scenarios. Based on these backtest data, the application effect of the target model is tested to determine whether the model meets the needs and expectations of the actual application scenarios.

[0142] Exemplarily, the specific process of application effect detection can be: obtaining a series of preset application effect detection rules or standards, such as AUC (Area Under Curve, used to evaluate the performance of the classification model, the larger the value, the better the performance of the model), KS (Kolmogorov-Smirnov, Kolmogorov-Smirnov test, used to measure the model's ability to distinguish between good and bad samples) and other evaluation indicators. The tool will compare these rules or standards with the extracted model back-check data to obtain the model application detection results.

[0143] Step S3064: Make an alarm decision based on the operating status detection result, the quality detection result and the model application detection result through the model alarm tool.

[0144] The model alarm tool combines the operation status test results, quality test results and model application test results to make alarm decisions. If any test result triggers the preset alarm condition (such as abnormal status, low-quality prediction results, poor application effect, etc.), the tool will generate an alarm message and send it to relevant personnel through the predetermined alarm channel.

[0145] Exemplarily, in a feasible implementation, the process of calling the model alarm tool to process the current node data based on the current node data to obtain the node output data may be: the first step is to configure the monitoring alarm. The user fills in the model job scheduling cycle, the expected daily successful operation time point, the maximum running time of each model job and other monitoring indicators in the offline model job monitoring configuration interface, fills in the alarm notifier or group information, and completes the monitoring indicator configuration. The second step is offline job running status monitoring: the job supports the scheduled sending of running status messages during operation, and the monitoring tool periodically consumes job running status messages, including job running status, running time, etc. Online model service running status monitoring: the tool counts the return code of each request in real time and calculates the success rate; the model service request flow is calculated by three levels of statistics such as minutes, hours, and days. The third step is to compare and verify with the alarm rules. Specifically, (1) obtain the job running status. When the status is a running failure, the job scheduling failure alarm is triggered; (2) calculate the time difference between the message and the job start to determine whether it is necessary to trigger the running timeout alarm; (3) obtain the running status. When it exceeds the expected running success time point, the job running overdue alarm rule is triggered. In this implementation, the statistical results such as request success rate and traffic can be periodically compared with the rule threshold. When the alarm threshold is reached, a monitoring alarm is issued. The fourth step is the job running status alarm notification. The tool sends standardized notification information to the alarm information notifier configured in the rule. The notification form supports instant messaging messages, text messages, and emails.

[0146] It is understandable that through the alarm decision and the sending of alarm information, relevant personnel can be reminded in time to pay attention to the operation status, quality problems and application effects of the model, so as to take corresponding measures to deal with and improve them. This helps to improve the stability and accuracy of the model and ensure that the model can continue to provide high-quality services for actual application scenarios.

[0147] For example, to help understand the implementation process of the model data processing method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 4 , Figure 4 A brief process diagram of full life cycle management is provided, which specifically includes the following 9 steps, namely, requirement writing, requirement approval, model development, model testing, deployment approval, deployment verification, model monitoring, model review, and model exit. Each step completes the corresponding node operation, among which the requirement writing node outputs the "Model Requirements Document"; the requirement approval node completes the requirement approval process; the model development node completes the model development work and outputs the "Model Development Report"; the model testing node completes the model testing work and outputs the "Model Testing Report"; the deployment approval node outputs the "Model Compliance Self-Inspection Report" and "Deployment Plan" and passes the model deployment approval process; the deployment verification node completes the model deployment verification operation and outputs the "Model Deployment Report"; the model monitoring node regularly conducts model monitoring and outputs the "Model Monitoring Report"; the model review node performs model review according to the preset period, such as model review every six months or one year, and outputs the "Model Review Report"; the model exit node outputs the "Model Exit Report" and passes the model exit approval process.

[0148] In this embodiment, a standardized approval process is proposed. The standardized approval process includes the following four steps, with four types of roles involved. The standardized approval process is applicable to demand approval, deployment approval, and model exit approval steps. Figure 5 As shown, the roles in the standardized approval process are: model development manager, model development manager leader, model business manager, and model business manager leader. The steps of the standardized approval process may include: Step 1: The model development manager initiates the approval process and uploads the model document. Step 2: The model development manager leader reviews the model document and outputs the review conclusion. If passed, proceed to the next step, otherwise the process terminates. Step 3: The model business manager reviews the model document and outputs the review conclusion. If passed, proceed to the next step, otherwise the process terminates. Step 4: The model business manager leader reviews the model document and outputs the review conclusion, and the approval process ends.

[0149] This embodiment needs to implement a compliance management toolset integrated in the machine learning model platform, that is, a preset toolset, which may specifically include the following tools:

[0150] (1) Document generation tools: Document generation tools can generate documents and reports related to process nodes in the entire life cycle management process. For example, please refer to Figure 6 ,When the process node is a model development node, when the model development report is generated at the model development node, the document-assisted generation tool can implement the following steps: Step 1: The document-assisted generation tool uses external data (i.e., machine learning platform buried data) and model process documents to extract system rule information. Specifically, the document-assisted generation tool extracts data source information, data processing logic, feature engineering configuration, algorithm model hyperparameter settings, modeling environment resource specification selection, algorithm model fitting process, algorithm model interpretability, task operation log, algorithm evaluation results, etc. from the machine learning platform buried data, generates description text through template matching, and converts the evaluation data into a picture (i.e., data source information is sql information extracted from the machine learning platform buried data, extracts data table information and where conditions in sql, and queries the field information in the data table on the metadata platform by calling the interface. The data processing logic is divided into low-code scenarios and high-code scenarios. 1) Low-code scenario: ① Query platform data to obtain operator node data of low-code data processing logic; ② Convert low-code operator node information data to generate pictures; ③ Extract low-code processing logic configuration parameters from the database and put them into documents. 2) High-code scenarios: Identify through keywords or standard APIs, and use rule templates or large voice models to summarize and identify processing logic. Feature engineering configuration is also divided into low-code scenarios and high-code scenarios. 1) Low-code scenarios: Generate feature engineering configuration content by extracting feature processing operator configuration parameters and combining rule templates. 2) High-code scenarios: Identify through keywords or standard APIs, and use rule templates or large voice models to summarize and identify feature engineering. Step 2: The document-assisted generation tool uses model requirement documents and algorithm codes to promote understanding, that is, to extract background summary content of the requirement documents, key logical parameters of the algorithm code and other information, and input the extracted information into the large language model to assist in semantic understanding and key information extraction to generate text. Step 3: Combine the above two parts to output a model development report.

[0151] (2) Compliance document scanning tool, which can implement the following steps: Step 1: Input the model process document and extract the specific content according to the chapter directory. Step 2: Completeness check, identify whether each chapter of the document actually exists to ensure the completeness of the filling. Step 3: Identify high-risk items, integrate the compliance checklist into the tool, identify high-risk items and clarify them, and combine them with the approval process to remind relevant parties to pay special attention. The compliance checklist will provide multiple options for each question, and some of the options have compliance risks. If the user selects the corresponding option, it will quickly identify and extract the detailed description filled in by the user, and place it in the approval process as a highlighted reminder.

[0152] (3) Model visualization monitoring and alarm tool, which can realize three types of model indicator monitoring and alarm: model execution status, model data quality, and model application effect. Model execution status monitoring supports abnormal status monitoring and alarm such as offline model job scheduling task failure and delay, and online model service success rate, traffic and other indicator monitoring and alarm. Model data quality monitoring supports abnormal value monitoring such as null value and extreme value, and supports monitoring the degree of feature distribution change based on PSI (Population Stability Index). Model application effect monitoring supports conventional model technical indicator backchecking, such as AUC, KS, etc.

[0153] Please refer to Figure 7 The monitoring and alarm process based on the model visualization monitoring and alarm tool can be: first, after the CTM (Collaborative Traceable Management, integrated collaborative security gateway series products, job scheduling or management system) scheduling job is completed, the CTM job is put into production, and then the investment monitoring configuration is performed; the CTM job is monitored and data monitoring is configured; in the data monitoring configuration stage, the model visualization monitoring and alarm tool calls the interface to obtain the table and field information and the job information configuration and the table statistical field alarm configuration; based on the table and field information and job information, the statistical job calculation index is triggered, and the alarm event is generated based on the statistical job calculation index and the table statistical field alarm configuration; after the alarm event is generated, the model visualization monitoring and alarm tool outputs the index data and alarm record, calls the data monitoring to display the index data and alarm record, and sends the alarm information at the same time, for example, the alarm information can be sent through the Aix (aiXcoder, virtual programming expert) robot.

[0154] Please refer to Figure 8 , the specific monitoring steps can be: the first step is to monitor the alarm configuration, which specifically includes the user creating a feature library, the user configuring a registration table, the user configuring an indicator alarm, and the user marking the table statistical job mark to be turned on, that is, the user marks which tables need to perform statistical operations. The second step is to monitor the running status of offline jobs, pre-set scheduled tasks, regularly obtain the tables that need to initiate statistical jobs, and then asynchronously schedule the jobs to perform statistical calculations. After the calculation is completed, the indicator results are written to the object storage. The third step is to compare and verify with the alarm rules. The visual monitoring alarm tool pulls the indicator data from the object storage and stores it in the database. According to the stored indicators, it searches for corresponding alarm configurations. According to the threshold in the alarm configuration, it calculates whether the current indicator data reaches or exceeds the threshold (that is, Figure 8 The calculation threshold shown in ) is used to determine whether it is an abnormal event; if an abnormal event is detected, the visual monitoring alarm tool sends an alarm record and stores it in the database. The fourth step is the job running status alarm notification, calling the AIX alarm interface to send alarm information. The tool sends standardized notification information to the alarm information notifier configured in the rule. The notification form supports instant messaging, SMS, and email.

[0155] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the model data processing method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0156] The present application also provides a model data processing device, which adopts the model data processing method in the above embodiment to solve the technical problem of low efficiency of model full life cycle management. Compared with the prior art, the beneficial effects of the model data processing device provided by the present application are the same as the beneficial effects of the model data processing method provided by the above embodiment, and other technical features in the model data processing device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0157] The present application provides a model data processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the model data processing method in the above-mentioned embodiments.

[0158] Reference below Fig. 9 , which shows a schematic diagram of the structure of a model data processing device suitable for implementing the embodiment of the present application. The model data processing device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9 The model data processing device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0159] like Fig. 9 As shown, the model data processing device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the model data processing device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the model data processing device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a model data processing device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0160] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0161] The model data processing device provided by the present application adopts the model data processing method in the above embodiment, which can solve the technical problem of low efficiency in the whole life cycle management of the model. Compared with the prior art, the beneficial effects of the model data processing device provided by the present application are the same as the beneficial effects of the model data processing method provided by the above embodiment, and the other technical features in the model data processing device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0162] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0163] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0164] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the model data processing method in the above-mentioned embodiment.

[0165] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0166] The computer-readable storage medium may be included in the model data processing device; or may exist independently without being assembled into the model data processing device.

[0167] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the model data processing device, the model data processing device implements the model data processing method.

[0168] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0169] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0170] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0171] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned model data processing method, and can solve the technical problem of low efficiency in model full life cycle management. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the model data processing method provided in the above-mentioned embodiment, and will not be repeated here.

[0172] The present application also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the model data processing method as described above are implemented.

[0173] The computer program product provided by this application can solve the technical problem of low efficiency in model full life cycle management. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the model data processing method provided by the above embodiment, which will not be repeated here.

[0174] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.< / modeldocllmquerydataentity> < / catalog> < / catalog>

Claims

1. A model data processing method, characterized in that: The model data processing method comprises: Obtaining the full life cycle management process of the machine learning model, and determining the current process node where the target model is currently located from each process node of the full life cycle management process; Determine a current node data processing tool corresponding to the current process node from a preset tool set; Determine the current node data corresponding to the current process node, and call the current node data processing tool to process the current node data to obtain node output data; The node output data is output according to a preset data transmission path corresponding to the current process node.

2. The model data processing method according to claim 1, characterized in that: The current process node is a model development node; the step of determining the current node data corresponding to the current process node, and calling the current node data processing tool to process the current node data to obtain node output data includes: Obtaining machine learning platform buried data and a preset model requirement document, and using the model requirement document and the machine learning platform buried data as current node data; The document generation tool is called to process the current node data to obtain node output data.

3. The model data processing method according to claim 2, characterized in that: The document generation tool includes a template matching module and a large language model; The step of calling the document generation tool to process the current node data to obtain node output data includes: Calling the template matching module to determine the target buried point data corresponding to the current process node from the buried point data of the machine learning platform; Filling the non-evaluation data in the target buried point data into a preset text template to obtain a description text through the template matching module, and converting the evaluation data in the target buried point data into an evaluation picture through the template matching module; Inputting the model requirement document into the large language model to obtain semantic text; The semantic text, the description text and the evaluation picture are used as node output data.

4. The model data processing method according to claim 1, characterized in that: The current process node is a deployment approval node; the step of determining the current node data corresponding to the current process node, and calling the current node data processing tool to process the current node data to obtain node output data includes: Obtain a model process document, and use the model process document as current node data, wherein the model process document is a document recording the target model development process and test process; A document scanning tool is called to process the current node data to obtain a scanning result, and node output data is generated based on the scanning result.

5. The model data processing method according to claim 4, characterized in that: The step of calling a document scanning tool to process the current node data to obtain a scanning result includes: Calling a document scanning tool to extract chapter contents according to each chapter of the model process document; Performing integrity check on the model process document based on the chapter content to obtain an integrity check result; Performing risk detection on the model process document based on the chapter content to obtain a risk detection result; The integrity detection result and the risk detection result are used as scanning results.

6. The model data processing method according to any one of claims 1 to 5, characterized in that: The current process node is a model monitoring node; the step of determining the current node data corresponding to the current process node, and calling the current node data processing tool to process the current node data to obtain node output data includes: Acquire model operation data collected by a model operation monitoring tool, and use the model operation data as current node data; The model operation alarm tool is called to process the current node data based on the current node data to obtain node output data.

7. The model data processing method according to claim 6, characterized in that: The step of calling the model alarm tool to process the current node data based on the current node data to obtain node output data includes: Calling a model alarm tool to extract operation status data from the model operation data, and performing an operation status detection on the target model based on the operation status data to obtain an operation status detection result; Calling the model alarm tool to extract model output data from the model operation data, and performing quality inspection on the target model based on the model output data to obtain a quality inspection result; Calling the model alarm tool to extract model check data from the model operation data, and performing application effect detection on the target model based on the model check data to obtain a model application detection result, wherein the model check data is used to evaluate the model application effect; The model alarm tool makes an alarm decision based on the operating status detection result, the quality detection result and the model application detection result.

8. A model data processing device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the model data processing method according to any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the model data processing method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the model data processing method according to any one of claims 1 to 7 are implemented.

Images