Multi-data source modeling method, system and device, storage medium and program product

Abstract

The specification provides a multi-data-source modeling method, system, device, storage medium and program product. The multi-data-source modeling system is composed of a user terminal, an agent platform and a plurality of data source nodes. The user terminal sends a modeling requirement in natural language, uploads a sample identifier and a supervision label of a sample to be modeled. The agent platform selects at least two data source nodes participating in modeling and respective target feature fields according to the modeling requirement and feature metadata obtained from the data source nodes, and then sends the sample identifier and the target feature fields to the nodes. The data source nodes construct the sample to be modeled according to the sample identifier, the target feature field and a local feature database, train a prediction model to be trained and generate an intermediate prediction result, and send the intermediate prediction result to the agent platform. The agent platform fuses the intermediate result to generate a final prediction result, generates first optimization information with the objective of minimizing the error of the final prediction result and the supervision label, and returns the first optimization information, and the data source nodes optimize the prediction model parameters according to the first optimization information.

Description

Technical Field

[0001] This specification relates to the field of model building technology, and more particularly to a multi-data source modeling method, system, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the rapid development of big data and artificial intelligence technologies, the explosive growth in data scale and dimensions has driven the upgrading of modeling needs, but the limitations of traditional modeling models are becoming increasingly prominent.

[0003] At the data level, existing modeling methods mostly rely on a single data source, which limits the data scope of a single entity, resulting in limited data coverage and insufficient sample diversity. This makes it difficult to meet the requirements for model generalization ability and prediction accuracy in complex scenarios. At the same time, due to privacy protection regulations and data security considerations, institutions generally hold a cautious attitude towards sharing raw data, further exacerbating the limitations of modeling data.

[0004] At the modeling operation level, the existing modeling process requires high professional technical skills from users. Many steps in the model training process often need to be completed manually by professionals. The operation process is cumbersome, time-consuming, and easily affected by human error, resulting in low execution efficiency and flexibility. Summary of the Invention

[0005] In view of the above, one or more embodiments of this specification provide the following technical solutions:

[0006] According to a first aspect of one or more embodiments of this specification, a multi-data source modeling system is proposed, including a user terminal, an intelligent agent platform, and multiple data source nodes;

[0007] The user terminal is used to send modeling requirement information described in natural language, sample identifiers of the samples to be modeled, and supervision labels to the intelligent agent platform;

[0008] The intelligent agent platform is used to determine at least two data source nodes to participate in the modeling task and their respective target feature fields based on the modeling requirement information and feature metadata obtained from the data source nodes, and to send the sample identifier and the target feature fields to the at least two data source nodes.

[0009] The data source node is used to construct a sample to be modeled corresponding to the sample identifier based on the sample identifier, the target feature field and the local feature database, and to train the prediction model to be trained using the sample to be modeled, generate intermediate prediction results and send them to the intelligent agent platform.

[0010] The intelligent agent platform is also used to fuse the intermediate prediction results to generate a final prediction result, and generate first optimization information and send it to the data source node with the optimization objective of minimizing the error between the final prediction result and the supervision label;

[0011] The data source node is also used to optimize the parameters of the prediction model based on the first optimization information.

[0012] According to a second aspect of one or more embodiments of this specification, a multi-data source modeling method is proposed, applied to an intelligent agent platform, the method comprising:

[0013] Receive modeling requirements information described in natural language, sample identifiers of the samples to be modeled, and supervision labels sent by the user client;

[0014] Based on the modeling requirement information and the feature metadata obtained from the data source nodes, at least two data source nodes to participate in the modeling task and their respective target feature fields are determined, and the sample identifier and the target feature fields are sent to the at least two data source nodes so that the data source nodes can construct the sample to be modeled corresponding to the sample identifier and perform model training.

[0015] Receive intermediate prediction results sent by the data source node during model training;

[0016] The intermediate prediction results are fused to generate a final prediction result. With the goal of minimizing the error between the final prediction result and the supervision label, first optimization information is generated and sent to the data source node so that the data source node optimizes the model parameters based on the first optimization information.

[0017] According to a third aspect of one or more embodiments of this specification, a multi-data source modeling method is proposed, applied to a data source node, the method comprising:

[0018] Receive sample identifiers and target feature fields sent by the intelligent agent platform;

[0019] Based on the sample identifier, the target feature field, and the local feature database, construct the sample to be modeled corresponding to the sample identifier;

[0020] The prediction model to be trained is trained using the sample to be modeled, intermediate prediction results are generated and sent to the intelligent agent platform;

[0021] The system receives first optimization information returned by the intelligent agent platform based on the intermediate prediction results, and optimizes the parameters of the prediction model based on the first optimization information.

[0022] According to a fourth aspect of the embodiments of this specification, an electronic device is provided, comprising:

[0023] processor;

[0024] Memory used to store processor-executable instructions;

[0025] Wherein, when the processor executes the executable instructions, it is used to implement the method described in the second aspect or the third aspect.

[0026] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in the second or third aspect.

[0027] According to a sixth aspect of the embodiments of this specification, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in the second or third aspect.

[0028] As demonstrated by the above embodiments, the process of transmitting modeling requirements is simplified through natural language interaction on the user end, lowering the operational threshold for users. The intelligent agent platform can analyze the modeling requirement information and feature metadata obtained from data source nodes to determine at least two data source nodes and target feature fields that meet the modeling requirements. This enables collaborative modeling across data sources, overcoming the limitations of a single data source and improving the breadth and dimensionality of data for model training. Each data source node completes sample construction and model training based on local data and only feeds back intermediate prediction results, ensuring data privacy and security. Furthermore, through result fusion and parameter optimization by the intelligent agent platform, it promotes collaborative iteration of prediction models in each data source node, ultimately improving the accuracy and generalization ability of the prediction model. This fully automates the entire process from natural language interaction to multi-data source modeling, offering advantages such as high efficiency, flexibility, security, and ease of use. It helps to unlock the value of multi-party data, improve modeling quality, and enhance service response speed.

[0029] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the structure of a multi-data source modeling system provided in an exemplary embodiment.

[0031] Figure 2 This is a schematic diagram of three intelligent agents included in an intelligent agent platform provided in an exemplary embodiment.

[0032] Figure 3 This is a schematic diagram illustrating the interaction between an intelligent agent platform and a data source node during the modeling process, provided in an exemplary embodiment.

[0033] Figure 4A This is a schematic diagram of the structure of another multi-data source modeling system provided in an exemplary embodiment.

[0034] Figure 4B This is a schematic diagram of the structure of another multi-data source modeling system provided in an exemplary embodiment.

[0035] Figure 5 This is a temporal interaction diagram of a user-uploaded sample identifier provided in an exemplary embodiment.

[0036] Figure 6 This is a temporal interaction diagram provided by an exemplary embodiment, showing the user client submitting a modeling request.

[0037] Figure 7 This is an exemplary embodiment that provides a time-series interaction diagram for user-side uploading of modeling files and supervision label files to achieve multi-data source modeling.

[0038] Figure 8 This is a flowchart of a multi-data source modeling method provided in an exemplary embodiment.

[0039] Figure 9 This is a flowchart of another multi-data source modeling method provided in an exemplary embodiment.

[0040] Figure 10 This is a schematic diagram of the structure of a device provided in an exemplary embodiment. Detailed Implementation

[0041] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0042] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of related data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.

[0043] Please see Figure 1This specification provides a multi-data source modeling system, including a user terminal 10, an intelligent agent platform 20, and multiple data source nodes 30.

[0044] Each data source node 30 includes a local feature database, which stores feature tuples corresponding to user identifiers. These tuples typically contain user behavior, attribute information, and other relevant information collected and accumulated by each data source node 30 during actual service provision or operation. The local feature databases held by different data source nodes 30 may contain different dimensions of data corresponding to the same user identifier, or they may contain only user data unique to their service scope, depending on whether the user has used the products or services provided by that node.

[0045] Therefore, in real-world scenarios, the same user identifier may be recorded in multiple data source nodes 30, and the recorded feature information may complement and corroborate each other, forming a cross-domain, multi-dimensional dataset. This provides a data foundation and feasibility for subsequent cross-node feature alignment, collaborative training, and model fusion based on sample identifiers (such as user identifiers). Simultaneously, the local feature database is only stored and used within the data source node 30, which helps ensure the security and privacy of user data and avoids the potential risks associated with directly sharing raw data across nodes.

[0046] The following is an illustrative example of the multi-data source modeling process:

[0047] The user terminal 10 is used to send modeling requirement information described in natural language, sample identifiers of the samples to be modeled, and supervision labels to the intelligent agent platform 20.

[0048] For example, the user terminal 10 may be held by an institution or research institution that needs to use multi-source data to carry out modeling analysis. Users can make modeling requests to the intelligent agent platform 20 through the user terminal 10, avoiding the tedious operation of configuring complex modeling parameters themselves.

[0049] Among them, the modeling requirements information is used to express the user's description of the model objectives, feature range, output requirements, etc., and can be directly input using natural language to improve the convenience and ease of use of interaction.

[0050] The sample identifiers and supervision labels of the samples to be modeled can be uploaded simultaneously with the modeling requirements information, or they can be uploaded in batches at different stages. For example, before the specific modeling goal is clear, the user can first submit the relevant sample identifiers to the intelligent agent platform 20 for preprocessing and saving. Later, after the modeling requirements are clarified, the user can upload the specific modeling requirements information and corresponding supervision labels for label comparison and effect verification during model training.

[0051] It should be noted that the user terminal 10 only provides sample identifiers and supervision labels, and does not directly upload the original sample data containing feature values. The input samples used for model training are extracted by each data source node 30 based on the local feature database and sample identifiers, thereby effectively avoiding cross-node transmission of original data and improving the level of data security and privacy protection.

[0052] The intelligent agent platform 20 is used to determine at least two data source nodes 30 to participate in the modeling task and their respective target feature fields based on modeling requirement information and feature metadata obtained from data source nodes 30, and to send the sample identifier and target feature fields to at least two data source nodes 30.

[0053] For example, the intelligent agent platform 20 can automate the entire process from multi-data source access to model generation through the division of labor and coordination among multiple intelligent agents, thereby improving modeling efficiency, lowering the technical threshold, and enhancing the system's flexibility and scalability. Please refer to... Figure 2 The intelligent agent platform 20 includes, but is not limited to, a feature engineering intelligent agent 21, a model building intelligent agent 22, and a model evaluation intelligent agent 23. The feature engineering intelligent agent 21 is used to perform tasks such as feature selection and determining feature processing strategies; the model building intelligent agent 22 is used to conduct model training and optimize model parameters; and the model evaluation intelligent agent 23 is used to perform security verification, correctness verification, and verification result analysis of the model.

[0054] The feature engineering agent 21 can receive sample identifiers of a batch of samples to be modeled uploaded by the user terminal 10, and organize, deduplicate, format, and verify the integrity of these sample identifiers to ensure that the sample identifiers are unique, non-duplicate, and complete. Based on this, the feature engineering agent 21 can automatically execute multiple preprocessing tasks related to the sample identifiers by calling preset functions or connecting to internal / external APIs (Application Programming Interfaces), such as grouping, sharding, batch management, label relationship matching, index generation, standardization recording, and status tracking of the sample identifiers. The above processing can be automatically coordinated and called by the feature engineering agent 21 according to the modeling requirements, without requiring manual operation by the user. For the processed sample identifiers and their corresponding batch, grouping, or status information, the feature engineering agent 21 can also perform unified storage and version management, facilitating efficient alignment and calling across multiple data source nodes 30, and supporting later backtracking, appending, or multiple iterations.

[0055] After receiving modeling requirement information described in natural language, the feature engineering agent 21 can determine at least two data source nodes 30 to participate in the modeling task and their respective target feature fields based on the modeling requirement information and the feature metadata obtained from the data source node 30.

[0056] For example, the feature engineering agent 21 can obtain feature metadata from each data source node 30. This metadata describes the feature fields and their attribute information held by each data source node 30, such as feature name, field type, value range, update frequency, and availability status. After obtaining the feature metadata from each data source node 30, the feature engineering agent 21 can combine the modeling requirement information uploaded by the user terminal 10 to generate modeling prompts (such as required feature categories, modeling objectives, and preferred scenario constraints). These prompts are then input into a preset language model. Utilizing the language model's contextual understanding and semantic reasoning capabilities, multiple candidate data source nodes 30 that meet the modeling requirements and candidate feature fields suitable for modeling within each candidate data source node 30 are automatically selected, forming a candidate feature set. Subsequently, the feature engineering agent 21 can further determine at least two data source nodes 30 that will ultimately participate in the modeling task and the target feature fields for each node from the candidate feature set. The sample identifier and target feature fields are then sent to the at least two data source nodes 30 that will participate in the modeling task.

[0057] Through the above approach, the feature engineering agent 21 fully leverages the language model's capabilities in natural language understanding and knowledge matching. This enables users to quickly transform service-oriented modeling needs into executable feature selection solutions for multiple data sources, reducing the manual costs and technical barriers of feature screening and matching. Simultaneously, by combining feature metadata from multiple data source nodes 30 with semantic reasoning, it can more flexibly discover potentially usable cross-domain features, enhancing the richness and diversity of input features. This contributes to improving the accuracy and generalization ability of the final model, maximizing the utilization of multi-source data value.

[0058] In one possible implementation, after obtaining the candidate feature set, the feature engineering agent 21 can send the candidate feature set to the user terminal 10. The user terminal 10 is also used to display the candidate feature set, allowing users to select and combine different data source nodes 30 and their candidate feature fields based on their own understanding and specific needs. The user terminal 10 is also used to send at least two data source nodes 30 selected by the user for the modeling task and their respective target feature fields to the feature engineering agent 21 for use in subsequent sample extraction and modeling processes. In this way, the user's subjective judgment ability in understanding the actual service scenario and data usage preferences can be fully utilized, making the finally determined feature fields and data source nodes 30 more in line with the actual service goals and context requirements, thereby improving the usability and interpretability of the model.

[0059] In another possible implementation, after obtaining the candidate feature set, the feature engineering agent 21 can perform automated quality analysis and selection on the candidate data source nodes 30 and their candidate feature fields based on preset automated evaluation logic. For example, it can rank or score each candidate based on multi-dimensional indicators such as feature coverage, availability, relevance to the modeling target, redundancy, and compliance, thereby automatically determining at least two data source nodes 30 and their respective target feature fields for modeling, and directly proceeding to the subsequent sample set construction and model training stages. This approach significantly improves the automation and execution efficiency of the modeling process, and is particularly suitable for scenarios with high requirements for data analysis expertise or those requiring rapid delivery.

[0060] After receiving the sample identifier and target feature field from the feature engineering agent 21, the data source node 30 can automatically construct the sample to be modeled corresponding to the sample identifier based on the sample identifier, the target feature field, and its local feature database. Specifically, the feature database maintained locally by each data source node 30 is used to store feature tuples associated with the sample identifier, and these feature tuples contain the values ​​of different feature fields. When the data source node 30 receives the specified target feature field, it can automatically query and extract the corresponding target feature field value from the local database for each sample identifier, and then organize these field values ​​into the sample to be modeled according to a predetermined format.

[0061] In this way, data source node 30 can dynamically extract feature field values ​​matching the sample identifier on demand without exposing the original complete database, generating sample data containing only the information needed for modeling. This process is completed independently locally by each data source node 30, without transmitting the complete original data externally, effectively meeting the requirements of minimizing data usage and protecting privacy in multi-agent scenarios. In addition, the ability to automatically construct samples can significantly reduce manual docking and data preparation work, improve the flexibility and operability of calling multi-source data, ensure data consistency and context alignment in the subsequent multi-node model training stage, and provide a reliable data foundation for multi-source collaborative modeling.

[0062] In an optional embodiment, when generating candidate feature sets, the language model can also combine user-inputted modeling requirements information and feature metadata to generate corresponding feature processing strategies for candidate feature fields in each candidate data source node 30. Here, "feature processing strategy" can be understood as a preprocessing, transformation, or normalization method that needs to be performed on a specific feature field before it participates in modeling, in order to improve the consistency, usability, and adaptability of features to the modeling objective.

[0063] For example, feature processing strategies may include, but are not limited to: performing missing value imputation, outlier correction, standardization, or normalization on numerical features; performing binning and encoding (such as one-hot encoding or label encoding) on ​​categorical features; performing time series segmentation or deriving features on time fields; and performing desensitization or aggregation on certain sensitive fields. These strategies can be automatically generated by the language model based on existing scene knowledge bases and semantic reasoning capabilities, and matched with contextual requirements for output.

[0064] In this scheme, the feature engineering agent 21, in addition to sending the determined target feature fields and sample identifiers to at least two data source nodes 30 participating in the modeling task, also sends the feature processing strategy for the target feature fields to the relevant data source nodes 30. Upon receiving this information, each data source node 30, based on the target feature fields and sample identifiers, queries the feature database for the target feature field value corresponding to the sample identifier, and processes the target feature field value based on the feature processing strategy (e.g., data preprocessing or transformation operations) to generate the sample to be modeled corresponding to the sample identifier.

[0065] This approach significantly improves the consistency and automation of feature processing strategies across multiple subjects and data sources, reducing model bias and performance instability caused by inconsistent feature processing standards across different data source nodes. Simultaneously, language model generation and automatic agent scheduling effectively reduce the complexity of manual intervention and parameter tuning, accelerating the preparation efficiency for multi-source data modeling. Furthermore, each data source node completes feature processing locally, avoiding the external transfer of raw data, further strengthening data privacy protection and compliant use, and promoting the sustainability and availability of multi-party collaboration.

[0066] After each data source node 30 generates the sample to be modeled, it can send a sample generation confirmation message to the agent platform 20 to indicate that it has completed sample preparation and that the sample is available for subsequent model training. Upon receiving all the sample generation confirmation messages, the model building agent 22 in the agent platform 20 can trigger the subsequent modeling process, specifically including obtaining the modeling file and sending it to at least two data source nodes 30 participating in the modeling task. The modeling file in this embodiment can be understood as a technical document describing or encapsulating key configuration information such as algorithm parameters, model structure, training process configuration, and input / output format definitions required for a specific modeling task. Its core function is to instruct each data source node 30 how to perform model initialization, parameter setting, training strategy, and output result formatting based on the locally generated sample to be modeled. It serves as a unified instruction set and technical basis for multi-node distributed collaborative training.

[0067] In one possible implementation, the modeling file can be directly generated by the user terminal 10 or selected and then uploaded to the intelligent agent platform 20. For example, the model building intelligent agent 22 can send back confirmation information about sample generation from each data source node 30 to the user terminal 10, prompting the user to upload the corresponding modeling file. For instance, based on their industry experience or internal requirements, the user can configure the modeling algorithm, parameter templates, etc., locally, and submit the complete modeling file to the intelligent agent platform 20 for distribution after confirming the sample preparation. This mode allows users to retain greater control over the algorithms and modeling details used, facilitating the fulfillment of personalized scenarios or compliance requirements, and enhancing user participation and customization capabilities in the modeling process.

[0068] In another possible implementation, the model-building agent 22 can input modeling requirements information and model generation templates into a language model. The language model then analyzes the modeling requirements information to determine the appropriate modeling algorithm (such as classification, regression, clustering, etc.) and generates modeling files based on the modeling algorithm and model generation template. This approach, through natural language understanding and intelligent generation, significantly reduces the user's reliance on technical details such as modeling algorithms and scripts. It helps non-technical users to quickly initiate multi-data source modeling with "zero barriers," greatly improving modeling efficiency. Furthermore, it is highly adaptable and facilitates flexible iteration.

[0069] In another possible implementation, the model-building agent 22 is also used to input modeling requirement information into a language model, which then determines the appropriate modeling algorithm by analyzing the information. However, unlike other implementations, the model-building agent 22 can directly retrieve ready-made modeling files corresponding to the modeling algorithm from a pre-maintained modeling file library. This library stores modeling files corresponding to different modeling algorithms for quick access in various scenarios. This approach, while ensuring algorithm compatibility, significantly reduces the time overhead of repeated generation and debugging through "ready-to-use" functionality. It is suitable for common or standardized modeling tasks, enabling batch, efficient, and automatic distribution and execution, further improving the system's modeling response speed and reusability.

[0070] After receiving the modeling file sent by the model building agent 22, each data source node 30 participating in the modeling task can initialize based on the configuration information such as model structure, training strategy, and hyperparameters contained in the modeling file, thereby obtaining the prediction model to be trained. This initialization process ensures that different data source nodes 30 can execute a consistent training process under the same modeling framework and configuration while maintaining their respective data privacy isolation, thereby ensuring the fusion and consistency of multi-source collaborative modeling.

[0071] Then please see Figure 3Each data source node 30 uses the generated samples to be modeled to train the locally initialized prediction model and obtain local intermediate prediction results. The intermediate prediction results may include model outputs corresponding to sample identifiers, such as prediction scores and class probabilities, and can be sent to the agent platform 20 without disclosing the original input data.

[0072] After receiving the intermediate prediction results returned by all data source nodes 30 participating in the modeling task, the model building agent 22 in the agent platform 20 can fuse the intermediate prediction results to generate a result that covers the final prediction result. For example, if there are three data source nodes 30 participating in the modeling, data source node 1 sends intermediate prediction result 1 for sample identifier "001" to the agent platform 20, data source node 2 sends intermediate prediction result 2 for sample identifier "001" to the agent platform 20, and data source node 3 sends intermediate prediction result 3 for sample identifier "001" to the agent platform 20; the model building agent 22 fuses intermediate prediction result 1, intermediate prediction result 2, and intermediate prediction result 3 to obtain the final prediction result corresponding to sample identifier "001".

[0073] The model-building agent 22 further compares and analyzes the final prediction result with the supervision labels uploaded by the user terminal 10, aiming to minimize the error between the final prediction result and the true label under the same sample identifier. It then generates first optimization information and feeds it back to the data source node 30. This first optimization information may include gradient information for model parameters, update strategies, or other optimization instructions that each data source node 30 can execute for local model fine-tuning.

[0074] After receiving the first optimization information, each data source node 30 optimizes the parameters of the prediction model based on the first optimization information. For example, it can incrementally update or fine-tune the parameters of the local prediction model based on the first optimization information, thereby improving the model's fitting ability.

[0075] On the one hand, the training process is ensured to be executed locally on each data source node 30, avoiding centralized transmission and exposure of raw feature data, and meeting high requirements for data security and privacy compliance. On the other hand, through the unified fusion of intermediate prediction results and global optimization based on supervised labels by the model building agent 22, the entire system can achieve distributed collaborative learning in multi-source heterogeneous data scenarios, significantly improving the model's generalization ability and prediction accuracy. In addition, the generation and cyclic distribution of the first optimization information enable the model training to be iteratively updated in multiple rounds based on joint feedback across nodes, realizing federated incremental optimization of model parameters across multiple data sources, further enhancing the system's sustainable learning ability and adaptability.

[0076] The following is an illustrative example of the process by which the model-building agent 22 fuses intermediate prediction results:

[0077] In one possible implementation, the model building agent 22 can fuse the intermediate prediction results returned by each data source node 30 based on a preset fusion rule to generate a final prediction result covering the global sample identifier. The preset fusion rule may include, but is not limited to: (1) weighted average, setting different weights according to the reliability or sample coverage of each data source node 30 model to achieve differentiated aggregation of prediction results; (2) voting mechanism, determining the final prediction category through majority voting or confidence voting in classification scenarios; (3) distributed aggregation, adopting a decentralized aggregation strategy for regression scenarios or multi-task scenarios to further enhance privacy and security.

[0078] This approach is simple, flexible, and can be quickly integrated. It is suitable for multi-data source joint modeling scenarios with high requirements for real-time performance or scalability, and ensures the controllability and interpretability of the model fusion process.

[0079] In another possible implementation, the model-building agent 22 can input all intermediate prediction results into a predefined fusion model to be trained. This fusion model then performs a deep learning-level nonlinear combination of the intermediate prediction results from multiple sources to generate a better final prediction result. The model-building agent 22 aims to minimize the error between the final prediction result and the supervision label, generating the aforementioned first optimization information and second optimization information for the fusion model, and using the second optimization information to optimize the parameters of the fusion model.

[0080] This approach can fully explore the complementarity and nonlinear relationships between different data source nodes 30 models, and is suitable for multi-stage modeling tasks with high fusion accuracy requirements in complex scenarios. It can significantly improve the prediction performance and generalization of the system.

[0081] In another possible implementation, the model-building agent 22 can input all intermediate prediction results into a language model (e.g., a large model or a dedicated generative inference model). The language model then adaptively parses, weights, and performs semantic fusion on the intermediate prediction results, automatically outputting the final prediction result. This approach, based on the contextual understanding and generative inference capabilities of the language model, allows for flexible combination of multi-source results even in the absence of explicit fusion rules or the inability to predefine a fixed model structure. Simultaneously, the language model can automatically generate explanatory analysis reports of the prediction results, improving the interpretability of the results and reducing subsequent manual analysis costs. This approach is suitable for scenarios with high requirements for flexibility and interpretability.

[0082] In some embodiments, after the prediction model in the data source node 30 has been trained, the model evaluation agent 23 in the agent platform 20 can perform the subsequent evaluation process. For example, the model evaluation agent 23 can perform security checks and / or correctness verifications on the trained prediction models in each data source node 30, and analyze the verification results to generate a performance evaluation report.

[0083] By introducing a model evaluation agent 23, the system can not only perform multi-dimensional security and accuracy control on the prediction model after multi-data source collaborative training, significantly reducing potential risks caused by model leakage of sensitive information or structural defects, but also help to promptly identify problems such as model accuracy deviations and insufficient generalization ability. Furthermore, the performance evaluation report can serve as an important basis for subsequent model parameter tuning, retraining, or fusion strategy optimization, forming an automated process from training to evaluation and then to closed-loop optimization.

[0084] The security verification includes at least one of the following:

[0085] (1) Sensitivity check of model parameters or output content: The weight parameters of the prediction model, the output of each intermediate layer during training and the final prediction output can be automatically scanned to detect whether there is a potential risk of leakage of user identity information, original input values ​​or other preset sensitive data, so as to avoid the leakage of sensitive information when it is transferred or output across nodes.

[0086] (2) Model structure security check: The network structure, external modules that can be called, and script logic of the prediction model can be analyzed to prevent security vulnerabilities caused by the introduction of third-party components or unexpected logic.

[0087] (3) Data transmission link security check during training and inference phase: used to verify whether the data interaction between the data source node 30 and each agent during training or inference complies with security requirements such as encrypted transmission and minimum available access permissions, to prevent data from being stolen or tampered with during transmission.

[0088] Correctness verification includes at least one of the following:

[0089] (1) Cross-validation: On the existing training set, the generalization ability of the model is quantified by dividing the training data, training and validation in multiple rounds, and possible overfitting problems are discovered.

[0090] (2) Accuracy index calculation: It can calculate various accuracy or robustness indices such as precision, recall, F1 score, and AUC (Area Under the Curve) to quantify the predictive performance of the model under the target task.

[0091] (3) Result interpretability verification: Combine the modeling requirements information input by the user (such as service scenario and expected interpretation granularity) to verify whether the prediction results conform to the service logic and whether they can be reasonably explained by interpretability methods (such as feature importance ranking, Shapley value, etc.) to improve the understandability of the results and user trust.

[0092] In one possible implementation, the model evaluation agent 23 is specifically used to input the verification results and a preset report template into the language model, so that the language model can analyze the verification results and generate a performance evaluation report in combination with the report template.

[0093] For example, the model evaluation agent 23 can input various verification results and preset report templates into the language model. The language model can combine the preset report templates, analysis rules, and contextual prompts to summarize, conclude, and explain the original verification results of security verification and correctness verification. For example: (1) For the security verification part, the language model can automatically analyze the detected risks of sensitive information leakage, structural security risks, or data transmission vulnerabilities, and output the risk level, possible causes, and suggested remedial measures in more readable natural language; (2) For the correctness verification part, the language model can compare various accuracy indicators, cross-validation results, etc., summarize the advantages and disadvantages of the model, and provide targeted improvement directions (such as recommending better hyperparameter combinations, supplementing training samples, replacing modeling algorithms, etc.); (3) For interpretability verification, the language model can automatically generate visual prompts or interpretable summaries based on the verification results, making it easier for users to understand why the model makes the current prediction.

[0094] By inputting the validation results into the language model to automatically generate performance evaluation reports, performance evaluation reports are automatically generated. This significantly reduces the time and expertise required for manual analysis of validation results, enabling non-technical personnel to clearly understand model quality. Furthermore, the language model can automatically identify outliers, potential risks, and optimization opportunities based on context, providing actionable improvement suggestions for subsequent model tuning and retraining. Thus, with the support of an integrated language model, the multi-data source modeling system can achieve an automated closed loop from model validation to report generation. While ensuring model security and correctness, it further enhances the ease of use and flexibility of external output, fully leveraging the intelligent and automated advantages of the Agent Platform 20.

[0095] In some embodiments, please refer to Figure 4A The multi-source modeling system also includes a multi-source engine 40, which serves as the central coordinating and scheduling hub for multiple data source nodes 30. It is a crucial central component for achieving efficient collaboration and data linkage between the various data source nodes 30 and the intelligent agent platform 20 within the entire system. The intelligent agent platform 20 interacts with each data source node 30 through the multi-source engine 40.

[0096] For example, the multi-source engine 40 may perform at least one of the following functions:

[0097] (1) Multi-source overall scheduling: The multi-source engine 40 can connect to multiple distributed and heterogeneous data source nodes 30, centrally manage the connection information, access status and access policies of each data source node 30, realize unified orchestration and dynamic scheduling of distributed data source nodes 30, and ensure efficient collaboration of different nodes in sample retrieval, feature query and model training.

[0098] (2) Enhanced multi-domain indicator set capability: Multi-source engine 40 supports the integration and maintenance of cross-domain indicator system. Users or intelligent agents can obtain feature metadata from different fields such as finance, medical care, retail, and government affairs based on multi-source engine 40, which facilitates multi-dimensional modeling and feature expansion, and improves the coverage and interpretability of the model.

[0099] (3) Centralized sample management: The multi-source engine 40 can be used as an index or management unit of the centralized sample repository. It can uniformly maintain the uploaded sample identifiers, supervision tags and their correspondence with each data source node 30, support sample version management, access control and cross-node sample consistency verification, and improve sample scheduling efficiency and security.

[0100] (4) Task management: The multi-source engine 40 can schedule and track the modeling tasks, feature query tasks and model optimization tasks initiated by multiple agents throughout the process, and supports task status monitoring, task result archiving and abnormal alarms to ensure the orderly process when multiple tasks are in parallel.

[0101] (5) Model backtracking: Multi-source engine 40 supports the full-process backtracking capability of model training, parameter tuning, result fusion and evaluation. It can automatically record the key operations and generated intermediate / final files in each modeling process, which is convenient for later review, traceability and compliance audit.

[0102] (6) One-click modeling: The multi-source engine 40 can be integrated with the intelligent agent platform 20 to form an "automated modeling pipeline", supporting an automated closed loop from receiving modeling requirements to sample scheduling, feature processing, model training, result fusion and verification, and enabling users to initiate modeling requirements with "low threshold and one-click".

[0103] (7) Model call and configuration management: Multi-source engine 40 can also provide unified call and configuration management services for pre-trained models, supporting model version control, permission management and visual call configuration, which facilitates rapid reuse in different service scenarios.

[0104] (8) Multi-source fusion strategy management: Multi-source engine 40 can also centrally define and manage the fusion method of samples and features from different data sources in the modeling process. For example, it supports configuration of feature alignment rules, missing value handling strategies, prediction result weighting or voting mechanisms, etc., to improve the flexibility and stability of model fusion.

[0105] By introducing a multi-source engine40 into the multi-source modeling system, on the one hand, efficient and unified management and secure scheduling of multiple distributed and heterogeneous data sources are achieved, significantly reducing the complexity of multi-source data access and scheduling; on the other hand, centralized management of samples, tasks, models, and fusion strategies can avoid the consistency and traceability risks brought about by decentralized operations.

[0106] In some embodiments, please refer to Figure 4B The multi-source modeling system also includes a database 50, which interfaces with the multi-source engine 40 and serves as one of the core supports for multi-source data scheduling and task management.

[0107] The database 50 can be used to centrally store various file data uploaded by the user terminal 10, such as sample identification files, supervision label files, and modeling files submitted by the user terminal 10 through the intelligent agent platform 20. These files are the core inputs necessary for multi-data source modeling tasks, and the centralized storage of the database 50 helps ensure data consistency, integrity, and efficiency of subsequent access. In addition, the database 50 can also be used to store auxiliary information related to multiple data source nodes 30, such as feature metadata reported by each data source node 30. By establishing a stable interface mechanism between the multi-source engine 40 and the database 50, the required sample identification, label files, modeling files, and feature metadata can be flexibly read or updated at different task stages, and file version backtracking, tracking, and management can be supported, thereby improving the data availability, security, and controllability of the entire multi-data source modeling process and reducing the additional overhead caused by repeated uploads and transmissions.

[0108] In one exemplary embodiment, please refer to Figure 5 This shows the timing interaction diagram of the user first uploading the sample identifier.

[0109] In step 501, the user client uploads the sample identification file to be modeled to the intelligent agent platform through interaction with the intelligent agent platform.

[0110] In step 502, after receiving the sample identifier file, the intelligent agent platform can perform preprocessing operations on the file content. Preprocessing includes, but is not limited to, organizing, deduplicating, formatting, and verifying the integrity of the sample identifier content. For example, the feature engineering intelligent agent can automatically execute multiple preprocessing tasks related to sample identifiers by calling preset functions or connecting to internal / external APIs. These tasks include grouping, slicing, batch management, label matching, index generation, standardization recording, and state tracking of sample identifiers. The preprocessing process can be automatically completed by the feature engineering intelligent agent based on modeling requirements, without requiring manual intervention from the user.

[0111] In step 503, the agent platform transmits the preprocessed sample identification file to the multi-source engine.

[0112] In step 504, the multi-source engine stores the received preprocessed sample identification file in the database to facilitate unified sample management and task scheduling across data source nodes.

[0113] In step 505, the multi-source engine returns confirmation information to the agent platform that the sample identification file has been successfully stored.

[0114] In step 506, the intelligent agent platform sends a "sample identification file uploaded successfully" response to the user, so that the user can confirm that the sample has been received by the system and can proceed to the subsequent processing.

[0115] In one exemplary embodiment, please refer to Figure 6 This shows the timing interaction diagram of the user's modeling request.

[0116] In step 601, the user sends modeling requirement information described in natural language to the intelligent agent platform, so that the user can make personalized modeling requests in an intuitive way.

[0117] In step 602, the intelligent agent platform generates a candidate feature set based on the modeling requirement information and combined with the feature metadata obtained from each data source node. The candidate feature set contains multiple candidate data source nodes that meet the modeling requirements and candidate feature fields suitable for modeling within each node.

[0118] In step 603, the intelligent agent platform feeds back the generated candidate feature set to the user terminal, allowing the user to flexibly choose from multiple options.

[0119] In step 604, the user terminal selects at least two data source nodes and their respective target feature fields from the candidate feature set according to actual service needs, and sends the selection results to the intelligent agent platform.

[0120] In step 605, the intelligent agent platform sends the user-selected data source node and target feature fields to the multi-source engine, triggering subsequent multi-source data scheduling.

[0121] In step 606, the multi-source engine retrieves the previously saved sample identifier files from the database to ensure the consistency and accuracy of subsequent sample construction.

[0122] In step 607, the multi-source engine sends the sample identification file and target feature fields to the corresponding data source nodes, prompting each data source node to generate samples based on its local feature database.

[0123] In step 608, the data source node automatically extracts the target feature field values ​​corresponding to the sample identifier based on the sample identifier file, the target feature field, and the local feature database, and constructs the sample to be modeled.

[0124] In step 609, after completing sample construction, the data source node returns a response message indicating that sample construction is complete to the multi-source engine.

[0125] In step 610, after the multi-source engine summarizes the sample generation status of each data source node, it returns a response message indicating that the sample construction is complete to the intelligent agent platform.

[0126] In step 611, the intelligent agent platform sends a response message indicating that the sample construction is complete to the user, prompting the user that the subsequent modeling steps are ready.

[0127] In one exemplary embodiment, please refer to Figure 7 This diagram illustrates the temporal interaction of user-uploaded modeling files and supervised label files of samples to be modeled in order to achieve multi-data source modeling.

[0128] In step 701, the user can upload the modeling file and the supervision label file corresponding to the sample to be modeled to the intelligent agent platform through the interactive interface with the intelligent agent platform for subsequent model initialization, training and validation.

[0129] In step 702, after receiving the modeling file and the supervision label file, the intelligent agent platform can perform security checks and / or correctness verification on the received files respectively, such as checking whether the file is complete, whether the format conforms to the preset standard, and whether there are sensitive fields, to ensure that the data is compliant and usable.

[0130] In step 703, after the intelligent agent platform completes the verification of the files, it sends the verified modeling files and supervision label files to the multi-source engine, which is responsible for the subsequent unified scheduling and distribution.

[0131] In step 704, after receiving the modeling file and supervision label file, the multi-source engine stores them in the database to ensure data consistency and traceability.

[0132] In step 705, the multi-source engine distributes the modeling files to the selected data source nodes participating in the modeling task.

[0133] In step 706, after receiving the modeling file, each data source node can initialize the prediction model to be trained based on the modeling algorithm and parameter requirements specified in the modeling file, and combine it with the previously constructed modeling samples to train the prediction model locally to generate intermediate prediction results.

[0134] In step 707, after completing local training, each data source node feeds back the intermediate prediction results to the multi-source engine.

[0135] In step 708, the multi-source engine summarizes the intermediate prediction results collected from each data source node and sends them to the agent platform.

[0136] In step 709, the intelligent agent platform performs fusion processing on the intermediate prediction results returned by all data source nodes to obtain the final prediction result. It also generates first optimization information with the optimization objective of minimizing the error between the final prediction result and the real supervision label under the same sample identifier, which is used to guide the further optimization of subsequent model parameters.

[0137] In step 710, the intelligent agent platform sends the first optimization information to the multi-source engine.

[0138] In step 711, the multi-source engine sends the first optimization information to each data source node participating in the modeling task.

[0139] In step 712, after each data source node receives the first optimization information, it can iteratively optimize the local prediction model parameters based on the first optimization information to improve the accuracy and generalization ability of the model.

[0140] It is understood that steps 707 to 712 can be repeated multiple times until the iteration termination condition is met, such as reaching the preset number of iterations or the performance index of the prediction model meeting the preset requirements. This embodiment does not impose any restrictions on this.

[0141] In step 713, after each data source node completes the optimization and training of its local prediction model, it sends a training completion response to the multi-source engine to synchronize its status.

[0142] In step 714, after receiving the training completion response from each data source node, the multi-source engine feeds back the training completion response to the agent platform.

[0143] In step 715, the intelligent agent platform can perform security verification and / or correctness verification on the trained prediction models in each data source node, such as checking whether the model contains sensitive information or whether it meets the output expectations, and perform comprehensive analysis on the verification results to generate a performance evaluation report.

[0144] In step 716, the intelligent agent platform feeds back the performance evaluation report to the user, so that the user can intuitively understand the training effect and compliance of the final model, thereby supporting subsequent operations such as deployment, application or further optimization.

[0145] In an exemplary application scenario, the multi-source modeling system provided in this specification can be applied to financial credit risk prediction scenarios. The multi-source data involved in the example include bank internal account transaction data, loan repayment records of partner banks, and user transaction behavior data of third-party e-commerce platforms.

[0146] Specifically, a bank, as the modeling requester, can submit modeling requirements described in natural language to the intelligent agent platform through the user terminal. For example, "Predict the default risk of this batch of loan applicants and incorporate e-commerce consumption behavior and credit records from other banks." At the same time, the user terminal uploads the sample identifier (such as user ID) of the sample to be modeled, as well as optional monitoring tags (such as past default markers).

[0147] Upon receiving a request, the feature engineering agent in the intelligent agent platform can automatically retrieve feature metadata from multiple data source nodes (such as e-commerce platforms, partner banks, and the bank's own data center), and automatically generate a candidate feature set based on the modeling requirements, or by calling a language model. This candidate feature set may include candidate feature fields and corresponding data source nodes such as "average monthly transaction count on e-commerce platforms," ​​"overdue records from other banks in the past six months," and "fluctuation rate of salary statements from the bank in the past three months." The candidate feature set can be fed back to the user, allowing the user to select or the intelligent agent platform to automatically determine the required target feature fields and data source nodes.

[0148] After determining the target feature fields, the intelligent agent platform can send the selected target feature fields and sample identifiers to the multi-source engine, which will then schedule and distribute them to the corresponding data source nodes. Each data source node, based on its local feature database, queries the matching feature values ​​according to the sample identifiers and automatically constructs the corresponding sample to be modeled.

[0149] After sample preparation is complete, the user can further upload the modeling files (such as those containing the modeling algorithm structure and hyperparameter configuration) and supervision label files used for this modeling process. The intelligent agent platform receives and verifies these files, and then distributes them to the data source nodes participating in the modeling process through the multi-source engine. Based on the modeling files, the data source nodes can initialize their respective prediction models to be trained, and perform local model training in conjunction with the locally generated models to be modeled. After obtaining intermediate prediction results, they are uploaded to the multi-source engine, which then aggregates them and sends them back to the intelligent agent platform.

[0150] The model-building agent in the intelligent agent platform can perform weighted averaging, voting, or fusion model inference on the intermediate prediction results uploaded from various data source nodes to generate the final prediction result. It then generates optimization information with the optimization objective of minimizing the error between the prediction result and the supervision label. This optimization information is transmitted back to each data source node via the multi-source engine, allowing each node to further update its local prediction model parameters.

[0151] After the model training is completed, the intelligent agent platform can perform security checks (such as checking whether it contains sensitive information) and correctness verification (such as cross-validation, accuracy or AUC) on the prediction model of each data source node through the model evaluation agent, and generate a performance evaluation report, which is finally fed back to the user to help the user make a comprehensive judgment on the model effect and deployment decision.

[0152] This multi-source modeling process exemplifies how the system can efficiently leverage distributed multi-source data to achieve joint modeling while protecting the data privacy of all parties. This significantly improves the model's coverage and prediction accuracy, and the operation process is more intuitive and automated for users, effectively lowering the modeling threshold and meeting the high standards of risk control required in financial scenarios.

[0153] The various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, the arbitrary combination of various technical features in the above embodiments is also within the scope of this specification.

[0154] In some embodiments, please refer to Figure 8 This specification also provides a multi-data source modeling method for an intelligent agent platform, which includes:

[0155] In S801, the system receives modeling requirements information described in natural language, sample identifiers of the samples to be modeled, and supervision labels sent by the user terminal.

[0156] In S802, based on the modeling requirement information and the feature metadata obtained from the data source nodes, at least two data source nodes to participate in the modeling task and their respective target feature fields are determined, and the sample identifier and target feature fields are sent to the at least two data source nodes so that the data source nodes can construct the sample to be modeled corresponding to the sample identifier and perform model training.

[0157] In S803, intermediate prediction results sent by the data source node during model training are received.

[0158] In S804, intermediate prediction results are fused to generate final prediction results. The first optimization information is generated and sent to the data source node with the optimization objective of minimizing the error between the final prediction results and the supervision labels, so that the data source node can optimize the model parameters based on the first optimization information.

[0159] For example, based on modeling requirement information and feature metadata obtained from data source nodes, at least two data source nodes to participate in the modeling task and their respective target feature fields are determined, including: generating modeling prompt information based on modeling requirement information and feature metadata, and inputting the modeling prompt information into a language model so that the language model generates a candidate feature set, the candidate feature set including multiple candidate data source nodes that meet the modeling requirements selected by the language model and candidate feature fields suitable for modeling in each candidate data source node; from the candidate feature set, at least two data source nodes to participate in the modeling task and target feature fields suitable for modeling are determined.

[0160] For example, determining at least two data source nodes participating in the modeling task and target feature fields suitable for modeling from a candidate feature set includes: sending the candidate feature set to the user terminal and receiving the user terminal's return of at least two data source nodes selected by the user to participate in the modeling task and their respective target feature fields.

[0161] For example, the candidate feature set also includes feature processing strategies for the candidate feature fields. The method further includes sending the feature processing strategies for the target feature fields to at least two data source nodes.

[0162] For example, the method further includes: acquiring a modeling file and sending it to at least two data source nodes to participate in the modeling task.

[0163] For example, obtaining a modeling file includes: (1) receiving a modeling file uploaded by a user; or, (2) inputting modeling requirement information and a model generation template into a language model so that the language model can determine the appropriate modeling algorithm by analyzing the modeling requirement information, and generate a modeling file based on the modeling algorithm and the model generation template; or, (3) inputting modeling requirement information into a language model so that the language model can determine the appropriate modeling algorithm by analyzing the modeling requirement information; obtaining the corresponding modeling file from the modeling file library based on the modeling algorithm determined by the language model, wherein the modeling file library is used to store modeling files corresponding to different modeling algorithms.

[0164] For example, the intermediate prediction results are fused to generate a final prediction result, and the first optimization information is generated and sent to the data source node with the optimization objective of minimizing the error between the final prediction result and the supervision label. This includes: inputting the intermediate prediction results into the fusion model to be trained to obtain the final prediction result, generating the first optimization information and the second optimization information with the optimization objective of minimizing the error between the final prediction result and the supervision label, sending the first optimization information to the data source node, and using the second optimization information to optimize the parameters of the fusion model.

[0165] For example, fusing intermediate prediction results to generate a final prediction result includes: inputting the intermediate prediction results into a language model so that the language model can fuse the intermediate prediction results to generate a final prediction result.

[0166] For example, the method further includes: performing security checks and / or correctness verifications on the trained prediction models in each data source node, and analyzing the verification results to generate a performance evaluation report; wherein, the security checks include at least one of the following: sensitivity checks on model parameters or output content; security checks on model structure; security checks on data transmission links during training and inference phases; and the correctness verifications include at least one of the following: cross-validation, accuracy metric calculation, and result interpretability verification.

[0167] For example, analyzing the verification results to generate a performance evaluation report includes: inputting the verification results and a preset report template into a language model, so that the language model can analyze the verification results and generate a performance evaluation report in combination with the report template.

[0168] In some embodiments, please refer to Figure 9 This specification also provides a multi-data source modeling method for data source nodes, which includes:

[0169] In S901, the sample identifier and target feature fields sent by the intelligent agent platform are received.

[0170] In S902, a sample to be modeled is constructed based on the sample identifier, target feature field, and local feature database.

[0171] In S903, the prediction model to be trained is trained using the samples to be modeled, intermediate prediction results are generated and sent to the agent platform.

[0172] In S904, the first optimization information returned by the intelligent agent platform based on the intermediate prediction results is received, and the parameters of the prediction model are optimized based on the first optimization information.

[0173] For example, based on the sample identifier, the target feature field, and the local feature database, a modeling sample corresponding to the sample identifier is constructed, including: receiving a feature processing strategy for the target feature field sent by the intelligent agent platform; querying the target feature field value corresponding to the sample identifier from the feature database based on the target feature field and the sample identifier; and processing the target feature field value based on the feature processing strategy to generate a modeling sample corresponding to the sample identifier.

[0174] For example, before training the prediction model to be trained using the sample to be modeled, the method further includes: receiving a modeling file sent by the intelligent agent platform, initializing based on the modeling file, and obtaining the prediction model to be trained.

[0175] In some embodiments, Figure 10 This is a schematic structural diagram of a device provided in an exemplary embodiment. For example... Figure 10 As shown, device 1000 mainly consists of a communication interface 1002, a user interface 1004, a processor 1006, and a data storage 1008. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 1010. The communication interface 1002 enables device 1000 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 1002 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 1002 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 1002 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 1002 may also include multiple physical communication interfaces, such as Wi-Fi interfaces, Bluetooth interfaces, and wide-area wireless interfaces.

[0176] User interface 1004 includes receiving user input and providing output to the user. Therefore, user interface 1004 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 1004 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 1004 may include software, circuitry, or other forms of logic capable of transmitting and receiving data from external user input / output devices. Additionally or alternatively, device 1000 may support remote access from other devices via communication interface 1002 or another physical interface (not shown). User interface 1004 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 1004 can also be configured as a display device for rendering or displaying text fragments.

[0177] Processor 1006 may contain one or more general-purpose processors and / or special-purpose processors.

[0178] Data storage 1008 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 1006. Data storage 1008 may include removable and non-removable components.

[0179] Processor 1006 is capable of executing program instructions 1018 (e.g., compiled or uncompiled program logic and / or machine code) stored in data storage 1008 to perform the various functions described herein. Data storage 1008 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 1000, enable device 1000 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Execution of program instructions 1018 by processor 1006 may result in processor 1006 using data 1012.

[0180] For example, program instructions 1018 may include an operating system 1022 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 1000 and one or more applications 1020 (e.g., a browser, social media application, or game application). Similarly, data 1012 may include operating system data 1016 and application data 1014. Operating system data 1016 is primarily accessible to the operating system 1022, while application data 1014 is primarily accessible to one or more applications 1020. Application data 1014 may reside in a file system visible or hidden from the user of device 1000.

[0181] Application 1020 can communicate with operating system 1022 through one or more application programming interfaces (APIs). These APIs help application 1020 read and / or write application data 1014, transmit or receive information via communication interface 1002, receive or display information on user interface 1004, etc.

[0182] In some terminology, application 1020 may be simply referred to as "app". Furthermore, application 1020 can be downloaded to device 1000 through one or more online app stores or app markets. However, applications can also be installed on device 1000 in other ways, such as through a web browser or a physical interface on device 1000 (e.g., a USB port).

[0183] For example, multi-data source modeling apparatuses can be applied to, for example, Figure 10 The device shown is used to implement the technical solution described in this specification. The multi-data source modeling device may include:

[0184] The receiving module is used to receive modeling requirement information described in natural language, sample identifiers of the samples to be modeled, and supervision labels sent by the user terminal.

[0185] The data processing and sending module is used to determine at least two data source nodes to participate in the modeling task and their respective target feature fields based on the modeling requirement information and the feature metadata obtained from the data source nodes, and to send the sample identifier and target feature fields to the at least two data source nodes so that the data source nodes can construct the sample to be modeled corresponding to the sample identifier and perform model training.

[0186] The receiving module is also used to receive intermediate prediction results sent by the data source node during model training.

[0187] The data processing and sending module is also used to fuse intermediate prediction results, generate final prediction results, and generate first optimization information and send it to the data source node with the optimization objective of minimizing the error between the final prediction results and the supervision labels, so that the data source node can optimize the model parameters based on the first optimization information.

[0188] For example, multi-data source modeling apparatuses can be applied to, for example, Figure 10 The device shown is used to implement the technical solution described in this specification. The multi-data source modeling device may include:

[0189] The receiving module is used to receive sample identifiers and target feature fields sent by the intelligent agent platform.

[0190] The sample construction module is used to construct the sample to be modeled corresponding to the sample identifier based on the sample identifier, target feature fields, and local feature database.

[0191] The model training module is used to train the prediction model to be trained using the samples to be modeled, generate intermediate prediction results, and send them to the agent platform.

[0192] The model optimization module is used to receive the first optimization information returned by the intelligent agent platform based on the intermediate prediction results, and optimize the parameters of the prediction model based on the first optimization information.

[0193] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more of these specifications, the functions of each module or unit can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0194] Based on the same concept as the methods described above, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor performs the steps of the method as described in any of the above embodiments by executing the executable instructions.

[0195] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0196] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0197] What those skilled in the art will understand is:

[0198] In this specification, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded.

[0199] In this specification, “a,” “an,” and “the” do not specifically refer to the singular, but may also include the plural.

[0200] In this specification, ordinal numbers such as "first," "second," etc., do not necessarily indicate order; they are often used to distinguish between objects. For example, "first server" and "second server" usually refer to two servers. To differentiate between these two servers, they are described as "first server" and "second server." Of course, sometimes these two servers may be the same server.

[0201] In this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can also mean indirect receiving and sending. For example, A receiving data sent by B can be understood as A directly receiving the data sent by B, or it can be understood as A indirectly receiving the data sent by B through other entities such as C. Similarly, B sending data to A can be understood as B sending the data directly to A, or it can be understood as B indirectly sending the data to A through other entities such as C. Here, C can be one entity, or it can be two or more entities.

[0202] In this specification, unless explicitly stated otherwise, the relationships between structures can be direct or indirect. For example, when describing "A is connected to B," unless it is explicitly stated that A and B are directly connected, it should be understood that A can be directly connected to B or indirectly connected to B. Similarly, when describing "A is on top of B," unless it is explicitly stated that A is directly above B (AB is adjacent and A is above B), it should be understood that A can be directly above B or indirectly above B (AB is separated by other elements, and A is above B). And so on.

[0203] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0204] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is only one of many possible execution orders and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims.

Claims

1. A multi-data-source modeling system, comprising a user terminal, an agent platform, and a plurality of data source nodes; the user terminal is configured to send modeling requirement information described in natural language, sample identification of a sample to be modeled, and a supervision label to the agent platform; the agent platform is configured to determine at least two data source nodes to be involved in a modeling task and respective target feature fields based on the modeling requirement information and feature metadata obtained from the data source nodes, and send the sample identification and the target feature fields to the at least two data source nodes; the data source nodes are configured to construct a sample to be modeled corresponding to the sample identification based on the sample identification, the target feature fields, and a local feature database, and train a prediction model to be trained using the sample to be modeled, generate an intermediate prediction result, and send the intermediate prediction result to the agent platform; the agent platform is further configured to fuse the intermediate prediction results to generate a final prediction result, and generate first optimization information to minimize an error between the final prediction result and the supervision label as an optimization objective, and send the first optimization information to the data source nodes; the data source nodes are further configured to optimize parameters of the prediction model based on the first optimization information. 2.The system of claim 1, wherein the agent platform comprises a feature engineering agent; the feature engineering agent is configured to generate modeling prompt information based on the modeling requirement information and the feature metadata, and input the modeling prompt information to a language model to enable the language model to generate a candidate feature set, the candidate feature set comprising a plurality of candidate data source nodes meeting modeling requirements and candidate feature fields suitable for modeling in each candidate data source node filtered by the language model; from the candidate feature set, at least two data source nodes involved in the modeling task and target feature fields suitable for modeling are determined. 3.The system of claim 2, wherein the feature engineering agent is specifically configured to send the candidate feature set to the user terminal; the user terminal is further configured to display the candidate feature set, and send at least two data source nodes involved in the modeling task and respective target feature fields selected by a user to the feature engineering agent; and / or, the candidate feature set further comprises feature processing strategies for the candidate feature fields; the feature engineering agent is further configured to send feature processing strategies for the target feature fields to the at least two data source nodes; and the data source nodes are specifically configured to query target feature field values corresponding to the sample identification from the feature database based on the target feature fields and the sample identification, and process the target feature field values based on the feature processing strategies to generate the sample to be modeled corresponding to the sample identification. 4.The system of claim 1, wherein the agent platform comprises a model construction agent; the model construction agent is configured to obtain a modeling file and send the modeling file to the at least two data source nodes involved in the modeling task. The data source node is further configured to initialize based on the modeling file to obtain the to-be-trained prediction model.

5. The system of claim 4, wherein the user terminal is further configured to upload the modeling file to the intelligent agent platform. Alternatively, the model construction intelligent agent is further configured to input the modeling requirement information and the model generation template into a language model, so that the language model determines an adaptive modeling algorithm by analyzing the modeling requirement information, and generates the modeling file based on the modeling algorithm and the model generation template. Alternatively, the model construction intelligent agent is further configured to input the modeling requirement information into a language model, so that the language model determines an adaptive modeling algorithm by analyzing the modeling requirement information. The modeling file corresponding to the modeling algorithm determined by the language model is obtained from a modeling file library, and the modeling file library is configured to store modeling files corresponding to different modeling algorithms.

6. The system of claim 4, wherein the model construction intelligent agent is further configured to input the intermediate prediction result into a to-be-trained fusion model to obtain a final prediction result, and generate second optimization information and optimize parameters of the fusion model by using the second optimization information, with minimizing an error between the final prediction result and the supervision label as an optimization objective. Alternatively, the model construction intelligent agent is further configured to input the intermediate prediction result into a language model, so that the language model fuses the intermediate prediction result to generate a final prediction result. Alternatively, the model construction intelligent agent is further configured to fuse the intermediate prediction results returned by the data source nodes based on a preset fusion rule to generate a final prediction result.

7. The system of claim 1, wherein the intelligent agent platform comprises a model evaluation intelligent agent. The model evaluation intelligent agent is configured to perform security checking and / or correctness verification on the trained prediction model in each data source node, and analyze the checking result to generate a performance evaluation report. wherein The security checking comprises at least one of the following: sensitivity checking of model parameters or output content; model structure security checking; and training and inference stage data transmission link security checking. The correctness verification comprises at least one of the following: cross-validation, accuracy index calculation, and result interpretability verification.

8. The system of claim 7, wherein the model evaluation intelligent agent is specifically configured to input the checking result and a preset report template into a language model, so that the language model analyzes the checking result and generates a performance evaluation report in combination with the report template.

9. The system of claim 1, further comprising a multi-source engine configured to serve as a unified scheduling center of the plurality of data source nodes, and the intelligent agent platform interacts with each data source node through the multi-source engine.

10. A multi-data-source modeling method applied to an intelligent agent platform of a multi-data-source modeling system as claimed in any one of claims 1 to 9, the method comprising: receiving modeling requirement information described in natural language, sample identification of a to-be-modeled sample, and supervision labels sent by a user terminal; ​ ​ determine at least two data source nodes and respective target feature fields to participate in the modeling task based on the modeling requirement information and feature metadata obtained from the data source nodes, and send the sample identifier and the target feature fields to the at least two data source nodes, so that the data source nodes build a to-be-modeled sample corresponding to the sample identifier and perform model training; receive intermediate prediction results sent by the data source nodes in the model training process; fuse the intermediate prediction results to generate a final prediction result, and generate first optimization information and send it to the data source nodes to minimize the error between the final prediction result and the supervision label as an optimization objective, so that the data source nodes optimize model parameters based on the first optimization information.

11. A multi-data-source modeling method applied to a data source node in the multi-data-source modeling system of any one of claims 1 to 9, the method comprising: receiving a sample identifier and a target feature field sent by an agent platform; building a to-be-modeled sample corresponding to the sample identifier based on the sample identifier, the target feature field, and a local feature database; training a to-be-trained prediction model using the to-be-modeled sample, generating an intermediate prediction result, and sending it to the agent platform; receiving first optimization information returned by the agent platform based on the intermediate prediction result, and optimizing parameters of the prediction model based on the first optimization information.

12. An electronic device, comprising: comprising: a processor; a memory for storing processor-executable instructions; wherein the processor implements the steps of the method of claim 10 or 11 by running the executable instructions.

13. A computer-readable storage medium, characterized in that, having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method of any one of claims 10 or 11.

14. A computer program product, characterised in that, comprising computer program / instructions, which, when executed by a processor, implement the steps of the method of any one of claims 10 or 11.

Images