Multi-data source modeling method, system and device, storage medium and program product

Abstract

The invention provides a multi-data source modeling method, system and device, a storage medium and a program product. The multi-data-source modeling system is composed of a user side, an intelligent agent platform and a plurality of data source nodes. A user side sends a modeling demand in a natural language, and uploads a sample identifier and a supervision label of a to-be-modeled sample. The intelligent agent platform selects at least two data source nodes participating in modeling and respective target feature fields according to modeling requirements and feature metadata obtained from the data source nodes, and then sends sample identifiers and the target feature fields to the nodes. And the data source node constructs a to-be-modeled sample according to the sample identifier, the target feature field and a local feature database, trains a to-be-trained prediction model, generates an intermediate prediction result, and sends the intermediate prediction result to the agent platform. The agent platform fuses the intermediate result to generate a final prediction result, first optimization information is generated and returned with the purpose of minimizing the error between the final prediction result and the supervision label, and the data source node optimizes prediction model parameters according to the first optimization information.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of model building technology, and in particular, to a multi-data source modeling method, system, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] With the vigorous development of big data and artificial intelligence technologies, the explosive growth of data scale and dimension has driven the upgrading of modeling needs, but the limitations of traditional modeling models have become increasingly prominent.

[0003] At the modeling data level, existing modeling approaches mostly rely on a single data source, constrained by the data scope of a single entity. This results in limited data coverage and insufficient sample diversity, making it difficult to meet the requirements for model generalization and predictive accuracy in complex scenarios. Furthermore, due to privacy regulations and data security concerns, institutions are generally cautious about sharing raw data, further exacerbating the limitations of modeling data.

[0004] At the modeling operation level, the existing modeling process requires high professional and technical capabilities of users. Many links in the model training process often need to be completed manually by professionals. The operation process is cumbersome and time-consuming, and the modeling effect is easily affected by human errors. The execution efficiency and flexibility are low. Summary of the Invention

[0005] In view of this, one or more embodiments of this specification provide the following technical solutions: According to a first aspect of one or more embodiments of this specification, a multi-data source modeling system is proposed, including a user terminal, an agent platform, and multiple data source nodes; The user terminal is used to send modeling requirement information described in natural language, sample identification and supervision label of the sample to be modeled to the intelligent agent platform; The agent platform is used to determine at least two data source nodes to participate in the modeling task and their respective target feature fields based on the modeling requirement information and the feature metadata obtained from the data source nodes, and send the sample identifier and the target feature field to the at least two data source nodes; The data source node is used to construct a sample to be modeled corresponding to the sample identifier based on the sample identifier, the target feature field and the local feature database, and use the sample to be modeled to train the prediction model to be trained, generate an intermediate prediction result and send it to the intelligent agent platform; The intelligent agent platform is further configured to fuse the intermediate prediction results to generate a final prediction result, and to generate first optimization information with minimizing the error between the final prediction result and the supervision label as an optimization goal, and to send the first optimization information to the data source node; The data source node is further used to optimize the parameters of the prediction model based on the first optimization information.

[0006] According to a second aspect of one or more embodiments of this specification, a multi-data source modeling method is proposed, which is applied to an intelligent agent platform. The method includes: Receive modeling requirement information described in natural language, sample identification and supervision label of the sample to be modeled sent by the user; Based on the modeling requirement information and the feature metadata obtained from the data source node, at least two data source nodes to be involved in the modeling task and their respective target feature fields are determined, and the sample identifier and the target feature field are sent to the at least two data source nodes, so that the data source nodes construct the sample to be modeled corresponding to the sample identifier and perform model training; Receiving the intermediate prediction results sent by the data source node during the model training process; The intermediate prediction results are fused to generate a final prediction result, and the optimization goal is to minimize the error between the final prediction result and the supervisory label, generate first optimization information and send it to the data source node, so that the data source node optimizes the model parameters based on the first optimization information.

[0007] According to a third aspect of one or more embodiments of this specification, a multi-data source modeling method is proposed, which is applied to a data source node. The method includes: Receive the sample identification and target feature fields sent by the intelligent agent platform; Based on the sample identifier, the target feature field and the local feature database, construct a sample to be modeled corresponding to the sample identifier; Using the to-be-modeled samples to train the prediction model to be trained, generating an intermediate prediction result and sending it to the intelligent agent platform; Receive first optimization information returned by the intelligent agent platform based on the intermediate prediction result, and optimize the parameters of the prediction model based on the first optimization information.

[0008] According to a fourth aspect of the embodiments of this specification, there is provided an electronic device, including: processor; a memory for storing processor-executable instructions; When the processor executes the executable instructions, it is used to implement the method described in the second aspect or the third aspect.

[0009] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the method described in the second aspect or the third aspect are implemented.

[0010] According to a sixth aspect of the embodiments of this specification, a computer program product is provided, comprising a computer program, which implements the steps of the method described in the second aspect or the third aspect when executed by a processor.

[0011] As can be seen from the above embodiments, the modeling requirement transmission process is simplified by the user end in the form of natural language interaction, which lowers the user operation threshold; the intelligent platform can analyze based on the modeling requirement information and the feature metadata obtained from the data source node, thereby determining at least two data source nodes and target feature fields that meet the modeling requirements, realizing cross-data source collaborative modeling, breaking through the limitations of a single data source, and improving the data breadth and dimension of model training; each data source node completes sample construction and model training based on local data and only feeds back intermediate prediction results, which not only ensures data privacy and security, but also promotes the collaborative iteration of prediction models in each data source node through the result fusion and parameter optimization of the intelligent platform, ultimately improving the accuracy and generalization ability of the prediction model. It realizes full process automation from natural language interaction to multi-data source modeling, with the advantages of efficiency, flexibility, security and ease of use, which helps to release the value of multi-party data and improve modeling quality and service response speed.

[0012] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a structural diagram of a multi-data source modeling system provided by an exemplary embodiment.

[0014] Figure 2 It is a schematic diagram of three agents included in the agent platform provided by an exemplary embodiment.

[0015] Figure 3 It is a schematic diagram of the interaction between the intelligent agent platform and the data source node during the modeling process provided by an exemplary embodiment.

[0016] Figure 4A It is a structural diagram of another multi-data source modeling system provided by an exemplary embodiment.

[0017] Figure 4B This is a structural diagram of another multi-data source modeling system provided by an exemplary embodiment.

[0018] Figure 5 This is a time sequence interaction diagram of a user uploading a sample identifier provided by an exemplary embodiment.

[0019] Figure 6 It is a time sequence interaction diagram of a user side proposing a modeling requirement provided by an exemplary embodiment.

[0020] Figure 7 This is a time-series interaction diagram provided by an exemplary embodiment in which a user uploads a modeling file and a supervision label file to implement multi-data source modeling.

[0021] Figure 8 This is a flowchart of a multi-data source modeling method provided by an exemplary embodiment.

[0022] Figure 9 This is a flowchart of another multi-data source modeling method provided by an exemplary embodiment.

[0023] Figure 10 It is a structural diagram of a device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0024] To help those skilled in the art better understand the technical solutions in this specification, the following will provide a clear and complete description of the technical solutions in the embodiments of this specification, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. All other embodiments derived by those skilled in the art based on the embodiments in this specification without creative effort shall fall within the scope of protection of this specification.

[0025] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0026] See also Figure 1 An embodiment of this specification provides a multi-data source modeling system, including a user terminal 10, an intelligent agent platform 20, and multiple data source nodes 30.

[0027] Each data source node 30 includes a local feature database, which is used to store feature tuples corresponding to user identifiers. These tuples typically contain user behavior, attribute information, and other related information collected and accumulated by each data source node 30 during the actual provision or operation of services. The local feature databases held by different data source nodes 30 may contain data of different dimensions corresponding to the same user identifier, or may contain only user data unique to the scope of their service, depending on whether the user has used the products or services provided by that node.

[0028] Therefore, in real-world scenarios, the same user ID may be recorded in multiple data source nodes 30, and the recorded feature information may complement and corroborate each other, forming a cross-domain, multi-dimensional data set. This provides the data foundation and feasibility for subsequent cross-node feature alignment, collaborative training, and model fusion based on sample identifiers (e.g., user IDs). Furthermore, the local feature database is stored and used only within the data source node 30, helping to ensure the security and privacy of user data and avoiding the potential risks associated with direct sharing of raw data across nodes.

[0029] The following is an example of the multi-data source modeling process: The user terminal 10 is used to send modeling requirement information described in natural language, sample identification and supervision labels of samples to be modeled to the intelligent agent platform 20.

[0030] For example, the user terminal 10 may be owned by an institution or research party that needs to use multi-source data to conduct modeling analysis. The user can submit a modeling request to the intelligent platform 20 through the user terminal 10, avoiding the tedious operation of configuring complex modeling parameters by themselves.

[0031] Among them, modeling requirement information is used to express the user's description of the model's objectives, feature range, output requirements, etc., and can be directly input using natural language to improve interactive convenience and ease of use.

[0032] The sample identifiers and supervisory labels of the samples to be modeled can be uploaded simultaneously with the modeling requirements information, or they can be uploaded in batches at different stages. For example, before the specific modeling goal is determined, the user can first submit the sample identifiers involved to the agent platform 20 for preprocessing and storage. Later, after the modeling requirements are clarified, the specific modeling requirements information and corresponding supervisory labels can be uploaded to supplement the label comparison and effect verification during model training.

[0033] It should be noted that the user terminal 10 only provides sample identification and supervision labels, and does not directly upload the original sample data containing feature values. The input samples used for model training are extracted by each data source node 30 based on the local feature database and sample identification, thereby effectively avoiding the cross-node transmission of original data and improving data security and privacy protection.

[0034] The intelligent agent platform 20 is used to determine at least two data source nodes 30 and their respective target feature fields to participate in the modeling task based on the modeling requirement information and the feature metadata obtained from the data source node 30, and send the sample identifier and target feature field to at least two data source nodes 30.

[0035] For example, the agent platform 20 can achieve full process automation from multi-data source access to model generation through division of labor and coordination among multiple agents, thereby improving modeling efficiency, lowering technical barriers, and enhancing system flexibility and scalability. Figure 2 The agent platform 20 includes but is not limited to a feature engineering agent 21, a model building agent 22, and a model evaluation agent 23. The feature engineering agent 21 is used to perform tasks such as feature screening and determining feature processing strategies; the model building agent 22 is used to train models and optimize model parameters; and the model evaluation agent 23 is used to perform security verification, correctness validation, and verification result analysis on the model.

[0036] The feature engineering agent 21 can receive a batch of sample identifiers of samples to be modeled uploaded by the user terminal 10, and organize, deduplicate, format and complete the batch of sample identifiers to ensure that the sample identifiers are unique, non-repetitive and without omissions. On this basis, the feature engineering agent 21 can automatically perform a number of pre-processing tasks related to sample identifiers by calling preset functions or connecting to internal / external APIs (Application Programming Interfaces), such as grouping, sharding, batch management, label relationship matching, index generation, standardized records and status tracking of sample identifiers. The above-mentioned processing can be completed by the feature engineering agent 21 automatically coordinating the call of the corresponding logic service module according to the modeling requirements, and the user does not need to manually operate the specific details. For the processed sample identifiers and their corresponding batch, group or status information, the feature engineering agent 21 can also perform unified storage and version management to facilitate subsequent efficient alignment calls between multiple data source nodes 30, and support later backtracking, appending or multiple rounds of iterations.

[0037] After receiving the modeling requirement information described in natural language, the feature engineering agent 21 can determine at least two data source nodes 30 to participate in the modeling task and their respective target feature fields based on the modeling requirement information and the feature metadata obtained from the data source node 30.

[0038] For example, the feature engineering agent 21 can obtain feature metadata from each data source node 30. This metadata describes the feature fields held by each data source node 30 and their attribute information, such as feature name, field type, value range, update frequency, and availability status. After obtaining the feature metadata from each data source node 30, the feature engineering agent 21 can combine the modeling requirement information uploaded by the user terminal 10 to generate modeling prompt information (such as required feature categories, modeling objectives, and prioritized scenario constraints). This modeling prompt information is then input into a preset language model. Leveraging the language model's contextual understanding and semantic reasoning capabilities, the agent automatically selects multiple candidate data source nodes 30 that meet the modeling requirements and candidate feature fields suitable for modeling within each candidate data source node 30, thereby forming a candidate feature set. The feature engineering agent 21 can then further determine from the candidate feature set at least two data source nodes 30 that will ultimately participate in the modeling task, as well as the target feature fields for each node. The agent then sends the sample identifier and target feature fields to the at least two data source nodes 30 that will participate in the modeling task.

[0039] Through this approach, the feature engineering agent 21 fully leverages the language model's capabilities in natural language understanding and knowledge matching, helping users quickly transform service-oriented modeling requirements into executable feature selection solutions for multiple data sources, reducing the labor costs and technical barriers to feature screening and matching. Furthermore, by combining feature metadata from multiple data source nodes 30 with semantic reasoning, potentially useful cross-domain features can be more flexibly discovered, increasing the richness and diversity of input features. This contributes to improved accuracy and generalization capabilities of the final model, maximizing the value of multi-source data.

[0040] In one possible implementation, after obtaining the candidate feature set, the feature engineering agent 21 can send the candidate feature set to the user terminal 10. The user terminal 10 is also used to display the candidate feature set, so that the user can select and combine different data source nodes 30 and their candidate feature fields based on their own understanding and specific needs. The user terminal 10 is also used to send at least two data source nodes 30 selected by the user to participate in the modeling task and their respective target feature fields to the feature engineering agent 21 for use in subsequent sample extraction and modeling processes. In this way, the user's subjective judgment ability in understanding actual service scenarios and data usage preferences can be fully utilized, so that the final feature fields and data source nodes 30 are more in line with actual service goals and context requirements, thereby improving model usability and interpretability.

[0041] In another possible implementation, after obtaining the candidate feature set, the feature engineering agent 21 can perform automated quality analysis and optimization screening on the candidate data source nodes 30 and their candidate feature fields based on a preset automated evaluation logic. For example, it can sort or score each candidate based on multi-dimensional indicators such as feature coverage, availability, relevance to the modeling target, redundancy, and compliance, and then automatically determine at least two data source nodes 30 and their respective target feature fields for the final modeling, and directly enter the subsequent sample set construction and model training phase. This method significantly improves the automation level and execution efficiency of the modeling process, and is particularly suitable for scenarios that require high data analysis expertise or require rapid delivery.

[0042] After receiving the sample identifier and target feature field from the feature engineering agent 21, the data source node 30 can automatically construct a sample to be modeled corresponding to the sample identifier based on the sample identifier, the target feature field, and its local feature database. Specifically, the feature database maintained locally by each data source node 30 is used to store feature tuples associated with the sample identifier, and these feature tuples contain values ​​of different feature fields. When the data source node 30 receives the specified target feature field, it can automatically query and extract the corresponding target feature field value in the local database for each sample identifier, and then organize these field values ​​into samples to be modeled according to a predetermined format.

[0043] In this way, the data source node 30 can dynamically extract the feature field values ​​that match the sample identifier on demand without exposing the original complete database, generating sample data that only contains the information required for modeling. This process is completed independently and locally by each data source node 30, without the need to transmit the complete original data externally, effectively meeting the requirements of minimizing data use and protecting privacy in multi-agent scenarios. In addition, the ability to automatically construct samples can significantly reduce manual docking and data preparation work, improve the flexibility and operability of calling multi-source data, ensure data consistency and context alignment in the subsequent multi-node model training phase, and provide a reliable data foundation for multi-source collaborative modeling.

[0044] In an optional embodiment, when generating a candidate feature set, the language model may also combine the modeling requirement information and feature metadata input by the user to generate a corresponding feature processing strategy for the candidate feature fields in each candidate data source node 30. The "feature processing strategy" here can be understood as the preprocessing, conversion, or regularization method required to be performed on a specific feature field before participating in modeling, in order to improve the consistency, usability, and adaptability of the features to the modeling objectives.

[0045] For example, feature processing strategies may include, but are not limited to: filling missing values, correcting outliers, standardizing, or normalizing numerical features; binning and encoding (such as one-hot encoding or label encoding) categorical features; segmenting time series or deriving derived features for time fields; and desensitizing or aggregating sensitive fields. These strategies can be automatically generated by a language model based on an existing scenario knowledge base and semantic reasoning capabilities, and matched and output based on contextual requirements.

[0046] In this solution, the feature engineering agent 21 not only sends the determined target feature field and sample identifier to at least two data source nodes 30 to be involved in the modeling task, but also sends the feature processing strategy for the target feature field to the relevant data source nodes 30. After receiving this information, each data source node 30 queries the target feature field value corresponding to the sample identifier from the feature database based on the target feature field and sample identifier, and processes the target feature field value based on the feature processing strategy (such as data preprocessing or conversion operations) to generate a sample to be modeled corresponding to the sample identifier.

[0047] In this way, the generation and distribution of feature processing strategies significantly improves the consistency and automation of feature processing in a cross-subject, multi-data source environment, reducing model deviations and unstable effects caused by the inconsistency of feature processing calibers of each data source node 30. At the same time, with the help of language model generation and automatic scheduling of intelligent agents, the complexity of manual intervention and manual parameter adjustment can be effectively reduced, and the preparation efficiency of multi-source data modeling can be accelerated. In addition, each data source node 30 completes feature processing locally to avoid the external flow of raw data, further strengthen data privacy protection and compliant use, and promote the sustainability of multi-party collaboration and data availability.

[0048] After completing the generation of the samples to be modeled, each data source node 30 can send sample generation confirmation information to the agent platform 20 to indicate that it has completed sample preparation and can be used for subsequent model training. After the model building agent 22 in the agent platform 20 receives all the sample generation confirmation information, it can trigger the subsequent modeling process, which specifically includes obtaining the modeling file and sending the modeling file to at least two data source nodes 30 to participate in the modeling task. Among them, the modeling file in the embodiment of this specification can be understood as a technical file that describes or encapsulates key configuration information such as algorithm parameters, model structure, training process configuration, input and output format definition, etc. required for a specific modeling task. Its core function is to instruct each data source node 30 how to perform model initialization, parameter setting, training strategy, output result format and other operations based on the locally generated samples to be modeled. It is a unified instruction set and technical basis for multi-node distributed collaborative training.

[0049] In one possible implementation, the modeling file can be directly generated by the user terminal 10 or uploaded to the agent platform 20 after selection. Exemplarily, the model-building agent 22 can feed back sample generation confirmation information from each data source node 30 to the user terminal 10, prompting the user to upload the corresponding modeling file. For example, based on their own industry experience or internal requirements, the user can locally configure the modeling algorithm, parameter template, etc., and after confirming that the sample preparation is complete, submit the complete modeling file to the agent platform 20 for distribution. This model allows users to retain greater control over the algorithms used and modeling details, making it easier to meet personalized scenarios or compliance requirements, and enhancing user participation and customization capabilities in the modeling process.

[0050] In another possible implementation, the model-building agent 22 can input modeling requirements and a model generation template into a language model. The language model analyzes the modeling requirements to determine an appropriate modeling algorithm (such as classification, regression, or clustering) and generates a modeling file based on the modeling algorithm and the model generation template. This approach, through natural language understanding and intelligent generation, significantly reduces users' reliance on technical details such as modeling algorithms and scripts. This allows non-technical users to quickly initiate multi-data source modeling with zero barriers to entry, significantly improving modeling efficiency. It also offers strong adaptability and facilitates flexible iteration.

[0051] In another possible implementation, the model-building agent 22 is also used to input modeling requirement information into the language model, so that the language model can determine the appropriate modeling algorithm by analyzing the modeling requirement information. However, the difference is that the model-building agent 22 can directly retrieve and obtain ready-made modeling files corresponding to the modeling algorithm from a pre-maintained modeling file library. The modeling file library is used to store modeling files corresponding to different modeling algorithms for rapid access in different scenarios. While ensuring algorithm adaptability, this solution significantly reduces the time overhead of repeated generation and debugging through "ready-to-use" technology. It is suitable for common or standardized modeling tasks and can achieve batch and efficient automatic distribution and execution, further improving the system's modeling response speed and reusability.

[0052] After receiving the modeling file sent by the model building agent 22, each data source node 30 participating in the modeling task can be initialized based on the model structure, training strategy, hyperparameters and other configuration information contained in the modeling file to obtain the prediction model to be trained. This initialization process ensures that different data source nodes 30 can execute consistent training processes under the same modeling framework and configuration while maintaining the privacy isolation of their respective data, thereby ensuring the fusion and consistency of multi-source collaborative modeling.

[0053] Then, see Figure 3Each data source node 30 trains its locally initialized prediction model using the samples to be modeled, generating a local intermediate prediction result. The intermediate prediction result may include model output corresponding to the sample identifier, such as prediction score, category probability, and other information. This intermediate prediction result can be sent to the agent platform 20 without leaking the original input data.

[0054] After receiving the intermediate prediction results returned by all data source nodes 30 participating in the modeling task, the model building agent 22 in the agent platform 20 can fuse the intermediate prediction results to generate a final prediction result that covers the entire task. For example, there are three data source nodes 30 participating in the modeling task. Data source node 1 sends intermediate prediction result 1 for sample identifier "001" to the agent platform 20, data source node 2 sends intermediate prediction result 2 for sample identifier "001" to the agent platform 20, and data source node 3 sends intermediate prediction result 3 for sample identifier "001" to the agent platform 20. The model building agent 22 fuses intermediate prediction results 1, intermediate prediction results 2, and intermediate prediction results 3 to obtain the final prediction result corresponding to sample identifier "001".

[0055] The model building agent 22 further compares and analyzes the final prediction result with the supervised label uploaded by the user terminal 10, with the optimization goal of minimizing the error between the final prediction result and the true label under the same sample identification. It generates first optimization information and feeds it back to the data source node 30. The first optimization information may include gradient information for model parameters, update strategies, or other optimization instructions that can be used by each data source node 30 to perform local model fine-tuning.

[0056] After receiving the first optimization information, each data source node 30 optimizes the parameters of the prediction model based on the first optimization information. For example, it may incrementally update or fine-tune the parameters of the local prediction model based on the first optimization information, thereby improving the model's fitting ability.

[0057] On the one hand, it ensures that the training process is executed locally on each data source node 30, avoiding the centralized transmission and exposure of the original feature data, and meeting the high requirements for data security and privacy compliance; on the other hand, through the unified fusion of the intermediate prediction results and the global optimization based on the supervision label by the model building agent 22, the entire system can realize distributed collaborative learning in multi-source heterogeneous data scenarios, significantly improving the generalization ability and prediction accuracy of the model; in addition, the generation and cyclic distribution of the first optimization information enables the model training to be updated for multiple rounds of iterative updates based on joint feedback across nodes, realizing the federated incremental optimization of model parameters between multiple data sources, further enhancing the sustainable learning ability and adaptability of the system.

[0058] The following is an exemplary description of the process of fusing the intermediate prediction results by the model building agent 22: In one possible implementation, the model building agent 22 may fuse the intermediate prediction results returned by each data source node 30 based on a preset fusion rule to generate a final prediction result that covers the global sample identification. The preset fusion rule may include, but is not limited to: (1) weighted averaging, which sets different weights based on the reliability or sample coverage of the model of each data source node 30 to achieve differentiated aggregation of prediction results; (2) voting mechanism, which determines the final prediction category through majority voting or confidence voting in classification scenarios; (3) distributed aggregation, which can adopt a decentralized aggregation strategy for regression scenarios or multi-task scenarios to further enhance privacy security.

[0059] This approach is simple, flexible, and can be quickly integrated. It is suitable for multi-data source joint modeling scenarios with high real-time or scalability requirements, ensuring the controllability and interpretability of the model fusion process.

[0060] In another possible implementation, model-building agent 22 may input all intermediate prediction results into a predefined fusion model to be trained. This fusion model then performs a deep learning-based nonlinear combination of the intermediate prediction results from multiple sources to generate a more optimal final prediction result. Model-building agent 22 generates the aforementioned first optimization information and second optimization information for the fusion model, with the goal of minimizing the error between the final prediction result and the supervisory label. The second optimization information is then used to optimize the parameters of the fusion model.

[0061] This method can fully explore the complementarity and nonlinear relationship between the models of different data source nodes 30, and is suitable for multi-stage modeling tasks with high fusion accuracy requirements in complex scenarios, which can significantly improve the prediction performance and generalizability of the system.

[0062] In another possible implementation, the model-building agent 22 can input all intermediate prediction results into a language model (e.g., a large model or a dedicated generative reasoning model). The language model then adaptively parses, weights, and semantically fuses the intermediate prediction results, automatically outputting the final prediction result. This approach leverages the language model's contextual understanding and generative reasoning capabilities. This approach allows for flexible combination of multi-source results when there are no clear fusion rules or when a fixed model structure cannot be predefined. Furthermore, the language model can automatically generate explanatory analysis reports of the prediction results, enhancing the interpretability of the results and reducing the cost of subsequent manual analysis. This approach is suitable for scenarios requiring high flexibility and interpretability.

[0063] In some embodiments, after the prediction model training in the data source node 30 is completed, the model evaluation agent 23 in the agent platform 20 may perform a subsequent evaluation process. For example, the model evaluation agent 23 may perform security verification and / or correctness verification on the trained prediction model in each data source node 30, and analyze the verification results to generate a performance evaluation report.

[0064] By introducing the model evaluation agent 23, the system not only ensures the multi-dimensional security and correctness of the prediction model after collaborative training with multiple data sources, significantly reducing potential risks associated with sensitive information leakage or structural flaws, but also helps promptly identify issues such as model accuracy deviations and insufficient generalization capabilities. Furthermore, performance evaluation reports serve as an important basis for subsequent model parameter tuning, retraining, or fusion strategy optimization, forming an automated process from training to evaluation and then to closed-loop optimization.

[0065] The security check includes at least one of the following: (1) Sensitivity check of model parameters or output content: The weight parameters of the prediction model, the output of each intermediate layer during the training process, and the final prediction output can be automatically scanned to detect whether there is a potential leakage risk containing user identity information, original input values ​​or other preset sensitive data, so as to avoid the leakage of sensitive information when it flows across nodes or is output.

[0066] (2) Model structure security check: The network structure, callable external modules, script logic, etc. of the prediction model can be analyzed to prevent security vulnerabilities caused by the introduction of third-party components or unexpected logic.

[0067] (3) Data transmission link security check during training and inference phases: used to verify whether the data interaction between the data source node 30 and each intelligent agent during training or inference complies with security requirements such as encrypted transmission and minimum available access rights, so as to prevent data from being stolen or tampered with during transmission.

[0068] Correctness verification includes at least one of the following: (1) Cross-validation: By splitting the training data, performing multiple rounds of training and validation on the existing training set, the generalization ability of the model is quantified to identify possible overfitting problems.

[0069] (2) Accuracy index calculation: Various accuracy or robustness indicators such as precision, recall, F1-score, AUC (Area Under the Curve) can be calculated to quantify the prediction effect of the model under the target task.

[0070] (3) Verification of result interpretability: Based on the modeling requirement information input by the user (such as service scenario and expected explanation granularity), verify whether the prediction results conform to the service logic and whether they can be reasonably explained through interpretability methods (such as feature importance ranking, Shapley value, etc.), thereby improving the understandability of the results and user trust.

[0071] In a possible implementation, the model evaluation agent 23 is specifically configured to input the verification results and a preset report template into a language model, so that the language model analyzes the verification results and generates a performance evaluation report in combination with the report template.

[0072] For example, the model evaluation agent 23 can input various verification results and preset report templates into the language model. The language model can combine the preset report templates, analysis rules and context prompts to summarize, conclude and explain the original verification results of security verification and correctness verification. For example: (1) For the security verification part, the language model can automatically analyze the detected sensitive information leakage risks, structural security risks or data transmission vulnerabilities, and output the risk level, possible causes and recommended repair measures in a more readable natural language; (2) For the correctness verification part, the language model can compare multiple accuracy indicators, cross-validation results, etc., summarize the advantages and disadvantages of the model, and provide targeted improvement directions (such as recommending a better hyperparameter combination, adding training samples, replacing modeling algorithms, etc.); (3) For interpretability verification, the language model can automatically generate visual prompts or interpretable summaries based on the verification results, making it easier for users to understand why the model makes the current prediction.

[0073] By inputting validation results into a language model, performance evaluation reports are automatically generated. This significantly reduces the time and expertise required for manual analysis of validation results, enabling even non-technical personnel to clearly understand model quality. Furthermore, the language model automatically identifies anomalies, potential risks, and optimization opportunities based on context, providing actionable improvement suggestions for subsequent model tuning and retraining. Consequently, the multi-data source modeling system, supported by an integrated language model, achieves an automated closed-loop from model validation to report generation. While ensuring model security and accuracy, it further enhances the ease of use and flexibility of external output, fully leveraging the intelligent and automated advantages of the Agent Platform 20.

[0074] In some embodiments, see Figure 4A The multi-data source modeling system also includes a multi-source engine 40, which serves as a coordination and scheduling center for multiple data source nodes 30. It is a key component for achieving efficient collaboration and data linkage between each data source node 30 and the agent platform 20 within the entire system. The agent platform 20 interacts with each data source node 30 through the multi-source engine 40.

[0075] Exemplarily, the multi-source engine 40 may undertake at least one of the following functions: (1) Multi-source coordinated scheduling: The multi-source engine 40 can connect to multiple distributed, heterogeneous data source nodes 30, centrally manage the connection information, access status and access policy of each data source node 30, realize unified orchestration and dynamic scheduling of distributed data source nodes 30, and ensure efficient collaboration of different nodes in sample retrieval, feature query, model training and other links.

[0076] (2) Ability to enrich multi-domain indicator sets: Multi-source engine 40 supports the integration and maintenance of cross-domain indicator systems. Users or intelligent agents can obtain feature metadata from different fields such as finance, medical care, retail, and government affairs based on the multi-source engine 40, which facilitates multi-dimensional modeling and feature expansion, and improves the coverage and interpretability of the model.

[0077] (3) Centralized sample management: The multi-source engine 40 can serve as an index or management unit for the centralized sample warehouse, uniformly maintaining the uploaded sample identification, supervision labels, and their corresponding relationships with each data source node 30, supporting sample version management, access permission control, and cross-node sample consistency verification, thereby improving sample scheduling efficiency and security.

[0078] (4) Task management: The multi-source engine 40 can schedule and track the entire process of modeling tasks, feature query tasks, and model optimization tasks initiated by multiple agents, support task status monitoring, task result archiving, and abnormal alarms, and ensure the orderly process when multiple tasks are performed in parallel.

[0079] (5) Model backtracking: The multi-source engine 40 supports full-process backtracking capabilities for model training, parameter tuning, result fusion, and evaluation. It can automatically record the key operations and generated intermediate / final files of each modeling process, facilitating subsequent review, tracing, and compliance audits.

[0080] (6) One-click modeling: The multi-source engine 40 can be integrated with the "automated modeling pipeline" that cooperates with the intelligent platform 20, supporting an automated closed loop from receiving modeling requirements to sample scheduling, feature processing, model training, result fusion and verification, and realizing support for users to initiate modeling requirements with "low threshold and one-click".

[0081] (7) Model call and configuration management: The multi-source engine 40 can also provide unified call and configuration management services for trained models, support model version control, permission management and visual call configuration, and facilitate rapid reuse in different service scenarios.

[0082] (8) Multi-source fusion strategy management: The multi-source engine 40 can also centrally define and manage the fusion methods of samples and features across data sources during the modeling process, such as supporting the configuration of feature alignment rules, missing value processing strategies, prediction result weighting or voting mechanisms, etc., to improve the flexibility and stability of model fusion.

[0083] By introducing a multi-source engine40 into the multi-data source modeling system, it enables efficient, unified management and secure scheduling of multiple distributed and heterogeneous data sources, significantly reducing the complexity of multi-source data access and scheduling. Furthermore, through centralized management of samples, tasks, models, and fusion strategies, it avoids the consistency and traceability risks brought about by decentralized operations.

[0084] In some embodiments, see Figure 4B The multi-source data modeling system further includes a database 50 , which interfaces with the multi-source engine 40 and serves as one of the core supports for multi-source data scheduling and task management.

[0085] The database 50 can be used to centrally store various file data uploaded by the user terminal 10, such as sample identification files, supervision label files, and modeling files submitted by the user terminal 10 through the intelligent platform 20. These files are the core inputs necessary for the implementation of multi-data source modeling tasks. The centralized storage of the database 50 helps to ensure the consistency, integrity, and efficiency of subsequent access to the data. In addition, the database 50 can also be used to store auxiliary information related to multiple data source nodes 30, such as feature metadata reported by each data source node 30. By establishing a stable docking mechanism between the multi-source engine 40 and the database 50, the required sample identification, label files, modeling files, and feature metadata can be flexibly read or updated at different task stages, and support backtracking, tracking, and management of file versions, thereby improving the data availability, security, and controllability of the entire multi-data source modeling process and reducing the additional overhead caused by repeated uploading and transmission.

[0086] In an exemplary embodiment, see Figure 5 , which shows a sequential interaction diagram in which the user end first uploads the sample identification.

[0087] In step 501, the user terminal uploads the sample identification file to be modeled to the intelligent agent platform through interaction with the intelligent agent platform.

[0088] In step 502, after receiving the sample identification file, the agent platform may perform preprocessing operations on the file content. The preprocessing includes but is not limited to: organizing, deduplicating, formatting, and integrity checking the sample identification content. For example, the feature engineering agent may automatically perform a number of preprocessing tasks related to sample identification by calling preset functions or connecting to internal / external APIs, such as grouping, sharding, batch management, label matching, index generation, standardized records, and status tracking of sample identifications. The preprocessing process can be completed by the feature engineering agent by automatically scheduling the corresponding logical service module based on modeling requirements, without the user having to manually intervene in specific operations.

[0089] In step 503, the intelligent agent platform transmits the pre-processed sample identification file to the multi-source engine.

[0090] In step 504, the multi-source engine performs a storage operation on the received pre-processed sample identification file and saves it in a database to facilitate subsequent unified sample management and task scheduling across data source nodes.

[0091] In step 505, the multi-source engine returns confirmation information to the intelligent agent platform that the sample identification file has been successfully stored.

[0092] In step 506, the intelligent agent platform feeds back a response message of "sample identification file uploaded successfully" to the user end, so that the user can confirm that the sample has been received by the system and enters the subsequent processing flow.

[0093] In an exemplary embodiment, see Figure 6 , which shows the sequential interaction diagram of the user side proposing modeling requirements.

[0094] In step 601, the user terminal sends modeling requirement information described in natural language to the intelligent agent platform, so that the user can make personalized modeling demands in an intuitive way.

[0095] In step 602, the intelligent agent platform generates a candidate feature set based on the modeling requirement information and in combination with the feature metadata obtained from each data source node. The candidate feature set includes multiple candidate data source nodes that meet the modeling requirements and candidate feature fields suitable for modeling in each node.

[0096] In step 603, the intelligent agent platform feeds back the generated candidate feature set to the user end, so that the user can flexibly choose from multiple options.

[0097] In step 604, the user terminal selects at least two data source nodes participating in the modeling task and their respective target feature fields from the candidate feature set according to actual service requirements, and sends the selection results to the intelligent agent platform.

[0098] In step 605, the intelligent agent platform sends the data source node and target feature field selected by the user to the multi-source engine, triggering subsequent multi-source data scheduling.

[0099] In step 606, the multi-source engine obtains the previously saved sample identification file from the database to ensure the consistency and accuracy of subsequent sample construction.

[0100] In step 607, the multi-source engine sends the sample identification file and the target feature field to the corresponding data source node, prompting each data source node to perform sample generation based on the local feature database.

[0101] In step 608, the data source node automatically extracts the target feature field value corresponding to the sample identification based on the sample identification file, the target feature field and the local feature database, and constructs the sample to be modeled.

[0102] In step 609, after completing the sample construction, the data source node returns a response message indicating that the sample construction is completed to the multi-source engine.

[0103] In step 610, after the multi-source engine summarizes the sample generation status of each data source node, it returns a response message indicating that the sample construction is completed to the intelligent agent platform.

[0104] In step 611, the intelligent agent platform feeds back the response information of the sample construction completion to the user end, prompting the user that the subsequent modeling steps are ready.

[0105] In an exemplary embodiment, see Figure 7 , which shows a temporal interaction diagram in which the user uploads the modeling file and the supervised label file of the sample to be modeled to realize multi-data source modeling.

[0106] In step 701, the user terminal can upload the modeling file and the supervision label file corresponding to the sample to be modeled to the intelligent agent platform through the interactive interface with the intelligent agent platform for subsequent model initialization, training and verification.

[0107] In step 702, after receiving the modeling file and the supervisory label file, the intelligent agent platform can perform security checks and / or correctness checks on the received files, such as checking whether the file is complete, whether the format meets the preset standards, whether there are sensitive fields, etc., to ensure that the data is compliant and available.

[0108] In step 703, after completing the verification of the files, the intelligent platform sends the verified modeling files and supervision label files to the multi-source engine, which is responsible for the subsequent unified scheduling and distribution.

[0109] In step 704, after receiving the modeling file and the supervisory label file, the multi-source engine stores them in the database to ensure data consistency and traceability.

[0110] In step 705, the multi-source engine sends the modeling file to the selected data source nodes participating in the modeling task.

[0111] In step 706, after receiving the modeling file, each data source node can initialize the prediction model to be trained based on the modeling algorithm and parameter requirements stated in the modeling file, and locally train the prediction model in combination with the previously constructed samples to be modeled to generate intermediate prediction results.

[0112] In step 707, after completing local training, each data source node feeds back the intermediate prediction results to the multi-source engine.

[0113] In step 708, the multi-source engine aggregates the intermediate prediction results collected from each data source node and sends them to the intelligent agent platform.

[0114] In step 709, the intelligent agent platform performs fusion processing on the intermediate prediction results returned by all data source nodes to obtain the final prediction result, and takes minimizing the error between the final prediction result and the true supervision label under the same sample identification as the optimization goal, and generates the first optimization information to guide the further optimization of subsequent model parameters.

[0115] In step 710 , the agent platform sends the first optimization information to the multi-source engine.

[0116] In step 711, the multi-source engine sends the first optimization information to each data source node participating in the modeling task.

[0117] In step 712, after receiving the first optimization information, each data source node may iteratively optimize the local prediction model parameters based on the first optimization information to improve the accuracy and generalization ability of the model.

[0118] It is understandable that steps 707 to 712 may be repeated multiple times until the iteration end condition is met, such as reaching a preset number of iterations or the performance indicators of the prediction model meet preset requirements, etc. This embodiment does not impose any restrictions on this.

[0119] In step 713, after each data source node completes the optimization and training of the local prediction model, it sends a training completion response to the multi-source engine for synchronization status.

[0120] In step 714, after receiving the training completion response from each data source node, the multi-source engine feeds back the training completion response to the intelligent agent platform.

[0121] In step 715, the intelligent agent platform may perform security checks and / or correctness verification on the trained prediction models in each data source node, such as checking whether the model contains sensitive information and whether it meets output expectations, and conduct a comprehensive analysis of the verification results to generate a performance evaluation report.

[0122] In step 716, the intelligent agent platform feeds back the performance evaluation report to the user end, so that the user can intuitively understand the training effect and compliance of the final model, thereby supporting subsequent operations such as online launch, application or re-tuning.

[0123] In an exemplary application scenario, the multi-data source modeling system provided in this specification can be applied to a financial credit risk prediction scenario. The multi-source data involved in the example include internal bank account flow data, credit repayment records of cooperative banks, and user transaction behavior data of third-party e-commerce platforms.

[0124] Specifically, a bank, as a modeling demander, can submit modeling demand information described in natural language to the intelligent agent platform through the user end, for example: "Predict the default risk of this batch of loan applicants, and introduce e-commerce consumption behavior and credit records of other banks." At the same time, the user end uploads the sample identifier (such as user ID) of the sample to be modeled, as well as optional supervision labels (such as past default marks).

[0125] Upon receiving a request, the feature engineering agent within the agent platform automatically retrieves feature metadata from multiple data sources (e.g., e-commerce platforms, partner banks, and the bank's own data center). Based on the modeling requirements, the agent automatically generates a candidate feature set, either automatically or by invoking a language model. This candidate feature set may include candidate feature fields and corresponding data source nodes, such as "average monthly number of e-commerce transactions," "overdue records at other banks over the past six months," and "volatility of the bank's salary flow over the past three months." The candidate feature set is fed back to the user, who then selects it, or the agent platform automatically determines the target feature fields and data source nodes.

[0126] After determining the target feature field, the agent platform sends the selected target feature field and sample identifier to the multi-source engine, which dispatches it to the corresponding data source node. Each data source node automatically constructs the corresponding sample to be modeled based on the matching feature value in its local feature database and the sample identifier.

[0127] After sample preparation is complete, the user can upload the modeling file (e.g., including the modeling algorithm structure, hyperparameter configuration, etc.) and the supervision label file for this modeling. The intelligent agent platform receives and verifies the data, and distributes it to each participating data source node through the multi-source engine. Based on the modeling file, the data source node initializes its own prediction model to be trained and performs local model training based on the locally generated sample to be modeled. The intermediate prediction results are uploaded to the multi-source engine, which then aggregates and transmits them back to the intelligent agent platform.

[0128] The model-building agent in the agent platform generates the final prediction result by weighted averaging, voting, or fusion model reasoning on the intermediate prediction results uploaded by each data source node. It then generates optimization information with the goal of minimizing the error between the prediction result and the supervisory label. This optimization information is transmitted back to each data source node via the multi-source engine, where it is used to further update the local prediction model parameters.

[0129] After model training is completed, the intelligent agent platform can use the model evaluation agent to perform security verification (such as checking whether it contains sensitive information) and correctness verification (such as cross-validation, accuracy or AUC, etc.) on the prediction model of each data source node, and generate a performance evaluation report, which is finally fed back to the user end to assist the user in making comprehensive judgments on the model effect and deployment decisions.

[0130] Through this multi-data source modeling process, it demonstrates how this system can fully mobilize distributed multi-source data to complete efficient joint modeling while protecting the data privacy of all parties, significantly improving the model's coverage and prediction accuracy. The operation process is more intuitive and automated for users, effectively lowering the modeling threshold and meeting the high standards for risk control in financial scenarios.

[0131] The various technical features in the above embodiments can be combined arbitrarily as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, the arbitrary combination of the various technical features in the above embodiments also falls within the scope of disclosure of this specification.

[0132] In some embodiments, see Figure 8 The embodiment of this specification also provides a multi-data source modeling method applied to an intelligent agent platform, the method comprising: In S801 , modeling requirement information described in natural language, sample identifiers of samples to be modeled, and supervision labels are received from a user terminal.

[0133] In S802, based on the modeling requirement information and the feature metadata obtained from the data source node, at least two data source nodes to participate in the modeling task and their respective target feature fields are determined, and the sample identifier and target feature field are sent to at least two data source nodes, so that the data source nodes construct the sample to be modeled corresponding to the sample identifier and perform model training.

[0134] In S803, the intermediate prediction results sent by the data source node during the model training process are received.

[0135] In S804, the intermediate prediction results are fused to generate a final prediction result, and the first optimization information is generated and sent to the data source node with minimizing the error between the final prediction result and the supervision label as the optimization goal, so that the data source node optimizes the model parameters based on the first optimization information.

[0136] Exemplarily, based on modeling requirement information and feature metadata obtained from the data source nodes, at least two data source nodes to be involved in the modeling task and their respective target feature fields are determined, including: generating modeling prompt information based on the modeling requirement information and feature metadata, and inputting the modeling prompt information into the language model so that the language model generates a candidate feature set, the candidate feature set including multiple candidate data source nodes that meet the modeling requirements and candidate feature fields suitable for modeling in each candidate data source node screened by the language model; from the candidate feature set, determining at least two data source nodes to be involved in the modeling task and the target feature fields suitable for modeling.

[0137] Exemplarily, determining at least two data source nodes participating in the modeling task and target feature fields suitable for modeling from a candidate feature set includes: sending the candidate feature set to a user terminal, and receiving at least two data source nodes participating in the modeling task and their respective target feature fields selected by the user from the user terminal.

[0138] Exemplarily, the candidate feature set also includes a feature processing strategy for the candidate feature field. The method further includes: sending the feature processing strategy for the target feature field to at least two data source nodes.

[0139] Exemplarily, the method further includes: obtaining a modeling file and sending the file to at least two data source nodes to participate in the modeling task.

[0140] Exemplarily, obtaining a modeling file includes: (1) receiving a modeling file uploaded by a user; or, (2) inputting modeling requirement information and a model generation template into a language model, so that the language model determines an adapted modeling algorithm by analyzing the modeling requirement information, and generates a modeling file based on the modeling algorithm and the model generation template; or, (3) inputting modeling requirement information into a language model, so that the language model determines an adapted modeling algorithm by analyzing the modeling requirement information; obtaining a corresponding modeling file from a modeling file library based on the modeling algorithm determined by the language model, where the modeling file library is used to store modeling files corresponding to different modeling algorithms.

[0141] Exemplarily, the intermediate prediction results are fused to generate a final prediction result, and the optimization goal is to minimize the error between the final prediction result and the supervisory label, and the first optimization information is generated and sent to the data source node, including: inputting the intermediate prediction results into the fusion model to be trained to obtain the final prediction result, and the optimization goal is to minimize the error between the final prediction result and the supervisory label, generating the first optimization information and the second optimization information, sending the first optimization information to the data source node, and using the second optimization information to optimize the parameters of the fusion model.

[0142] Exemplarily, fusing the intermediate prediction results to generate the final prediction result includes: inputting the intermediate prediction results into a language model, so that the language model fuses the intermediate prediction results to generate the final prediction result.

[0143] Exemplarily, the method further includes: performing security checks and / or correctness verification on the trained prediction models in each data source node, and analyzing the verification results to generate a performance evaluation report; wherein, the security check includes at least one of the following: sensitivity check of model parameters or output content; model structure security check; data transmission link security check during training and inference phases; correctness verification includes at least one of the following: cross-validation, accuracy index calculation, and result interpretability verification.

[0144] Exemplarily, analyzing the verification results to generate a performance evaluation report includes: inputting the verification results and a preset report template into a language model, so that the language model analyzes the verification results, and generates a performance evaluation report in combination with the report template.

[0145] In some embodiments, see Figure 9 , the embodiment of this specification also provides a multi-data source modeling method, applied to a data source node, the method comprising: In S901, the sample identification and target feature field sent by the intelligent agent platform are received.

[0146] In S902 , based on the sample identifier, the target feature field and the local feature database, a sample to be modeled corresponding to the sample identifier is constructed.

[0147] In S903, the prediction model to be trained is trained using the samples to be modeled, and an intermediate prediction result is generated and sent to the intelligent agent platform.

[0148] In S904, first optimization information returned by the intelligent agent platform based on the intermediate prediction result is received, and the parameters of the prediction model are optimized based on the first optimization information.

[0149] Exemplarily, based on the sample identifier, the target feature field and the local feature database, a sample to be modeled corresponding to the sample identifier is constructed, including: receiving a feature processing strategy for the target feature field sent by the intelligent platform, querying the target feature field value corresponding to the sample identifier from the feature database based on the target feature field and the sample identifier, and processing the target feature field value based on the feature processing strategy to generate a sample to be modeled corresponding to the sample identifier.

[0150] Exemplarily, before using the samples to be modeled to train the prediction model to be trained, the method further includes: receiving a modeling file sent by the intelligent agent platform, initializing based on the modeling file, and obtaining the prediction model to be trained.

[0151] In some embodiments, Figure 10 This is a schematic structural diagram of a device provided by an exemplary embodiment. Figure 10 As shown, device 1000 primarily consists of a communication interface 1002, a user interface 1004, a processor 1006, and data storage 1008. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 1010. Communication interface 1002 enables device 1000 to communicate with other devices, access networks, and transmission networks using analog or digital modulation. For example, communication interface 1002 may include a chipset and antenna for wireless communication with a radio access network or access point. Communication interface 1002 may also be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, a Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, communication interface 1002 may also support other physical layer interfaces and standard or proprietary communication protocols. Communication interface 1002 may also include multiple physical communication interfaces, such as a Wi-Fi interface, a Bluetooth interface, and a wide-area wireless interface.

[0152] The user interface 1004 includes receiving user input and providing output to the user. Therefore, the user interface 1004 can include input components such as a keypad, keyboard, touch-sensitive or presence-sensing panel, computer mouse, trackball, joystick, microphone, still camera and video camera, and can also include output components such as a display screen (which can be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, other known or future developed similar devices. The user interface 1004 can also generate auditory output through speakers, speaker jacks, audio output ports, audio output devices, headphones, and other known or future developed similar devices. In certain embodiments, the user interface 1004 can include software, circuitry, or other forms of logic capable of transmitting data to and receiving data from external user input / output devices. In addition or alternatively, the device 1000 can support remote access from other devices via the communication interface 1002 or another physical interface (not shown). The user interface 1004 can be configured to receive user input, and its position and movement can be indicated by an indicator or cursor as described herein. The user interface 1004 may also be configured as a display device for rendering or displaying text snippets.

[0153] Processor 1006 may include one or more general-purpose processors and / or special-purpose processors.

[0154] The data store 1008 may include one or more volatile and / or non-volatile storage components and may be integrated in whole or in part with the processor 1006. The data store 1008 may include removable and non-removable components.

[0155] Processor 1006 is capable of executing program instructions 1018 (e.g., compiled or non-compiled program logic and / or machine code) stored in data storage 1008 to implement the various functions described herein. Data storage 1008 may include a non-transitory computer-readable medium having program instructions stored thereon that, when executed by device 1000, enable device 1000 to perform any of the methods, processes, or functions disclosed in this specification and / or the accompanying figures. Execution of program instructions 1018 by processor 1006 may result in processor 1006 using data 1012.

[0156] For example, program instructions 1018 may include an operating system 1022 (e.g., an operating system kernel, device drivers, and / or other modules) and one or more applications 1020 (e.g., a browser, a social application, or a gaming application) installed on device 1000. Similarly, data 1012 may include operating system data 1016 and application data 1014. Operating system data 1016 is primarily accessible to operating system 1022, while application data 1014 is primarily accessible to one or more applications 1020. Application data 1014 may be located in a file system that is visible or hidden to a user of device 1000.

[0157] Applications 1020 can communicate with operating system 1022 through one or more application programming interfaces (APIs). These APIs facilitate application 1020 reading and / or writing application data 1014, transmitting or receiving information via communication interface 1002, receiving or displaying information on user interface 1004, and the like.

[0158] In some terminology, application 1020 may be simply referred to as an "app." Furthermore, application 1020 may be downloaded to device 1000 through one or more online application stores or marketplaces. However, applications may also be installed on device 1000 through other means, such as through a web browser or a physical interface on device 1000 (e.g., a USB port).

[0159] For example, the multi-data source modeling apparatus can be applied to Figure 10 The device shown in the figure is used to implement the technical solution of this specification. The multi-data source modeling device may include: The receiving module is used to receive modeling requirement information described in natural language, sample identification and supervision label of the sample to be modeled sent by the user end.

[0160] The data processing and sending module is used to determine at least two data source nodes to participate in the modeling task and their respective target feature fields based on the modeling requirement information and the feature metadata obtained from the data source nodes, and send the sample identification and target feature field to at least two data source nodes, so that the data source nodes construct the samples to be modeled corresponding to the sample identification and perform model training.

[0161] The receiving module is also used to receive the intermediate prediction results sent by the data source node during the model training process.

[0162] The data processing and sending module is also used to fuse the intermediate prediction results to generate the final prediction result, and to minimize the error between the final prediction result and the supervision label as the optimization goal, generate the first optimization information and send it to the data source node, so that the data source node optimizes the model parameters based on the first optimization information.

[0163] For example, the multi-data source modeling apparatus can be applied to Figure 10 The device shown in the figure is used to implement the technical solution of this specification. The multi-data source modeling device may include: The receiving module is used to receive the sample identification and target feature fields sent by the intelligent agent platform.

[0164] The sample construction module is used to construct the sample to be modeled corresponding to the sample identifier based on the sample identifier, target feature field and local feature database.

[0165] The model training module is used to train the prediction model to be trained using the samples to be modeled, generate intermediate prediction results and send them to the intelligent agent platform.

[0166] The model optimization module is used to receive the first optimization information returned by the intelligent agent platform based on the intermediate prediction results, and optimize the parameters of the prediction model based on the first optimization information.

[0167] For the convenience of description, the above devices are described as being functionally divided into various modules or units. Of course, when implementing one or more of the present specifications, the functions of each module or unit can be implemented in the same or multiple software and / or hardware, or the module that implements the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of the units is merely a logical functional division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0168] Based on the same concept as the above method, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor implements the steps of the method described in any of the above embodiments by running the executable instructions.

[0169] Based on the same concept as the above method, this specification also provides a computer-readable storage medium on which computer instructions are stored. When the instructions are executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0170] Based on the same concept as the above method, this specification also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any of the above embodiments when executed by a processor.

[0171] Those skilled in the art will understand that: In this specification, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, product, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, product, or apparatus. In the absence of further limitations, it does not preclude the presence of additional identical or equivalent elements in a process, method, product, or apparatus that includes the elements.

[0172] In this specification, "a", "an" and "the" do not specifically refer to the singular but may also include the plural.

[0173] In this specification, ordinal numbers such as "first" and "second" do not necessarily indicate order; they are often used to distinguish objects. For example, "server 1" and "server 2" typically refer to two servers. To distinguish these two servers, they are referred to as "server 1" and "server 2." Of course, these two servers may also be the same server.

[0174] In this specification, unless explicitly stated otherwise, "data reception and transmission" does not necessarily refer to direct reception and transmission; it can be indirect. For example, when A receives data from B, it can be understood as A directly receiving the data from B, or it can be understood as A indirectly receiving the data from B through another entity, such as C. Similarly, when B sends data to A, it can be understood as B sending the data directly to A, or it can be understood as B indirectly sending the data to A through another entity, such as C. Here, C can be a single entity, or two or more entities.

[0175] In this specification, unless explicitly stated otherwise, the associations between structures can be direct or indirect. For example, when describing "A is connected to B," unless it is explicitly stated that A and B are directly connected, it should be understood that A can be directly connected to B or indirectly connected to B. For another example, when describing "A is above B," unless it is explicitly stated that A is directly above B (AB is adjacent and A is above B), it should be understood that A can be directly above B or indirectly above B (AB is separated by other elements and A is above B). And so on.

[0176] This specification uses specific terms to describe the embodiments of this specification. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a certain feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment," "an embodiment," or "an alternative embodiment" mentioned twice or more in different places in this specification does not necessarily refer to the same embodiment. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, unless they are mutually inconsistent.

[0177] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understandable that the order of steps listed in the embodiments or flowcharts is only one way of executing the steps among many orders, and does not represent the only order of execution. Therefore, when the claims involve method steps, changes and adjustments to the order of such steps, or parallelism between steps are also within the scope of protection of the claims.

Claims

1. A multi-data source modeling system, including a user terminal, an intelligent agent platform, and multiple data source nodes; The user terminal is used to send modeling requirement information described in natural language, sample identification and supervision label of the sample to be modeled to the intelligent agent platform; The agent platform is used to determine at least two data source nodes to participate in the modeling task and their respective target feature fields based on the modeling requirement information and the feature metadata obtained from the data source nodes, and send the sample identifier and the target feature field to the at least two data source nodes; The data source node is used to construct a sample to be modeled corresponding to the sample identifier based on the sample identifier, the target feature field and the local feature database, and use the sample to be modeled to train the prediction model to be trained, generate an intermediate prediction result and send it to the intelligent agent platform; The intelligent agent platform is further configured to fuse the intermediate prediction results to generate a final prediction result, and to generate first optimization information with minimizing the error between the final prediction result and the supervision label as an optimization goal, and to send the first optimization information to the data source node; The data source node is further used to optimize the parameters of the prediction model based on the first optimization information.

2. The system according to claim 1, wherein the agent platform comprises a feature engineering agent; The feature engineering agent is used to generate modeling prompt information based on the modeling requirement information and the feature metadata, and input the modeling prompt information into the language model so that the language model generates a candidate feature set, wherein the candidate feature set includes a plurality of candidate data source nodes that meet the modeling requirements and are screened by the language model, and candidate feature fields suitable for modeling in each candidate data source node; At least two data source nodes participating in the modeling task and target feature fields suitable for modeling are determined from the candidate feature set.

3. The system according to claim 2, wherein the feature engineering agent is specifically configured to send the candidate feature set to the user terminal; the user terminal is further configured to display the candidate feature set and send at least two data source nodes selected by the user for participating in the modeling task and their respective target feature fields to the feature engineering agent; and / or, The candidate feature set also includes a feature processing strategy for the candidate feature field; the feature engineering agent is also used to send the feature processing strategy for the target feature field to the at least two data source nodes; the data source node is specifically used to query the target feature field value corresponding to the sample identifier from the feature database based on the target feature field and the sample identifier, and process the target feature field value based on the feature processing strategy to generate a sample to be modeled corresponding to the sample identifier.

4. The system according to claim 1, wherein the agent platform comprises a model building agent; The model building agent is used to obtain a modeling file and send it to at least two data source nodes to participate in the modeling task; The data source node is further used to perform initialization based on the modeling file to obtain the prediction model to be trained.

5. The system according to claim 4, wherein the user terminal is further configured to upload the modeling file to the agent platform; Alternatively, the model building agent is further configured to input the modeling requirement information and the model generation template into a language model, so that the language model determines an adapted modeling algorithm by analyzing the modeling requirement information, and generates the modeling file based on the modeling algorithm and the model generation template; Alternatively, the model building agent is further configured to input the modeling requirement information into a language model, so that the language model determines an adapted modeling algorithm by analyzing the modeling requirement information; The modeling algorithm determined based on the language model obtains the corresponding modeling file from a modeling file library, and the modeling file library is used to store modeling files corresponding to different modeling algorithms.

6. The system according to claim 4, wherein the model building agent is further configured to input the intermediate prediction result into the fusion model to be trained to obtain a final prediction result, and to generate second optimization information with the optimization goal of minimizing the error between the final prediction result and the supervision label, and to optimize the parameters of the fusion model using the second optimization information; Alternatively, the model building agent is further configured to input the intermediate prediction results into a language model, so that the language model fuses the intermediate prediction results to generate a final prediction result; Alternatively, the model building agent is further used to perform fusion processing on the intermediate prediction results returned by the data source nodes based on preset fusion rules to generate a final prediction result.

7. The system according to claim 1, wherein the agent platform includes a model evaluation agent; The model evaluation agent is used to perform security verification and / or correctness verification on the trained prediction model in each of the data source nodes, and analyze the verification results to generate a performance evaluation report; in, The security check includes at least one of the following: sensitivity check of model parameters or output content; Model structure safety check; Security check of data transmission links during training and inference phases; The correctness verification includes at least one of the following: cross-validation, accuracy indicator calculation, and result interpretability verification.

8. According to the system of claim 7, the model evaluation agent is specifically used to input the verification results and a preset report template into a language model, so that the language model analyzes the verification results and generates a performance evaluation report in combination with the report template.

9. The system according to claim 1 further comprises a multi-source engine, which is used to serve as a coordination and dispatching center for the multiple data source nodes; the intelligent platform interacts with each of the data source nodes through the multi-source engine.

10. A multi-data source modeling method, applied to an agent platform in a multi-data source modeling system according to any one of claims 1 to 9, the method comprising: Receive modeling requirement information described in natural language, sample identification and supervision label of the sample to be modeled sent by the user; Based on the modeling requirement information and the feature metadata obtained from the data source node, at least two data source nodes to be involved in the modeling task and their respective target feature fields are determined, and the sample identifier and the target feature field are sent to the at least two data source nodes, so that the data source nodes construct the sample to be modeled corresponding to the sample identifier and perform model training; Receiving the intermediate prediction results sent by the data source node during the model training process; The intermediate prediction results are fused to generate a final prediction result, and the optimization goal is to minimize the error between the final prediction result and the supervisory label, generate first optimization information and send it to the data source node, so that the data source node optimizes the model parameters based on the first optimization information.

11. A multi-data source modeling method, applied to a data source node in a multi-data source modeling system according to any one of claims 1 to 9, the method comprising: Receive the sample identification and target feature fields sent by the intelligent agent platform; Based on the sample identifier, the target feature field and the local feature database, construct a sample to be modeled corresponding to the sample identifier; Using the to-be-modeled samples to train the prediction model to be trained, generating an intermediate prediction result and sending it to the intelligent agent platform; Receive first optimization information returned by the intelligent agent platform based on the intermediate prediction result, and optimize the parameters of the prediction model based on the first optimization information.

12. An electronic device, characterized in that: include: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method according to claim 10 or 11 by running the executable instructions.

13. A computer-readable storage medium, characterized in that Computer instructions are stored thereon, and when the instructions are executed by a processor, the steps of the method according to any one of claims 10 or 11 are implemented.

14. A computer program product, characterized in that The method comprises a computer program / instruction which, when executed by a processor, implements the steps of the method according to any one of claims 10 or 11.

Images