Model fusion method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a model fusion method and device, electronic equipment and a storage medium. The method comprises the steps that at least two participants participating in financial scene model fusion are determined, and data between the at least two participants are mutually isolated; for each participant, acquiring a student model uploaded by the participant, the student model being obtained by performing knowledge distillation according to a teacher model of the participant, and the teacher model being obtained by performing model training on a to-be-trained model according to a data set of the participant; and according to the student model of each participant, fusion processing is performed on all the student models to obtain a technical means of a global model, so that the data security is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model fusion method, device, electronic device and storage medium. Background Art

[0002] Existing large-model training methods in the financial field usually rely on centralized data and computing resources, which poses significant challenges in terms of privacy protection and data security. Data in the financial field is usually highly sensitive and confidential, including personal account information, transaction records, loan applications, credit scores, etc. The traditional centralized training method requires all data to be concentrated on a server in a bureau for training, which is prone to data leakage and privacy infringement. Summary of the invention

[0003] The embodiments of the present application provide a model fusion method, device, electronic device and storage medium to achieve data security.

[0004] In a first aspect, an embodiment of the present application provides a model fusion method, including:

[0005] Determine at least two parties involved in the integration of financial scenario models, and isolate the data between at least two parties;

[0006] For each participant, obtain the student model uploaded by the participant. The student model is obtained by performing knowledge distillation based on the participant's teacher model. The teacher model is obtained by training the to-be-trained model based on the participant's data set.

[0007] According to the student model of each participant, all student models are fused to obtain the global model.

[0008] In a second aspect, an embodiment of the present application provides a model fusion device, including: a determination module, used to determine at least two participants participating in the fusion of the financial scenario model, and the data between the at least two participants are isolated from each other;

[0009] An acquisition module is used to acquire, for each participant, a student model transmitted by the participant, where the student model is obtained by performing knowledge distillation based on the participant's teacher model, and the teacher model is obtained by training the model to be trained based on the participant's data set;

[0010] The fusion module is used to fuse all student models according to the student model of each participant to obtain a global model.

[0011] In a third aspect, an embodiment of the present application provides a model fusion device, including: a memory, a processor;

[0012] Memory stores computer-executable instructions;

[0013] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0015] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] Figure 1 A schematic diagram of a scenario of the model fusion method provided in this application;

[0018] Figure 2 Schematic diagram of the model fusion method provided in this application Figure 1 ;

[0019] Figure 3 Schematic diagram of the model fusion method provided in this application Figure 2 ;

[0020] Figure 4 A schematic diagram of the structure of the model fusion device provided in this application;

[0021] Figure 5 A schematic diagram of the structure of the model fusion device provided in this application.

[0022] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0023] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0024] Figure 1 A schematic diagram of a scenario of the model fusion method provided in this application, such as Figure 1 As shown, the specific application scenario of the present application includes at least two participants and a server, wherein each participant performs model training on the model to be trained according to the data set of each participant to obtain a teacher model, then performs knowledge distillation on the teacher model to obtain a student model, and finally uploads the student model to the server. The server fuses all student models according to the student models uploaded by each participant to obtain a global model.

[0025] Combined with the above scenarios, it can be seen that existing large-model training methods in the financial field usually rely on centralized data and computing resources, which poses significant challenges in terms of privacy protection and data security. Data in the financial field is usually highly sensitive and confidential, including personal account information, transaction records, loan applications, credit scores, etc. The traditional centralized training method requires all data to be concentrated on a server in a bureau for training, which is prone to data leakage and privacy infringement. At the same time, as the scale of data increases, the centralized training method requires huge computing resources and storage space, which is often difficult to achieve in practical applications.

[0026] In response to at least one of the above technical problems, the model fusion method provided by the present application determines at least two participants participating in the fusion of the financial scenario model, and the data between the at least two participants are isolated from each other; for each participant, a student model uploaded by the participant is obtained, and the student model is obtained by performing knowledge distillation on the teacher model of the participant, and the teacher model is obtained by training the training model based on the data set of the participant; based on the student model of each participant, all student models are fused to obtain a global model, which solves the technical problems of data leakage and privacy infringement.

[0027] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0028] Figure 2 Schematic diagram of the model fusion method provided in this application Figure 1 ,like Figure 2 As shown, the method includes:

[0029] S201. Determine at least two parties that participate in the integration of the financial scenario model, and isolate data between the at least two parties.

[0030] In this embodiment, since the data between the participants in the financial scenario involves personal privacy and is not easy to share, the model fusion method is used for model training. The participants are determined based on the content of the specific model fusion. For example, if it is necessary to jointly train an anti-fraud model with multiple banks, the participating banks can be participants.

[0031] S202. For each participant, obtain a student model uploaded by the participant, the student model is obtained by performing knowledge distillation based on the participant's teacher model, and the teacher model is obtained by training a to-be-trained model based on the participant's data set.

[0032] In this embodiment, the model to be trained may refer to a large model, and knowledge distillation may refer to the ability to achieve lightweight and efficient reasoning of the model by migrating the knowledge of the large model to the small model. The large model is used as the teacher model, and the small model is used as the student model. The output of the teacher model is approximated by training the student model, so that the small model can significantly reduce the consumption of computing resources while maintaining high accuracy. In the financial field, large models usually require powerful computing resources and storage space. The small model after knowledge distillation has higher efficiency in the reasoning process and is suitable for financial applications that require real-time response.

[0033] S203: Based on the student model of each participant, all student models are integrated to obtain a global model.

[0034] In this embodiment, after obtaining the student models of each participant, the parameters of the global model are obtained by fusing the parameters of the student models, thereby obtaining the global model. This solves the data privacy and distributed processing problems in the financial field and improves the generalization ability and accuracy of the model.

[0035] For example, assuming that the participants are A, B and C, the main steps may include:

[0036] 1. Data distribution and local training

[0037] Each participant first pre-processes the local data to obtain a local data set, including data cleaning, deduplication, normalization and other operations to ensure data quality.

[0038] Then the local dataset is used for model training to generate preliminary large models M_A, M_B and M_C.

[0039] 2. Local Model Distillation

[0040] The knowledge distillation of the locally trained large models M_A, M_B and M_C is performed to generate lightweight student models m_A, m_B and m_C. The student model is trained locally using knowledge distillation based on intermediate layer features, so that it can learn the knowledge of the teacher model more effectively.

[0041] 3. Model encryption and transmission

[0042] The generated student models m_A, m_B and m_C are encrypted to ensure data security during transmission. The encrypted student models are transmitted to the central server.

[0043] 4. Model fusion and global model update

[0044] The central server uses the optimized federated averaging algorithm on the received student models to generate the global model M_g.

[0045] The updated global model M_g is fed back to each participant, and the participant continues the next round of local training and distillation based on it.

[0046] The model fusion method provided in the embodiment of the present application determines at least two participants participating in the fusion of the financial scenario model, and the data between the at least two participants are isolated from each other; for each participant, a student model uploaded by the participant is obtained, and the student model is obtained by performing knowledge distillation based on the teacher model of the participant, and the teacher model is obtained by training the training model based on the data set of the participant; based on the student model of each participant, all student models are fused to obtain a global model, thereby improving data security.

[0047] Figure 3 Schematic diagram of the model fusion method provided in this application Figure 2 ,like Figure 3 As shown, in this embodiment Figure 2 Based on the embodiment, the model fusion method is described in detail, and the method includes:

[0048] S301. Determine at least two parties involved in the integration of financial scenario models.

[0049] S302: For each participant, obtain the student model uploaded by the participant.

[0050] In this embodiment, for the sake of data security, the student model uploaded by the participant is encrypted data, so the received student model needs to be decrypted, for example, by using algorithms such as AES or RSA.

[0051] Specifically, the student model can be obtained by:

[0052] For each participant, knowledge distillation is performed based on the participant’s teacher model to obtain the initial student model;

[0053] According to the preset disturbance range, the model parameters in the initial student model are differentially privately processed to obtain the student model;

[0054] Upload the student model.

[0055] In this embodiment, the preset disturbance range can ensure that the disturbance of the parameter is within an acceptable range. Differential privacy processing can refer to a mathematical technique that prevents anyone from obtaining information about individuals in a data set by adding controlled randomness to the data set. The core idea is to introduce noise into the data analysis process so that the increase or decrease of any single record will not significantly affect the analysis results, thereby protecting personal privacy. Specifically, in the process of knowledge distillation, the model parameters of the obtained student model are differentially privately processed to further enhance data privacy protection.

[0056] After obtaining the student model, in order to ensure data security, the student model can be encrypted according to a preset encryption algorithm to obtain an encrypted student model, and then the encrypted student model can be uploaded through a secure transmission protocol. For example, the preset encryption algorithm can be AES or RSA, and the secure transmission protocol can be HTTPS.

[0057] In some implementations, the method of obtaining a teacher model may include:

[0058] 1. Data preprocessing

[0059] Step 1: Data cleaning: Through automated data cleaning technology, machine learning algorithms and rule engines are used to automatically detect and process missing values, outliers, and duplicate data to improve data cleaning efficiency and accuracy. Anomaly detection algorithms such as isolation forests are used to perform multi-level and multi-dimensional outlier detection to improve data quality.

[0060] Step 2: Data deduplication: Delete duplicate data, detect and delete duplicate data to avoid data redundancy and model overfitting.

[0061] Step 3: Data normalization: Standardize the data to the same range and convert data from different sources into a unified format for subsequent processing and analysis, including data type conversion, unit conversion, and format unification. Perform necessary transformation and processing on the data and perform data normalization to improve the efficiency and effectiveness of model training.

[0062] Step 4: Feature selection: Select features that have an important impact on model training, extract, generate and select features to enhance the performance of the model, including feature extraction, feature combination, feature selection and dimensionality reduction techniques.

[0063] 2. Local training

[0064] Step 1: Model architecture definition: Define the network structure of the model.

[0065] Step 2: Optimizer selection: Select an optimizer, such as SGD.

[0066] Step 3: Training strategy setting: Set the training hyperparameters (such as learning rate, batch size, etc.), use the adaptive learning rate adjustment algorithm, dynamically adjust the learning rate according to the change of loss during training, and improve the model convergence speed and stability.

[0067] Step 4: Model training: Use local data to train the model and generate a large model. At the same time, use incremental training technology to continue training based on the original model to improve training efficiency and adapt to dynamic changes in data.

[0068] In some implementations, for each participant, performing knowledge distillation based on the participant's teacher model to obtain an initial student model may include:

[0069] For each participant, based on the participant's teacher model, obtain the high-dimensional feature map output by the intermediate layer of the teacher model and the model result output by the output layer;

[0070] The model to be trained is trained according to the high-dimensional feature map, model results and the preset distillation loss function to obtain the initial student model.

[0071] In this embodiment, a smaller model structure is first defined. The model structure should be similar to the teacher's large model structure on the backbone network, so that the feature distillation of the intermediate layer will have a better effect. Then, after initializing the parameters of the small model, a feature-based distillation loss function is defined, and the student model is trained using the distillation loss function.

[0072] The label knowledge distillation method is widely used in the prior art. Although this method is simple, its limitation is also obvious, that is, the label prediction expression information learned by the student model is limited. The embodiment of the present application uses a feature-based knowledge distillation method to perform knowledge distillation training on the student model using a teacher model, and obtains high-level feature knowledge with more representation ability in the middle layer of the teacher model and transfers it to the student model. This method effectively solves the problem of limited knowledge information in the output layer of the teacher model, and can provide the student model with rich middle-layer feature knowledge information, thereby meeting the needs of lightweight small models for complex knowledge. Specifically, knowledge transfer can be achieved by minimizing the maximum average difference metric of the distribution of neuron selective features between the teacher network and the student network. The high-dimensional feature map output by the middle layer of the teacher model and the output of the last layer are used as the knowledge learned by the student model to match the feature activations of the teacher model and the student model. The feature maps corresponding to the teacher model and the student model are matched by the transformation function. The distillation loss is calculated to obtain the implicit knowledge in the teacher network, and the lightweight student model fits the distribution of the teacher model to improve the generalization ability of the student model. In order to reduce the bias of the model and make the model more stable, this embodiment uses a cross entropy loss function to constrain the classifier of the student model.

[0073] Furthermore, the distillation loss function satisfies:

[0074]

[0075]

[0076]

[0077] in, is the distillation loss function, is the cross entropy loss function, is the classifier of the student model, is the transformation function, is the number of intermediate layers, The first The output of the middle layer, For the student model The output of the middle layer.

[0078] In some embodiments,

[0079] S303. For each participant, determine the amount of training data and the data quality score when the participant performs model training based on data quality indicators. The data quality indicators include data distribution consistency check indicators, data noise detection indicators, and data missing rate calculation indicators.

[0080] Among them, the data distribution consistency check indicator can be used to evaluate whether the distribution of the data set conforms to the expected pattern or standard. By checking the distribution consistency of the data, outliers or deviations in the data set can be identified. For example, assuming that a data set should conform to a normal distribution, statistical analysis (such as histograms, QQ plots, etc.) can be used to check whether the actual data deviates from this distribution. This check is crucial to ensure the representativeness of the data and the reliability of the analysis results.

[0081] Data noise in data noise detection metrics can refer to random errors or inaccurate data points in a data set. Noise can come from errors in the data collection process, sensor failures, or human input errors. Data noise detection metrics can be used to identify and quantify these inaccurate data points. Common methods include outlier detection, signal processing techniques (such as filters), and machine learning algorithms (such as anomaly detection models). By identifying and processing noise, the accuracy of data analysis and the performance of the model can be improved.

[0082] Missing data in the missing data rate calculation indicator can refer to the situation where some values ​​in the data set are missing or not recorded. The missing data rate calculation indicator can be used to quantify the proportion or number of missing values ​​in the data set. The calculation method is usually to count the number of missing values ​​for each variable or the entire data set, and then divide it by the total number of data points to obtain the missing rate. Knowing the missing data rate can help data scientists and analysts decide how to deal with missing data, such as filling missing values ​​through interpolation, deletion, or using model predictions. A high missing rate may affect the accuracy of the analysis results, so special attention should be paid.

[0083] S304. Obtain a model performance score of the student model according to the student model and the model performance indicators. The model performance indicators include model accuracy, loss value, and F1-score.

[0084] Among them, model accuracy can be expressed as the ratio of the number of samples predicted correctly by the model to the total number of samples.

[0085] Loss Value can be a key indicator for optimizing the model, usually calculated during the training process. It quantifies the difference between the model prediction and the actual target. Different tasks use different loss functions. For example, in regression problems, a commonly used loss function is the mean squared error (MSE), while in classification problems, the cross-entropy loss is a common choice. The smaller the loss value, the closer the model prediction is to the true value. During the training process, optimization algorithms (such as gradient descent) adjust the model parameters by minimizing the loss value.

[0086] F1-score can be a comprehensive indicator for evaluating the performance of a binary classification model, especially for datasets with imbalanced categories. It is the harmonic mean of precision and recall. Precision indicates the proportion of samples predicted as positive by the model that are actually positive, while recall indicates the proportion of samples that are actually positive that are correctly predicted as positive. F1-score strikes a balance between the two, especially for situations where both false positive and false negative errors need to be considered.

[0087] The federated averaging algorithm in the prior art only performs weighted averaging based on the amount of data of each participant. The basic idea is to upload the parameters of the local model to the server, which calculates the average of all model parameters and then broadcasts this average back to all participants. This process can be iterated multiple times until convergence. However, in federated learning, there may be significant differences in the amount of data, data quality, and teacher model performance of different participants. The weighted averaging method based solely on the amount of data may lead to a decrease in the performance of the global model. In order to better utilize the advantages of each participant, this example proposes a federated averaging algorithm based on data volume, data quality, and model performance. Taking the above factors into consideration, the model parameters are weighted, which can be flexibly configured according to specific task scenarios and task requirements to improve the accuracy and robustness of the global model.

[0088] The embodiment of the present application optimizes the federated average algorithm, dynamically adjusting the model weight of each student model not only according to the data volume of each participant, but also according to the data quality and model performance, thereby improving the accuracy and generalization ability of the global model. The participant with more data volume, higher data quality and higher model performance will have a greater model weight.

[0089] S305. Obtain the model weight of the student model according to the data volume, data quality score and model performance score.

[0090] Specifically, the formula for model weight can be:

[0091]

[0092] in, For the The model weights of the participants, For the The amount of data from each participant, For the The data quality scores of the participants, For the The model performance scores of the participants, is the number of participants, , and is the weight coefficient and satisfies .

[0093] S306. Obtain global model parameters according to each student model and the model weight of each student model.

[0094] Specifically, the formula of the global model parameters can be:

[0095]

[0096] in, are global model parameters, For the The model weights of the participants, For the The model parameters of the student model of each participant.

[0097] S307: Obtain a global model according to the global model parameters.

[0098] In this embodiment, the global model parameters are obtained to obtain the global model, and the fused global model can be subjected to multi-level optimization processing, including local parameter correction, regularization processing, and multi-round model pruning, to further improve the performance and stability of the model.

[0099] In some implementations, the server may encrypt the updated global model and distribute it to each participant, supporting incremental updates and full updates. Specifically, the following steps may be included:

[0100] Step 1: Model encryption: Use AES or RSA algorithm to encrypt the global model.

[0101] Step 2: Model distribution: Distribute the encrypted global model to all participants via HTTPS.

[0102] Step 3: Data integrity check: Use hash verification or digital signature to check the integrity of the global model before and after transmission to ensure that the model has not been tampered with.

[0103] Step 4: Node synchronization: Responsible for the synchronization of nodes of each participant, ensuring that each participant node starts using the new global model at the same time (clock synchronization of each participant node is required).

[0104] Step 5: Feedback and monitoring: Collect abnormal problems encountered by each participant node during the use of the global model, and monitor and analyze the use of the global model.

[0105] The model fusion method provided in the embodiment of the present application is by determining at least two participants who participate in the fusion of the financial scenario model. The student model uploaded by each participant is obtained, and then the data volume and data quality score of the training data when the participant performs model training are determined based on the data quality index, and the model performance score of the student model is obtained based on the student model and the model performance index. Then, the model weight of the student model is obtained based on the data volume, data quality score and model performance score. Then, the global model parameters are obtained based on each student model and the model weight of each student model, and finally, the global model is obtained based on the global model parameters. The technical problems of data leakage and privacy infringement are solved, and the accuracy and generalization ability of the global model are improved.

[0106] Figure 4 A schematic diagram of the structure of the model fusion device provided in this application, such as Figure 4 As shown, the model fusion device 40 provided in this embodiment includes:

[0107] A determination module 401 is used to determine at least two participants who participate in the fusion of the financial scenario model, and the data between the at least two participants are isolated from each other;

[0108] An acquisition module 402 is used to acquire, for each participant, a student model transmitted by the participant, where the student model is obtained by performing knowledge distillation on the teacher model of the participant, where the teacher model is obtained by performing model training on a to-be-trained model based on the data set of the participant;

[0109] The fusion module 403 is used to fuse all student models according to the student model of each participant to obtain a global model.

[0110] In a possible implementation, the fusion module 403 is further configured to:

[0111] For each participant, the amount of training data and the data quality score when the participant conducts model training are determined based on the data quality indicators. The data quality indicators include data distribution consistency check indicators, data noise detection indicators, and data missing rate calculation indicators.

[0112] According to the student model and model performance indicators, the model performance score of the student model is obtained. The model performance indicators include model accuracy, loss value and F1-score;

[0113] According to the data volume, data quality score and model performance score, the model weight of the student model is obtained;

[0114] According to each student model and the model weight of each student model, the global model parameters are obtained;

[0115] According to the global model parameters, a global model is obtained.

[0116] In a possible implementation, the fusion module 403 is further configured to:

[0117] satisfy:

[0118]

[0119]

[0120] in, are global model parameters, For the The model weights of the participants, For the The model parameters of the student model of each participant, For the The amount of data from each participant, For the The data quality scores of the participants, For the The model performance scores of the participants, is the number of participants, , and is the weight coefficient and satisfies .

[0121] The model fusion device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effects are similar, and will not be described in detail in this embodiment.

[0122] Figure 5 This is a schematic diagram of the structure of the model fusion device provided in this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the device 50 also includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected via a bus 504.

[0123] In a specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that at least one processor 501 executes the above method.

[0124] The specific implementation process of the processor 501 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0125] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0126] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0127] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0128] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0129] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0130] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0131] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0132] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0135] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0136] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0137] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A model fusion method, characterized in that: include: Determining at least two participants who participate in the fusion of the financial scenario model, wherein data between the at least two participants are isolated from each other; For each participant, obtain a student model uploaded by the participant, the student model is obtained by performing knowledge distillation based on the teacher model of the participant, and the teacher model is obtained by performing model training on a to-be-trained model based on the data set of the participant; According to the student model of each participant, all student models are fused to obtain the global model.

2. The method according to claim 1, characterized in that According to the student model of each participant, all student models are fused to obtain a global model, including: For each participant, determine the amount of training data and the data quality score when the participant performs model training according to the data quality indicators, wherein the data quality indicators include a data distribution consistency check indicator, a data noise detection indicator, and a data missing rate calculation indicator; Obtaining a model performance score of the student model according to the student model and the model performance indicators, wherein the model performance indicators include model accuracy, loss value, and F1-score; Obtaining a model weight of the student model according to the data volume, the data quality score and the model performance score; According to each student model and the model weight of each student model, the global model parameters are obtained; A global model is obtained according to the global model parameters.

3. The method according to claim 2, characterized in that According to each student model and the model weight of each student model, the global model parameters are obtained to satisfy: in, are the global model parameters, For the The model weights of the participants, For the The model parameters of the student model of each participant, For the The amount of data from each participant, For the The data quality scores of the participants, For the The model performance scores of the participants, is the number of participants, , and is the weight coefficient and satisfies .

4. The method according to any one of claims 1 to 3, characterized in that: The student model is obtained in the following way: For each participant, knowledge distillation is performed based on the teacher model of the participant to obtain an initial student model; According to a preset disturbance range, differential privacy processing is performed on the model parameters in the initial student model to obtain a student model; Upload the student model.

5. The method according to claim 4, characterized in that For each participant, knowledge distillation is performed according to the teacher model of the participant to obtain an initial student model, including: For each participant, according to the teacher model of the participant, obtain the high-dimensional feature map output by the intermediate layer of the teacher model and the model result output by the output layer; The model to be trained is trained according to the high-dimensional feature map, the model result and a preset distillation loss function to obtain an initial student model.

6. The method according to claim 5, characterized in that The distillation loss function satisfies: in, is the distillation loss function, is the cross entropy loss function, is the classifier of the student model, is the transformation function, is the number of the intermediate layers, The first The output of the middle layer, For the student model The output of the middle layer.

7. The method according to claim 4, characterized in that The uploading of the student model comprises: According to a preset encryption algorithm, the student model is encrypted to obtain an encrypted student model; The encrypted student model is uploaded via a secure transmission protocol.

8. A model fusion device, characterized in that: include: A determination module, used to determine at least two participants participating in the fusion of the financial scenario model, wherein the data between the at least two participants are isolated from each other; An acquisition module is used to acquire, for each participant, a student model transmitted by the participant, wherein the student model is obtained by performing knowledge distillation on a teacher model of the participant, and the teacher model is obtained by performing model training on a to-be-trained model according to a data set of the participant; The fusion module is used to fuse all student models according to the student model of each participant to obtain a global model.

9. A model fusion device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images