Customer portrait optimization method and system based on federal transfer learning

Abstract

The invention discloses a customer portrait optimization method and system based on federal transfer learning, and belongs to the technical field of information processing, and the method comprises the steps: S1, collecting data from a plurality of data sources based on a distributed crawler; s2, performing first processing on the collected data to obtain target data, and performing second processing on the target data to obtain classified data; and S3, respectively constructing user feature models in corresponding business scenes based on the plurality of data sources, inputting the classified data into the user feature models for analysis, respectively outputting customer feature parameters, and reconstructing a customer portrait based on the customer feature parameters. The problem that accurate risk assessment cannot be carried out on the customer due to the fact that the customer portrait feature is relatively single is solved.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and specifically to a customer portrait optimization method and system based on federated transfer learning. Background Art

[0002] With the rapid development of the financial technology industry, in order to ensure the stability and security of the financial system, it is particularly important to conduct accurate and comprehensive risk assessment of customers. The existing technology mainly constructs customer portraits based on bank data, and conducts risk assessment on customers based on the constructed customer portraits. For example, a bank customer portrait generation method, equipment, storage medium and device with patent number CN112487284A include: obtaining initial data of target bank customers, and extracting the initial data to obtain initial attribute data and initial behavior data; standardizing the initial attribute data to obtain target attribute data, and generating customer attribute labels according to the target attribute data; normalizing the initial behavior data to obtain target behavior data; generating customer behavior labels through a target neural network model according to the target behavior data; determining the customer portrait of the target bank customer according to the customer attribute label and the customer behavior label, and processing the bank data to improve the accuracy of the user portrait. However, the data in the above scheme only comes from the bank, and the scenario is relatively single. Therefore, there is a problem that the customer portrait features are relatively single, resulting in an inability to accurately assess the risk of the customer. Summary of the invention

[0003] In view of the problem that the existing technology has a relatively simple customer portrait feature, which makes it impossible to accurately assess the risk of customers, the present invention provides a customer portrait optimization method and system based on federated transfer learning, which obtains data from multiple data sources, inputs the data into the user feature model under the corresponding business scenario, analyzes and outputs the user feature parameters under the corresponding business scenario, and finally reconstructs the customer image based on the user feature parameters under the corresponding business scenario, solving the problem that the customer portrait feature is relatively simple, which makes it impossible to accurately assess the risk of customers. The accuracy and reliability of risk assessment are guaranteed, while effectively protecting the privacy information of users, thereby ensuring the stability and security of the financial system.

[0004] In order to solve the above technical problems, the present invention provides a customer profile optimization method based on federated transfer learning, comprising: S1: Collect data from several data sources based on distributed crawlers; S2: Process the collected data for the first time to obtain target data, and process the target data for the second time to obtain classified data; S3: Build user feature models for corresponding business scenarios based on several data sources, input the classified data into the user feature model for analysis and output customer feature parameters respectively, and reconstruct the customer portrait based on the customer feature parameters.

[0005] After adopting the above technical solution, the present invention has the following advantages: Some data sources are banks, social media and e-commerce platforms. Distributed crawlers are used to obtain data from multiple data sources, and the data is input into the corresponding model for analysis to reconstruct customer portraits. This ensures the diversity and richness of the data, allowing financial institutions to have a deeper understanding of customers from multiple dimensions, thereby enriching the characteristics of customer portraits and ensuring the comprehensiveness and accuracy of customer portrait characteristics. The business scenarios are banking, social media, and e-commerce platforms. By processing the collected data multiple times, valuable data is extracted and input into the user feature model in the corresponding business scenario, which reduces the complexity and redundancy of data processing and enables data analysis to focus on specific problems and needs in different business scenarios, helping to improve the efficiency and accuracy of the model's data analysis. By building their own models, the customer image is reconstructed based on the parameters output by the models. During the interaction, only the model parameters are exchanged instead of the original data, which avoids data leakage, helps protect data privacy and security, and breaks through the boundaries of data islands between different business scenarios. It solves the problem of being unable to conduct accurate risk assessment on customers due to the relatively simple characteristics of customer portraits.

[0006] Preferably, the S1 comprises: S11: starting to collect data from a number of data sources based on preset network bandwidth resources; S12: During the data collection process, the historical data of each crawler node is obtained, and based on the historical data of each crawler node and the current task queue, the future bandwidth requirements of each crawler node are predicted, and network bandwidth resources are allocated to each crawler node based on the prediction results. Data is collected from several data sources based on the network bandwidth resources allocated to each crawler node. If the data collection task is not completed, continue to execute S112. If the data collection task is completed, execute S2.

[0007] When multiple crawler nodes run at the same time and try to collect a large amount of data from multiple data sources, some crawler nodes may be wasted due to excess bandwidth resources, while other nodes may not be able to complete data collection tasks due to insufficient bandwidth. Therefore, the bandwidth resources of each crawler node are adjusted dynamically through prediction. The current crawler node task queue length and the total historical data crawled by the current crawler node have a decisive influence on the prediction of the current crawler node network bandwidth resources. Therefore, predictions are made based on the historical data of each crawler node and the current task queue. This allows each crawler node to obtain bandwidth resources that match its current task requirements, avoiding the waste of bandwidth resources. At the same time, it breaks through the shortcomings of distributed crawlers in the process of obtaining multiple data sources, thereby ensuring the diversity and richness of data, allowing financial institutions to understand customers more deeply from multiple dimensions, thereby enriching customer portrait features and ensuring the comprehensiveness of customer portrait features.

[0008] Preferably, S2 includes: S21: Acquire attribute data from the collected data, construct a weight matrix based on the category of the attribute data and the intuitive fuzzy correspondence table, and calculate the weight of each attribute data under the corresponding category according to the weight matrix; S22: Mark the attribute data whose weights of the attribute data under the corresponding category meet the preset conditions as target data.

[0009] Since different business scenarios use different standards when conducting risk assessments on customers, and the intuitive fuzzy correspondence table reflects the fuzziness and uncertainty of the attributes of things, a weight matrix is ​​constructed based on the category of attribute data and the intuitive fuzzy correspondence table, which fully considers the interaction and mutual influence between different data, thereby ensuring the rationality of the attribute data weight. Since the attribute data that meets the preset conditions has a greater impact on the accuracy of the customer feature portrait, it is marked as target data. This helps to identify the key factors that have a greater impact on the customer feature portrait and risk assessment results, ensures the accuracy of the customer portrait features, and further improves the accuracy and effectiveness of risk assessment.

[0010] Preferably, in S21, the weight matrix is ​​constructed based on the category of the attribute data and the intuitive fuzzy correspondence table, and the weight of each attribute data under the corresponding category is calculated according to the weight matrix, which includes: The weight matrix is ​​Q = (q ij ) n*n , n is the number of attribute data in each category, q ij ∈(a, b), a represents the degree to which the evaluator is biased towards i when evaluating i and j, and b represents the degree to which the evaluator is biased towards j when evaluating i and j. By mathematically expressing the evaluator's preference, the subjectivity and arbitrariness in the decision-making process can be reduced, making the decision results more objective and fair, thereby improving the accuracy of customer profile features.

[0011] Preferably, in S2, the second processing of the target data to obtain the classified data includes: Binding the data of the same individual in the first internal data in the target data according to the user unique identifier to obtain the first classification data of the same individual; The first external data in the target data is compared with the preset table. If the comparison result meets the first requirement, the first external data that meets the first requirement is marked as valid external data, and the associated data associated with the valid external data is obtained, and the associated data that meets the second requirement is bound to obtain the second classification data of the same individual.

[0012] The first internal data is data collected from the banking system, and the user information in the banking system includes the user's unique identification information, so the first classification data of the same individual can be obtained based on the user's unique identifier. The first external data is data collected from social media and e-commerce platforms, and social media and e-commerce platforms are isolated and have inconsistent unique identifiers for users, so multiple requirements need to be set to obtain the second classification data of the same individual. By obtaining all the internal data of the same individual in the first internal data and all the external data of the same individual in the first external data, the accuracy and comprehensiveness of the customer portrait features are improved.

[0013] Preferably, S3 includes: S31: Build an external user feature model and an internal user feature model, and train the external user feature model and the internal user feature model based on historical data and federated transfer learning; S32: Input the first classification data of the same individual into the trained internal user feature model for analysis and output of internal feature parameters, and input the second classification data of the same individual into the trained external user feature model for analysis and output of external feature parameters. By inputting different data of the same individual into different models for analysis, the parameters finally outputted can more accurately reflect the user characteristics in each business scenario, which helps to improve the data analysis efficiency of the model, so that each model focuses on the customer feature analysis in each business scenario, enhances the accuracy and reliability of the model, and thus improves the accuracy of the customer portrait feature.

[0014] Preferably, in S31, the training of the external user feature model and the internal user feature model based on historical data and federated transfer learning includes: Collect historical data based on a preset parsing rule, process the historical data to obtain second internal data and second external data of the same individual, input the second internal data into the constructed internal user feature model to train the constructed internal user feature model once, output a first result, input the second external data into the constructed external user feature model to train the constructed external user feature model once, and output a second result; The first result and the second result are respectively input into the constructed external user feature model and the constructed internal user feature model to perform secondary training on the constructed external user feature model and the constructed internal user feature model. The first training outputs feature parameters, and the second training uses the feature parameters to construct the customer feature image. Through multiple trainings, the model continuously learns and optimizes, thereby improving the generalization ability and accuracy of the model.

[0015] The beneficial effects of this program: Several data sources are banks, social media and e-commerce platforms. Distributed crawlers are used to obtain data from multiple data sources, and the data is input into the corresponding models for analysis to reconstruct customer portraits. This allows each model to focus on customer feature analysis in each business scenario, ensuring the diversity and richness of data while enhancing the accuracy and reliability of the model, allowing financial institutions to understand customers more deeply and accurately from multiple dimensions, thereby enriching customer portrait features and ensuring the comprehensiveness and accuracy of customer portrait features. The business scenarios are banking, social media, and e-commerce platforms. By processing the collected data multiple times, valuable data is extracted and input into the user feature model in the corresponding business scenario, which reduces the complexity and redundancy of data processing and enables data analysis to focus on specific problems and needs in different business scenarios, helping to improve the efficiency and accuracy of the model's data analysis. By building their own models, the customer image is reconstructed based on the parameters output by the models. During the interaction, only the model parameters are exchanged instead of the original data, which avoids data leakage, helps protect data privacy and security, and breaks through the boundaries of data islands between different business scenarios. It solves the problem of being unable to conduct accurate risk assessment on customers due to the relatively simple characteristics of customer portraits.

[0016] The present invention also provides a customer portrait optimization system based on federated transfer learning, including a data acquisition module and a customer feature learning module. The data acquisition module is used to collect data from several data sources, and the customer feature learning module is used to reconstruct customer portraits.

[0017] Preferably, the customer feature learning module includes a first data processing submodule and a second data processing submodule, wherein the first data processing submodule is used to construct a weight matrix based on the category of attribute data and the intuitive fuzzy correspondence table, and calculate the weight of each attribute data under the corresponding category according to the weight matrix; The second data processing submodule is used to obtain the first classification data and the second classification data of the same individual.

[0018] Preferably, the customer feature learning module includes a model building submodule and a model training submodule. The model building submodule is used to build user feature models for corresponding business scenarios based on several data sources. The model training submodule includes a first training unit and a second training unit. The first training unit is used to perform a first training on the constructed internal user feature model and the external user feature model. The second training unit is used to perform a second training on the constructed internal user feature model and the external user feature model.

[0019] Beneficial effects of this program: The data collection module collects data from multiple data sources, overcoming the shortcomings of distributed crawlers in obtaining multiple data sources, thereby ensuring the diversity and richness of data, and further ensuring the comprehensiveness of customer portrait features; Since different business scenarios use different standards for risk assessment of customers, and the intuitive fuzzy correspondence table reflects the fuzziness and uncertainty of the attributes of things, the first data processing submodule fully considers the interaction and mutual influence between different data, and the customer feature learning module is used to build and train models in different business scenarios, so that each model focuses on the customer feature analysis in each business scenario, while improving the accuracy of customer portrait features; Through multiple trainings of the model training submodule, the model is continuously learned and optimized, which improves the generalization ability and accuracy of the model. It solves the problem of being unable to conduct accurate risk assessment on customers due to the relatively simple characteristics of customer portraits.

[0020] After adopting the above technical solution, the present invention has the following advantages: Several data sources are banks, social media and e-commerce platforms. Distributed crawlers are used to obtain data from multiple data sources, and the data is input into the corresponding models for analysis to reconstruct customer portraits. This allows each model to focus on customer feature analysis in each business scenario, ensuring the diversity and richness of data while enhancing the accuracy and reliability of the model, allowing financial institutions to understand customers more deeply and accurately from multiple dimensions, thereby enriching customer portrait features and ensuring the comprehensiveness and accuracy of customer portrait features. The business scenarios are banking, social media, and e-commerce platforms. By processing the collected data multiple times, valuable data is extracted and input into the user feature model in the corresponding business scenario, which reduces the complexity and redundancy of data processing and enables data analysis to focus on specific problems and needs in different business scenarios, helping to improve the efficiency and accuracy of the model's data analysis. By building their own models, the customer image is reconstructed based on the parameters output by the models. During the interaction, only the model parameters are exchanged instead of the original data, which avoids data leakage, helps protect data privacy and security, and breaks through the boundaries of data islands between different business scenarios. It solves the problem of being unable to conduct accurate risk assessment on customers due to the relatively simple characteristics of customer portraits. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Other features, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are only for the purpose of illustrating preferred embodiments and are not to be considered as limiting the present invention. Also, the same reference symbols are used throughout the drawings to represent the same parts.

[0022] Figure 1 A method flow chart of a customer portrait optimization method based on federated transfer learning of the present invention. DETAILED DESCRIPTION

[0023] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation method described herein is only an optimal embodiment of the present invention, which is only used to explain the present invention and does not limit the scope of protection of the present invention. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0024] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the operations (or steps) as sequential processes, many of the operations (or steps) therein can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the drawings; the process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0025] Embodiment 1: like Figure 1As shown in the figure, a customer profile optimization method based on federated transfer learning includes: S1: Collect data from several data sources based on distributed crawlers.

[0026] The S1 includes: S11: starting to collect data from a number of data sources based on preset network bandwidth resources; S12: During the data collection process, the historical data of each crawler node is obtained, and based on the historical data of each crawler node and the current task queue, the future bandwidth requirements of each crawler node are predicted, and network bandwidth resources are allocated to each crawler node based on the prediction results. Data is collected from several data sources based on the network bandwidth resources allocated to each crawler node. If the data collection task is not completed, continue to execute S112. If the data collection task is completed, execute S2.

[0027] If data is collected from e-commerce platforms, banks and social platforms respectively, corresponding control nodes are allocated to the e-commerce platforms, banks and social platforms respectively, which are node A, node B and node C respectively. The corresponding control nodes contain multiple crawler nodes. The total amount of data crawled by each crawler node in a period of time is obtained, and the total amount of data is divided by the time difference of the period of time to obtain the average bandwidth of each crawler node in the period of time. The task queue length at the starting time of the period of time is obtained, and the average bandwidth is divided by the task queue length to obtain the current task queue length. The product of the current task queue length and the average bandwidth divided by the task queue length is calculated to obtain the predicted bandwidth demand. When allocating bandwidth resources, if the bandwidth resources are insufficient, the priority of each task in the current task queue of each crawler node is obtained, and the bandwidth demand of tasks with higher priority is met first. Through reasonable allocation, each crawler node can obtain bandwidth resources that match its current task requirements, avoiding the waste of bandwidth resources. At the same time, it breaks through the shortcomings of distributed crawlers in obtaining multiple data sources, thereby ensuring the diversity and richness of data, allowing financial institutions to understand customers more deeply from multiple dimensions, thereby enriching customer portrait features and ensuring the comprehensiveness of customer portrait features.

[0028] S2: Perform a first processing on the collected data to obtain target data, and perform a second processing on the target data to obtain classified data.

[0029] The S2 includes: S21: Acquire attribute data from the collected data, construct a weight matrix based on the category of the attribute data and the intuitive fuzzy correspondence table, and calculate the weight of each attribute data under the corresponding category according to the weight matrix; S22: Mark the attribute data whose weights of the attribute data under the corresponding category meet the preset conditions as target data.

[0030] In S21, the weight matrix is ​​constructed based on the category of attribute data and the intuitive fuzzy correspondence table, and the weight of each attribute data under the corresponding category is calculated according to the weight matrix, including: The weight matrix is ​​Q = (q ij ) n*n , n is the number of attribute data in each category, q ij ∈(a, b), where a represents the degree to which the evaluator is biased towards i when evaluating i and j, and b represents the degree to which the evaluator is biased towards j when evaluating i and j.

[0031] In S2, the second processing of the target data to obtain the classified data includes: Binding the data of the same individual in the first internal data in the target data according to the user unique identifier to obtain the first classification data of the same individual; The first external data in the target data is compared with the preset table. If the comparison result meets the first requirement, the first external data that meets the first requirement is marked as valid external data, and the associated data associated with the valid external data is obtained, and the associated data that meets the second requirement is bound to obtain the second classification data of the same individual.

[0032] In this embodiment, several data sources are banks, social media, and e-commerce platforms. The attribute data are divided into the attribute data of banks and the attribute data of social media and e-commerce platforms. The intuitive fuzzy correspondence table reflects the fuzziness and uncertainty of the attributes of things. If the income certificate, loan record, overdue situation, bank transaction record, asset-liability situation, occupation, and marital status in the bank's attribute data meet the preset conditions, when calculating the weights of the income certificate, loan record, overdue situation, bank transaction record, asset-liability situation, occupation, and marital status in the bank's attribute data, n is 7, and the weight matrix is If the purchase records, payment behaviors, credit ratings, and social interactions in the attribute data of social media and e-commerce platforms meet the preset conditions, when calculating the weights of the purchase records, payment behaviors, credit ratings, and social interactions in the attribute data of social media and e-commerce platforms, n is 4, and the weight matrix is The weight of each attribute data is calculated by the formula Calculated. The attributes in the preset table include name, ID card, phone number, and email address. If any two of the above preset table attributes exist in the first external data, the first requirement is met. The associated data associated with the valid external data is the specific value corresponding to the valid external data. If the name of an individual in social media is xx and the phone number is 12324154324, and the name of an individual in the e-commerce platform is also xx and the phone number is 12324154324, then the second requirement is met. It can be seen that the two are the same individual, so they are bound. Since the first external data is the data of social media and e-commerce platforms, and social media and e-commerce platforms are isolated and the unique identifiers of users are inconsistent, it is necessary to set multiple conditions to associate the data of the same individual in social media and e-commerce platforms. At this time, the second classification data of the same individual can be obtained. Since the first internal data is the data of the bank, the unique identifiers of users in the bank are basically the same, and the unique identifier can be the ID card, so the first classification data of the same individual can be obtained through the ID card. It fully considers the interactions and mutual influences between different data, ensures the diversity and richness of data by acquiring data from multiple scenarios, and reduces the complexity and redundancy of data processing, enabling financial institutions to understand customers more deeply from multiple dimensions, thereby enriching customer portrait features and ensuring the comprehensiveness and accuracy of customer portrait features.

[0033] S3: Build user feature models for corresponding business scenarios based on several data sources, input classified data into the user feature model for analysis and output customer feature parameters respectively, and reconstruct customer portraits based on the customer feature parameters.

[0034] The S3 includes: S31: Build an external user feature model and an internal user feature model, and train the external user feature model and the internal user feature model based on historical data and federated transfer learning; S32: inputting the first classification data of the same individual into the trained internal user feature model for analysis and outputting internal feature parameters, and inputting the second classification data of the same individual into the trained external user feature model for analysis and outputting external feature parameters.

[0035] In S31, the training of the external user feature model and the internal user feature model based on historical data and federated transfer learning includes: Collect historical data based on a preset parsing rule, process the historical data to obtain second internal data and second external data of the same individual, input the second internal data into the constructed internal user feature model to train the constructed internal user feature model once, output a first result, input the second external data into the constructed external user feature model to train the constructed external user feature model once, and output a second result; The first result and the second result are respectively input into the constructed external user feature model and the constructed internal user feature model to perform secondary training on the constructed external user feature model and the constructed internal user feature model.

[0036] Federated transfer learning allows each participant to train models locally without centralizing the data to a central server, so that the data of each participant is kept locally, avoiding the risk of data leakage and protecting the privacy of users. In this embodiment, the business scenarios are business scenarios of banks, social media, and e-commerce platforms, the external user feature model is a model for social media and e-commerce platforms, the internal user feature model is a model for banks, and the historical data is historical data in banks, social media, and e-commerce platforms. Since the data in the financial industry often contains a large number of tables, charts, and texts containing industry terms, traditional information extraction methods may be difficult to accurately capture this information. Collecting historical data according to preset parsing rules can identify and accurately extract key information from complex pages. After key information such as proof of income, loan records, overdue status, bank transaction records, assets and liabilities, occupations, marital status, etc. are input into the internal user feature model, user feature parameters such as credit score, repayment ability index, and career stability are output, and then the internal user feature model is used to extract key information from complex pages. The user feature model is trained once, and key information such as purchase records, payment behavior, credit evaluation, social interaction and other data are input into the external user feature model, and then user feature parameters such as credit rating, consumption preference, payment habit, and quality of life index are output. Then the external user feature model is trained once, and user feature parameters such as credit score, repayment ability index, and career stability are input into the external user feature model for secondary training of the external user feature model. User feature parameters such as credit rating, consumption preference, payment habit, and quality of life index are input into the internal user feature model for secondary training of the internal user feature model to further improve the user feature image. When reconstructing the user image, the trained model is used to conduct multi-dimensional analysis of the user, and then multiple user feature parameters are obtained, and the customer image is reconstructed based on the user characteristics. By extracting valuable data and inputting it into the user feature model in the corresponding business scenario, the complexity and redundancy of data processing are reduced, so that data analysis can focus on specific problems and needs in different business scenarios, which helps to improve the efficiency and accuracy of the model's data analysis; by building their own models, the customer image is reconstructed based on the parameters output by the model. During the interaction process, only model parameters are exchanged instead of original data, which avoids data leakage, helps to protect data privacy and security, and at the same time breaks down the boundaries of data islands between different business scenarios.

[0037] Embodiment 2: A customer portrait optimization system based on federated transfer learning includes a data collection module and a customer feature learning module. The data collection module is used to collect data from several data sources, and the customer feature learning module is used to reconstruct customer portraits.

[0038] The customer feature learning module includes a first data processing submodule and a second data processing submodule. The first data processing submodule is used to construct a weight matrix based on the category of attribute data and the intuitive fuzzy correspondence table, and calculate the weight of each attribute data under the corresponding category according to the weight matrix; The second data processing submodule is used to obtain the first classification data and the second classification data of the same individual.

[0039] The customer feature learning module includes a model building submodule and a model training submodule. The model building submodule is used to build user feature models for corresponding business scenarios based on several data sources. The model training submodule includes a first training unit and a second training unit. The first training unit is used to perform a first training on the constructed internal user feature model and the external user feature model. The second training unit is used to perform a second training on the constructed internal user feature model and the external user feature model.

[0040] In this embodiment, the data collection module is used to collect data from several data sources, which are banks, social media, and e-commerce platforms. The first data processing submodule is used to construct a weight matrix based on the category of attribute data and the intuitive fuzzy correspondence table, calculate the weight of each attribute data under the corresponding category according to the weight matrix, and divide the attribute data into the attribute data of the bank and the attribute data of social media and e-commerce platforms. The intuitive fuzzy correspondence table reflects the fuzziness and uncertainty of the attributes of things. If the income certificate, loan record, overdue situation, bank transaction record, asset-liability situation, occupation, and marital status in the bank's attribute data meet the preset conditions, when calculating the weights of the income certificate, loan record, overdue situation, bank transaction record, asset-liability situation, occupation, and marital status in the bank's attribute data, n is 7, and the weight matrix is If the purchase records, payment behaviors, credit ratings, and social interactions in the attribute data of social media and e-commerce platforms meet the preset conditions, when calculating the weights of the purchase records, payment behaviors, credit ratings, and social interactions in the attribute data of social media and e-commerce platforms, n is 4, and the weight matrix is The weight of each attribute data is calculated by the formula Calculated. The second data processing submodule is used to obtain the first classification data and the second classification data of the same individual. If any two of the attributes in the preset table exist in the first external data, the first requirement is met. The attributes in the preset table include name, ID card, telephone number, and email address. The associated data associated with the valid external data is the specific value corresponding to the valid external data. If the name of an individual in the social media is xx and the telephone number is 12324, and the name of an individual in the e-commerce platform is also xx and the telephone number is 12324, then the second requirement is met, and it can be known that the two are the same individual, so they are bound. Since the first external data includes data from social media and e-commerce platforms, and social media and e-commerce platforms are isolated and the unique identifiers of users are inconsistent, it is necessary to set multiple conditions to associate the data of the same individual in social media and e-commerce platforms. At this time, the second classification data of the same individual can be obtained. Since the first internal data is the data of the bank, the unique identifiers of users in the bank are basically the same, and the unique identifier can be an ID card, so the first classification data of the same individual can be obtained through the ID card. It fully considers the interactions and mutual influences between different data, ensures the diversity and richness of data by acquiring data from multiple scenarios, and reduces the complexity and redundancy of data processing, enabling financial institutions to understand customers more deeply from multiple dimensions, thereby enriching customer portrait features and ensuring the comprehensiveness and accuracy of customer portrait features.

[0041] In this embodiment, the business scenario is the business scenario of banks, social media, and e-commerce platforms. The external user feature model is a model for social media and e-commerce platforms, the internal user feature model is a model for banks, and the historical data is historical data in banks, social media, and e-commerce platforms. Since the data in the financial industry often contains a large number of tables, charts, and texts containing industry terms, traditional information extraction methods may be difficult to accurately capture this information. Collecting historical data according to preset parsing rules can identify and accurately extract key information from complex pages. After key information such as proof of income, loan records, overdue status, bank transaction records, assets and liabilities, occupations, marital status, etc. are input into the internal user feature model, user feature parameters such as credit score, repayment ability index, and career stability are output. Then, the internal user feature model is trained once, and key information such as purchase records, payment behavior, credit evaluation, social interaction and other data are input into the external user feature model, and then user feature parameters such as credit rating, consumption preference, payment habit, and quality of life index are output. Then, the external user feature model is trained once, and user feature parameters such as credit score, repayment ability index, and career stability are input into the external user feature model for secondary training of the model. User feature parameters such as credit rating, consumption preference, payment habit, and quality of life index are input into the internal user feature model for secondary training of the model to further improve the user feature image. When reconstructing the user image, the trained model is used to conduct a multi-dimensional analysis of the user, and then multiple user feature parameters are obtained, and the customer image is reconstructed based on the user features. By extracting valuable data and inputting it into the user feature model in the corresponding business scenario, the complexity and redundancy of data processing are reduced, so that data analysis can focus on specific problems and needs in different business scenarios, which helps to improve the efficiency and accuracy of the model's data analysis; by building their own models, the customer image is reconstructed based on the parameters output by the model. During the interaction process, only model parameters are exchanged instead of original data, which avoids data leakage, helps to protect data privacy and security, and at the same time breaks down the boundaries of data islands between different business scenarios.

[0042] The specific implementation described above is a preferred implementation of a customer portrait optimization method and system based on federated transfer learning of the present invention, and is not intended to limit the specific implementation scope of the present invention. The scope of the present invention includes but is not limited to this specific implementation. All equivalent changes made in accordance with the shape and structure of the present invention are within the protection scope of the present invention.

Claims

1. A customer profile optimization method based on federated transfer learning, characterized in that: include: S1: Collect data from several data sources based on distributed crawlers; S2: Perform a first processing on the collected data to obtain target data, and perform a second processing on the target data to obtain classified data; S3: Build user feature models for corresponding business scenarios based on several data sources, input classified data into the user feature model for analysis and output customer feature parameters respectively, and reconstruct customer portraits based on the customer feature parameters.

2. According to the customer profile optimization method based on federated transfer learning in claim 1, it is characterized in that: The S1 includes: S11: starting to collect data from a number of data sources based on preset network bandwidth resources; S12: During the data collection process, the historical data of each crawler node is obtained, and based on the historical data of each crawler node and the current task queue, the future bandwidth requirements of each crawler node are predicted, and network bandwidth resources are allocated to each crawler node based on the prediction results. Data is collected from several data sources based on the network bandwidth resources allocated to each crawler node. If the data collection task is not completed, continue to execute S112. If the data collection task is completed, execute S2.

3. According to the customer profile optimization method based on federated transfer learning in claim 1, it is characterized in that: The S2 includes: S21: Acquire attribute data from the collected data, construct a weight matrix based on the category of the attribute data and the intuitive fuzzy correspondence table, and calculate the weight of each attribute data under the corresponding category according to the weight matrix; S22: Mark the attribute data whose weights of the attribute data under the corresponding category meet the preset conditions as target data.

4. According to claim 3, a customer profile optimization method based on federated transfer learning is characterized in that: In S21, the weight matrix is ​​constructed based on the category of attribute data and the intuitive fuzzy correspondence table, and the weight of each attribute data under the corresponding category is calculated according to the weight matrix, including: The weight matrix is ​​Q = (q ij ) n*n , n is the number of attribute data in each category, q ij ∈(a, b), where a represents the degree to which the evaluator is biased towards i when evaluating i and j, and b represents the degree to which the evaluator is biased towards j when evaluating i and i.

5. According to the customer profile optimization method based on federated transfer learning in claim 1, it is characterized in that: In S2, the second processing of the target data to obtain the classified data includes: Binding the data of the same individual in the first internal data in the target data according to the user unique identifier to obtain the first classification data of the same individual; The first external data in the target data is compared with the preset table. If the comparison result meets the first requirement, the first external data that meets the first requirement is marked as valid external data, and the associated data associated with the valid external data is obtained, and the associated data that meets the second requirement is bound to obtain the second classification data of the same individual.

6. The customer profile optimization method based on federated transfer learning according to claim 1, characterized in that: The S3 includes: S31: Build an external user feature model and an internal user feature model, and train the external user feature model and the internal user feature model based on historical data and federated transfer learning; S32: inputting the first classification data of the same individual into the trained internal user feature model for analysis and outputting internal feature parameters, and inputting the second classification data of the same individual into the trained external user feature model for analysis and outputting external feature parameters.

7. The customer profile optimization method based on federated transfer learning according to claim 6 is characterized in that: In S31, the training of the external user feature model and the internal user feature model based on historical data and federated transfer learning includes: collecting historical data based on a preset parsing rule, processing the historical data to obtain second internal data and second external data of the same individual, inputting the second internal data into the constructed internal user feature model to train the constructed internal user feature model once, and outputting a first result, inputting the second external data into the constructed external user feature model to train the constructed external user feature model once, and outputting a second result; The first result and the second result are respectively input into the constructed external user feature model and the constructed internal user feature model to perform secondary training on the constructed external user feature model and the constructed internal user feature model.

8. A customer profile optimization system based on federated transfer learning, applicable to a customer profile optimization method based on federated transfer learning as claimed in any one of claims 1 to 7, characterized in that: It includes a data collection module and a customer feature learning module. The data collection module is used to collect data from several data sources, and the customer feature learning module is used to reconstruct customer portraits.

9. The customer profile optimization system based on federated transfer learning according to claim 8, characterized in that: The customer feature learning module includes a first data processing submodule and a second data processing submodule. The first data processing submodule is used to construct a weight matrix based on the category of attribute data and the intuitive fuzzy correspondence table, and calculate the weight of each attribute data under the corresponding category according to the weight matrix; The second data processing submodule is used to obtain the first classification data and the second classification data of the same individual.

10. The customer profile optimization system based on federated transfer learning according to claim 8, characterized in that: The customer feature learning module includes a model building submodule and a model training submodule. The model building submodule is used to build user feature models for corresponding business scenarios based on several data sources. The model training submodule includes a first training unit and a second training unit. The first training unit is used to perform a first training on the constructed internal user feature model and the external user feature model. The second training unit is used to perform a second training on the constructed internal user feature model and the external user feature model.

Images