Information fusion methods, devices, electronic devices, storage media and software products

By constructing a similarity fusion model and a hierarchical clustering algorithm, the problem of customer information not being able to be shared in the securities firm's business system was solved, and the identification of the same customer with high accuracy and cross-business data sharing were achieved.

CN123116129APending Publication Date: 2026-11-17申万宏源证券有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611246113.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-17
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

The independent operation of various business systems within securities firms leads to a lack of interoperability of customer information and a lack of unified identification, making it difficult to achieve high-accuracy identification of the same customer.

Method used

By collecting data, extracting features, and constructing a similarity fusion model, a weighted similarity model based on official identifiers, account information, and name features is used, combined with a hierarchical clustering algorithm, to automatically identify and merge records of the same customer.

Benefits of technology

It achieves efficient and accurate fusion of multiple features, breaks through the limitations of traditional customer identification, adapts to different business scenarios, improves the accuracy of identifying the same customer, breaks down data barriers between heterogeneous business systems, and supports cross-business data interoperability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123116129A_ABST
    Figure CN123116129A_ABST
Patent Text Reader

Abstract

This invention discloses an information fusion method, apparatus, electronic device, storage medium, and program product, belonging to the field of data fusion technology. The method includes: collecting data; extracting features from the data; determining similarity based on the features; constructing a similarity fusion model based on the similarity, a first weight coefficient, and a second weight coefficient; training the similarity fusion model; and aggregating groups based on the trained similarity fusion model. The training of the similarity fusion model includes: constructing a labeled sample set similar to the information distribution based on the distribution characteristics of the information; initializing the first weight coefficient, the second weight coefficient, and setting convergence conditions. This application breaks through the limitations of traditional customer entity recognition relying solely on name features, fully utilizing the strong recognition characteristics of official identifiers and account information to achieve efficient and accurate fusion of multiple features.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of data fusion technology, specifically relating to an information fusion method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] Currently, securities firms' various business systems operate independently, creating data silos. Different business systems use inconsistent standards for collecting customer information and storing it in different formats. The same real customer may have different records in multiple systems, and the lack of a unified association identifier makes data sharing difficult, statistical analysis challenging, and hinders the construction of customer profiles.

[0003] Some securities firms use a manual method to link customer information across systems. This involves staff regularly exporting customer data from various business systems and manually merging records for the same customer by comparing customer names and simple identifying information. The steps of this approach are: 1) Exporting basic customer information, account information, and other data from various business systems; 2) Manually filtering duplicate records and linking the same customer based on fuzzy matching using the full customer name; 3) Manually integrating customer information into a unified table. In addition, a few financial institutions use a single-dimensional matching method, such as linking customers solely through a unified credit code. However, this method only applies to customers with complete identification and cannot handle the need to link customers without official unique identifiers or with missing identifiers. Its applicability is limited, and it cannot achieve high accuracy in identifying the same customer.

[0004] Therefore, it is necessary to develop a new information fusion method, device, electronic device, storage medium, and program product to solve the existing problems. Summary of the Invention

[0005] The purpose of this invention is to provide an information fusion method, apparatus, electronic device, storage medium, and program product to solve the problem of being unable to identify the same customer with high accuracy.

[0006] Firstly, this application provides an information fusion method, including:

[0007] Collect data;

[0008] Extract features from the data;

[0009] Similarity is determined based on the aforementioned features;

[0010] A similarity fusion model is constructed based on the aforementioned similarity, the first weight coefficient, and the second weight coefficient.

[0011] Train the similarity fusion model;

[0012] Aggregate groups based on a trained similarity fusion model;

[0013] The training of the similarity fusion model includes:

[0014] Based on the distribution characteristics of the information, a labeled sample set similar to the information distribution is constructed;

[0015] Initialize the first weight coefficient, the second weight coefficient, and set the convergence condition;

[0016] The first and second weight coefficients are iteratively updated. After each iteration, the loss function value is determined and the parameter direction is adjusted until the convergence condition is met, and the first optimal weight parameter and the second optimal weight coefficient are obtained.

[0017] In one implementation of the first aspect, obtaining the first optimal weight parameter and the second optimal weight coefficient includes:

[0018] Based on the combined loss function consisting of cross-entropy loss function and pairing loss function, the deviation between the similarity predicted by the similarity fusion model and the true label of the sample is measured, and the first optimal weight coefficient and the second optimal weight coefficient are solved iteratively.

[0019] In one implementation of the first aspect, the aggregation of groups based on the similarity fusion model includes:

[0020] The entire set of information is paired up in pairs. The final similarity of each pair of information is determined by the trained similarity fusion model. All pairwise similarity results are stored in matrix form to form an information similarity matrix.

[0021] A test set was constructed by extracting 10% of the full information and testing the clustering performance of the trained similarity fusion model under different similarity thresholds. The F1 score was used as the evaluation index, and the threshold with the highest F1 score was selected as the basis for clustering.

[0022] Information that meets the conditions is merged into the same cluster based on the highest threshold of the F1 value, while information that does not meet the threshold requirement is classified into different clusters.

[0023] In one implementation of the first aspect, determining similarity based on the features includes: unique identifier similarity, account feature similarity, and name feature similarity;

[0024] When the unique identifiers are identical, account feature similarity and name feature similarity are not calculated.

[0025] When the similarity of unique identifiers is inconsistent or missing, the final similarity is determined by weighted fusion of the similarity of account features and the similarity of name features.

[0026] In one implementation of the first aspect, the account feature similarity includes:

[0027] Extract customer account information from different business systems and construct a set of account features for each customer record;

[0028] The account overlap of the two records is calculated based on the account feature set, the number of intersection elements of the two customer account sets, and the number of union elements of the two customer account sets, and is used as the account feature similarity.

[0029] The name feature similarity includes:

[0030] For product category customer names that cannot be segmented normally, N-gram encoding is used to generate feature vectors.

[0031] For institutional customer names that can be segmented normally, feature vectors are generated using TF-IDF-based word segmentation encoding.

[0032] Determine the cosine similarity of the feature vectors, take the average cosine similarity as the initial name similarity, and transform it to the [0,1] interval through a normalization formula to obtain the name feature similarity.

[0033] In one implementation of the first aspect, the labeled sample set includes: positive samples of different system records of the same real customer and negative samples of different customer records that have no relation to each other;

[0034] The ratio of positive samples to negative samples was 1:1.2.

[0035] Secondly, this application provides an information fusion apparatus, comprising:

[0036] The data acquisition module is used to collect data.

[0037] The extraction module is used to extract features of information from the data;

[0038] A similarity determination module is used to determine similarity based on the features;

[0039] A similarity fusion model construction module is used to construct a similarity fusion model based on the similarity, the first weight coefficient, and the second weight coefficient.

[0040] The training module is used to train the similarity fusion model;

[0041] The grouping module is used to aggregate groups based on a trained similarity fusion model;

[0042] The training module includes:

[0043] The labeled sample set construction module is used to construct a labeled sample set that is similar to the information distribution based on the distribution characteristics of the information.

[0044] The initialization module is used to initialize the first weight coefficient, the second weight coefficient, and set the convergence conditions.

[0045] The iterative update module is used to iteratively update the first weight coefficient and the second weight coefficient. After each iteration, the loss function value is determined and the parameter direction is adjusted until the convergence condition is met, and the first optimal weight parameter and the second optimal weight coefficient are obtained.

[0046] Thirdly, this application provides an electronic device, including: a processor and a memory;

[0047] The memory stores computer-executed instructions;

[0048] The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in the first aspect.

[0049] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method described in the first aspect.

[0050] Fifthly, this application provides a computer program product comprising a computer program that, when run on a computer, causes the computer to perform the method described in the first aspect.

[0051] According to the above-mentioned technical solution proposed in this application, the following technical effects can be achieved: This information fusion method, device, electronic device, storage medium, and program product break through the limitation of traditional customer entity identification relying solely on name features, fully utilize the strong identification characteristics of official identifiers and account information, and achieve efficient and accurate fusion of multiple features; it adaptively adapts to the distribution of customer data under different business scenarios, effectively improving the accuracy of similarity calculation and solving the problem of poor adaptability of traditional fixed-weight models; for the diversified business scenarios of securities companies, it adopts a dual approach of N-gram similarity encoding and rule-based word segmentation encoding to extract customer name features, solving the identification difficulties caused by name abbreviations, order adjustments, and character missingness; based on the strategy of supplementing official unique identifier information based on the differentiation of customer subject types, combined with a multi-dimensional weighted similarity model, it achieves high-accuracy identification of the same customer; through a fully automated solution for data collection, cleaning, feature extraction, model training, and cluster fusion, it breaks down data barriers between heterogeneous business systems, and achieves unified management of customer information and cross-business data interoperability. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below only involve some embodiments of this disclosure and are not intended to limit this disclosure. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0053] Figure 1 This is a schematic diagram of the structure of the method in the embodiments of this application;

[0054] Figure 2 This is a diagram illustrating the multi-source data acquisition architecture of the method in the embodiments of this application;

[0055] Figure 3 This is a frame diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as “comprising” or “including” mean that the element or object preceding the word encompasses the element or object listed following the word and its equivalents, without excluding other elements or objects. Terms such as “connected” or “linked” are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as “upper,” “lower,” “left,” and “right” are used only to indicate relative positional relationships, which may change accordingly when the absolute position of the described objects changes. It should be understood that the various steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0058] Firstly, according to one or more embodiments, the client information of securities firms' proprietary trading, investment advisory, client-facing, and financing businesses is currently stored in multiple heterogeneous business systems, lacking a unified management mechanism. This results in the inability to link records of the same real client in different systems, leading to data inconsistencies; at the same time, it is impossible to achieve cross-business fund size statistics, making it difficult to explore the cross-business needs of existing clients, and resulting in low efficiency in customer acquisition for auxiliary businesses.

[0059] With the diversification of securities firms' businesses, covering multiple areas such as proprietary trading, investment advisory, client-side trading, and financing, the clients include institutions and products, and the business roles encompass counterparties, bond issuers, clients, and custodians. The completeness and consistency of client information are crucial for business decision-making and client operations.

[0060] This application provides, as follows: Figure 1 , Figure 2 This information fusion method involves a process of data collection, information completion and cleaning, feature extraction, similarity calculation and aggregation, and information fusion. It collects customer data from multiple heterogeneous business systems within a securities firm, processes it to extract multi-dimensional features, and uses a weighted similarity model and clustering algorithm to identify different records of the same real customer. Ultimately, it achieves customer information fusion, forming a unified customer view. It automatically identifies and merges multiple entity records of the same real customer, enabling unified management of cross-business entity information, ensuring data consistency, supporting cross-business customer statistics and mining, and improving customer acquisition efficiency.

[0061] The specific implementation steps are as follows:

[0062] 1. Multi-source customer entity data collection:

[0063] ETL tools (such as Kettle) are used to build data acquisition channels, connecting to heterogeneous business systems such as proprietary business systems, investment advisory business systems, and customer transaction systems. The collected data dimensions include:

[0064] Basic information: Full name of the client, registered address (e.g., "Shenwan Hongyuan Co., Ltd." or "No. XX, Taipingqiao Street, Xicheng District, Beijing");

[0065] Official and unique identification information: Unified Social Credit Code and 21-digit interbank code for domestic enterprises; LEI code and business code for overseas enterprises; fund code for fund products; and filing number for wealth management products, etc.

[0066] Account information: Securities account number, funds account number (e.g., "Shanghai A Securities Account: A123456789" "Funds Account: 888800001111");

[0067] Transaction Information: All transaction details;

[0068] Data collection supports full data collection (executed once daily at midnight) to ensure data integrity.

[0069] 2. Customer information completion and cleaning:

[0070] Official Unique Identifier Completion: Using the client's full name as the search keyword, authoritative third-party databases (such as the National Enterprise Credit Information Publicity System and the China Foreign Exchange Trading Center's downstream interface) are used to complete the identifier differently based on the entity type. For example, for domestic enterprises, "Shanghai XX Fund Management Co., Ltd.", the unified social credit code is completed through name search; for overseas enterprises, "XX International Investment Co., Ltd.", the LEI code is completed; and for fund products, "XX Growth Hybrid Fund", the fund code is completed. Entities that still lack an identifier after completion are marked as "No Identifier".

[0071] According to one or more embodiments, the identifier completion method further includes: for trust product filing databases without public APIs, web crawling technology can be used to obtain identifier information.

[0072] Effective customer screening: Based on transaction information, screening rules are set to retain customer entities with transaction records or ongoing business dealings, while invalid records without any business traces are removed.

[0073] Duplicate data removal: Identify and delete completely duplicate records using the official unique identifier + account number as the composite primary key; for records without an official identifier, remove duplicate data using the customer's full name + registration address as the secondary primary key, and do not delete missing values.

[0074] 3. Extraction of core customer features:

[0075] Three core features were extracted from the cleaned customer records for subsequent similarity calculation:

[0076] Name feature extraction (dual encoding to word vectors):

[0077] The name feature extraction adopts a dual coding strategy: 1) For customer names containing uncommon words and that cannot be normally segmented (such as financial product names), N-gram coding is preferentially adopted. For example, for "Yuan Rong Le Xiang Bo Shi Fund", the segmentation tool cannot perform normal word segmentation. Therefore, N-gram coding is first used: the full customer name is split into 2-gram and 3-gram character fragments. For example, "Yuan Rong Le Xiang Bo Shi Fund" is split into 2-gram: "Yuan Rong", "Rong Le", "Le Xiang", "Xiang Bo", "Bo Shi", "Shi Ji", "Ji Jin"; 3-gram: "Yuan Rong Le", "Rong Le Xiang", "Le Xiang Bo", "Xiang Bo Shi", "Bo Shi Ji", "Shi Ji Jin", and the occurrence frequency of each fragment is counted to construct a word vector. 2) For customer names that can be normally segmented (such as ordinary enterprise names), rule-based word segmentation coding (TF-IDF algorithm) is adopted. For example, "CITIC Securities (Shanghai)" is split into "CITIC", "Securities", "Shanghai", and the weight of each key entity is calculated through the TF-IDF algorithm to generate a word vector.

[0078] Official unique identifier feature extraction: Perform standardization processing (removing redundant symbols and unifying the format) on the completed identifier information, which is used as a rigid matching feature; records without identifiers are marked as "none".

[0079] Account feature extraction: Extract securities accounts and capital accounts to construct an account feature set.

[0080] According to one or more embodiments, the name feature extraction can also use the Word2Vec model to replace the TF-IDF algorithm to generate segmented word vectors, learn the semantic features of names through training corpora, and can also achieve key information matching;

[0081] 4. Similarity calculation and aggregation grouping:

[0082] 4.1 Different from existing customer identification methods, most of which use text information as customer features and do not distinguish the identification capabilities of each feature. This application constructs three types of customer features with progressive identification strengths, calculates the similarity of each feature respectively, and provides a basis for subsequent similarity fusion, as follows specifically:

[0083] (1) Official unique identifier similarity X1: Perform standardization processing (removing redundant characters and unifying the coding format) on the official identifiers of customers such as the Unified Social Credit Code, the 21-digit bank code, and the LEI code. If the official identifiers of two customer records are completely consistent, the official unique identifier similarity X1=1; if the identifiers are inconsistent or missing, the official unique identifier X1=0. The official unique identifier feature is a rigid matching feature with the highest identification priority.

[0084] (2) Account Feature Similarity X2: Extract core account information such as securities account and fund account number of customers in different business systems, construct the account feature set of each customer record, and use the Jaccard coefficient to calculate the account overlap between two records as the account feature similarity. The calculation formula is as follows: in, Let A be the Jaccard coefficient, and let A and B be the sets of account features for the two customer records, respectively. This represents the number of elements in the intersection of two sets of customer accounts. This represents the number of elements in the union of two sets of customer accounts. Account features are strong correlation features for identifying financial customer entities. Compared with name features commonly used in traditional methods, they have the characteristics of uniqueness and stability, and have a significantly higher degree of distinguishability for the same customer entity. They also have a corrective effect on identification bias caused by non-standard names and confusing abbreviations.

[0085] (3) Name Feature Similarity X3: To address issues such as special characters, abbreviations, word order reversals, and typos in financial customer names, a dual encoding strategy is employed to generate name feature vectors. Specifically: For product-type customer names that cannot be properly segmented (e.g., "Longding No. 3"), N-gram encoding is used to generate feature vectors; for institutional customer names that can be properly segmented, TF-IDF-based segmentation encoding is used to generate feature vectors. The cosine similarity between the two feature vectors is calculated, and the arithmetic mean of the cosine similarity is taken as the initial name similarity. Then, it is normalized to the [0,1] interval using a normalization formula to obtain the final name feature similarity X3. The normalization formula is as follows:

[0086]

[0087] Here, x3 is the cosine similarity of the dual-encoded vectors, with a value range of [-1, 1]. After normalization, the value range of x3 is [0, 1]. A higher score indicates a higher similarity between the names of the two records.

[0088] 4.2 Improved Linear Weighted Similarity Fusion Model:

[0089] This application proposes a novel linear weighted similarity fusion model that breaks through the limitation of traditional customer entity recognition relying solely on name features. It fully utilizes the strong recognition characteristics of official identifiers and account information to achieve efficient and accurate fusion of multiple features. The similarity fusion model formula is as follows:

[0090]

[0091] in, These are the weighting coefficients for account feature similarity X2 and name feature similarity X3, respectively, satisfying the constraints. y represents the final similarity between two customer records, with a value range of [0,1]. A higher score indicates a greater likelihood that the two records belong to the same customer entity.

[0092] The innovation of the similarity fusion model is reflected in the following aspects, which are also the key improvements that distinguish this invention from existing technologies:

[0093] (1) Progressive identification logic optimization: adopting a rigid identifier priority design idea, clarifying the highest identification priority of the official unique identifier. When the similarity of the official unique identifier X1=1, the final similarity y=1 is output, and there is no need to calculate the similarity of account features and name features, reducing invalid calculations and improving the efficiency of customer entity identification. When the similarity of the official unique identifier X1=0, the final similarity is calculated by weighted fusion of the similarity of account features and name features.

[0094] (2) Adaptive weight learning mechanism: Unlike the fixed weights or manually set weights in traditional similarity fusion models, this application uses supervised learning to obtain the optimal weight parameters that are suitable for the data distribution of this business scenario through training with labeled samples. .in, This reflects the contribution of account features to customer entity identification. The contribution of name features is reflected in the model, and the specific values ​​of both are determined by the matching rules of the training samples, rather than by subjective human settings. This allows for adaptive adaptation to customer data distributions in different business scenarios, effectively improving the accuracy of similarity calculations and solving the problem of poor adaptability in traditional fixed-weight models. The weights of each dimension can be dynamically adjusted according to the business scenario.

[0095] 4.3 Model Training and Parameter Optimization Based on Combined Loss Function:

[0096] To obtain the optimal weight parameters This paper employs a supervised learning approach, using a combined loss function of cross-entropy loss and pairwise loss to measure the deviation between the model's predicted similarity and the true labels of the samples. The optimal parameters are iteratively solved using gradient descent. In financial customer identification scenarios, the focus is not only on the accuracy of the similarity of individual samples and whether the "N most similar customers" are ranked first, but also on the numerical discriminative power between the similarity scores of customer pairs belonging to the same entity and those belonging to different entities. To meet the needs of financial customer identification scenarios and improve the accuracy and ranking rationality of similarity predictions for customer identification, this application innovatively designs a combined loss function that integrates cross-entropy loss and pairwise loss. During the training of the weighted fusion model, it simultaneously optimizes the numerical accuracy of similarity scores and the relative ranking of customer entity pairs. This guides the model to assign higher similarity scores to customer pairs belonging to the same entity than to customer pairs belonging to different entities, improving the discriminative power of the model's similarity scores for customer identification and enhancing the rationality of clustering. The weights trained by the pairwise loss function widen the granularity of similarity scores, resulting in a larger numerical difference between similar and dissimilar customer pairs, preventing them from concentrating within a single numerical range. The sample true label refers to the true value. If two customers are the same entity, the true value is 1; otherwise, the true value is 0.

[0097] Loss function: ;

[0098] in: Cross-entropy loss ensures that the similarity values ​​closely match the true labels;

[0099] : Pair ranking loss, ensuring that similar customer pairs score higher than dissimilar customer pairs;

[0100] λ: Balance coefficient (0 < λ ≤ 1), used to control the strength of the ranking constraint. λ is the weighting coefficient that balances the cross-entropy loss and the pairwise loss, determined through a grid search on the validation set. In this embodiment, λ = 0.5 is experimentally preferred, allowing the model to simultaneously consider similarity prediction accuracy and customer matching ranking performance.

[0101] ;

[0102] Where N is the total number of samples, and dividing by N is the mean of the loss, y ture This indicates that the true label, or the true marker of the sample, means that the two customers belong to the same entity: y ture =1; The two customers do not belong to the same entity: y ture =0, Y is the predicted similarity output by the model, with a value of [0,1]. The higher the score, the more similar the model judges the two customers to be. ;

[0103] Where M is the total number of samples, and the average loss is obtained by dividing by M; Ypos is the positive pair similarity, which is the predicted similarity output by the model for two customers belonging to the same entity.

[0104] Yneg represents negative pairing similarity, indicating the predicted similarity score output by the model for two customers belonging to different entities.

[0105] The margin is the interval coefficient (usually between 0.1 and 0.3), which forces similar customers to have higher scores than non-similar customers. The margin is the difference between the two. In this example, margin=0.3 is selected.

[0106] The specific steps for model training are as follows:

[0107] (1) Constructing a training set: Statistically analyze the distribution characteristics of all customer information (including the proportion of missing official identifiers, account type distribution, etc.) and construct a labeled sample set similar to the distribution of the full data. Among them, positive samples are different system records of the same real customer confirmed by manual review, and negative samples are different customer records with no relation. The ratio of positive to negative samples is controlled at 1:1.2 to ensure the representativeness and balance of the training samples and avoid model overfitting.

[0108] (2) Set training parameters: Initialize weight parameters The learning rate is set to 0.05, the maximum number of iterations is set to 1000, and the convergence condition is that the difference between the loss functions of two adjacent iterations is ≤1e-3. Training stops when the convergence condition is met or the maximum number of iterations is reached.

[0109] (3) Solving for the optimal parameters: Iteratively updating the weight parameters using the gradient descent method. After each iteration, the loss function value is calculated, and the parameter direction is adjusted to reduce the loss value until the convergence condition is met, ultimately obtaining the optimal weight parameters adapted to this business scenario. The training of the similarity fusion model is completed.

[0110] 4.4 Aggregation Grouping Based on Hierarchical Clustering

[0111] Based on the trained similarity fusion model, a hierarchical clustering algorithm is used to aggregate and group all customer records. Records with similarity that meet the threshold requirement are clustered together and regarded as the same customer entity. The specific steps are as follows:

[0112] (1) Construct a similarity matrix: Pair all customer information records in pairs, calculate the final similarity y of each pair of records through the trained similarity fusion model, store all pair similarity results in matrix form to form a customer record similarity matrix, and provide data support for subsequent clustering and grouping.

[0113] (2) Clustering threshold optimization: A test set (with the same data distribution as the training set and without overlap) is constructed by extracting 10% of all customer information. The clustering performance of the model is tested under different similarity thresholds (0.7, 0.75, 0.8, 0.85). The F1 score (harmonic mean of precision and recall) is used as the evaluation index, and the threshold with the highest F1 score is selected as the basis for clustering. In this embodiment, when the threshold is set to 0.8, the model's F1 score reaches the highest (0.92), so the clustering threshold is determined to be 0.8. The F1 score is a general indicator for selecting a clustering threshold. Customers with similarity exceeding this threshold are grouped together. The F1 score is the harmonic mean of precision and recall. The F1 score is a comprehensive score of the clustering effect, used to judge whether the set threshold is effective in grouping similar customers together and separating different customers. The maximum score is 1. The closer to 1, the more accurate the clustering; the closer to 0, the more chaotic the clustering. It can be used to balance whether similar customers that should not be separated have been separated (recall) and dissimilar customers that should not be grouped together have been incorrectly merged (precision). Considering only a single indicator, the accuracy is poor. The F1 score compromises between the two and selects the most reliable clustering threshold.

[0114] (3) Hierarchical Clustering Grouping: A bottom-up hierarchical clustering algorithm is adopted. A similarity of ≥0.8 is used as the record merging condition. Customer records that meet the condition are merged into the same cluster, which is regarded as the same customer entity. For example, the comprehensive similarity between "Shanghai CITIC Securities Co., Ltd." and "CITIC Securities (Shanghai) Co., Ltd." is 0.85, which meets the threshold requirement and is classified into the same cluster; the comprehensive similarity between "China Construction Bank Corporation" and "Industrial and Commercial Bank of China Limited" is 0.32, which does not meet the threshold requirement and is classified into different clusters.

[0115] According to one or more embodiments, alternative clustering algorithms include: DBSCAN algorithm or graph clustering algorithm can be used instead of hierarchical clustering. DBSCAN does not require a preset number of clusters and is more adaptable to discretely distributed customer records; graph clustering can use customer records as nodes and similarity as edge weights to achieve aggregation and grouping through community detection.

[0116] 5. Customer information integration and output:

[0117] Customer records within the same cluster are merged according to the principle of "full data retrieval and deduplication": the official unique identifier and core basic information are merged with non-null values ​​and stored in a unified customer master data platform, supporting synchronization with business systems.

[0118] Secondly, according to one or more embodiments, an information fusion apparatus is provided, comprising:

[0119] The data acquisition module is used to collect data.

[0120] The extraction module is used to extract features of information from the data;

[0121] A similarity determination module is used to determine similarity based on the features;

[0122] A similarity fusion model construction module is used to construct a similarity fusion model based on the similarity, the first weight coefficient, and the second weight coefficient.

[0123] The training module is used to train the similarity fusion model;

[0124] The grouping module is used to aggregate groups based on a trained similarity fusion model;

[0125] The training module includes:

[0126] The labeled sample set construction module is used to construct a labeled sample set that is similar to the information distribution based on the distribution characteristics of the information.

[0127] The initialization module is used to initialize the first weight coefficient, the second weight coefficient, and set the convergence conditions.

[0128] The iterative update module is used to iteratively update the first weight coefficient and the second weight coefficient. After each iteration, the loss function value is determined and the parameter direction is adjusted until the convergence condition is met, and the first optimal weight parameter and the second optimal weight coefficient are obtained.

[0129] Thirdly, according to one or more embodiments, an electronic device is provided, including: a processor and a memory;

[0130] The memory stores computer-executed instructions;

[0131] The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in the first aspect.

[0132] Fourthly, according to one or more embodiments, a computer-readable storage medium is provided, wherein a computer program is stored therein, which, when run on a computer, causes the computer to perform the method as described in the first aspect.

[0133] Fifthly, according to one or more embodiments, a computer program product is provided, the computer program product including a computer program that, when run on a computer, causes the computer to perform the method as described in the first aspect.

[0134] like Figure 3 As shown, the information fusion method corresponding to the above embodiment, Figure 3This is a structural block diagram of an information fusion apparatus provided in an embodiment of this application. For ease of explanation, only the parts relevant to the embodiments of this disclosure are shown.

[0135] To implement the above embodiments, embodiments of this application also provide an electronic device. (See reference...) Figure 3 This document illustrates a structural schematic of an electronic device 900 used to implement this embodiment. This schematic is merely an example and should not be construed as limiting the functionality or scope of this embodiment. The electronic device 900 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), tablet computers, portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers.

[0136] Electronic device 900 may include processor 910 (such as central processing unit, graphics processor, etc.), which can perform various actions and processes according to programs stored in read-only memory 920 (ROM) or programs loaded from storage device 930 into random access memory 940 (RAM). Random access memory 940 also stores various programs and data required for the operation of electronic device 900.

[0137] The processor 910, read-only memory 920, and random access memory 940 are interconnected via bus 950.

[0138] The bus 950 is also connected to an input / output interface 960; the input / output interface 960 can connect to: input devices 970 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 980 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 930 including, for example, magnetic tape, hard disk, etc.; and communication devices 990 that allow the electronic device 900 to communicate wirelessly or wiredly with other devices to exchange data. It should be understood that it is not required to implement or possess all the devices shown in this application. More or fewer devices may be implemented or possessed alternatively.

[0139] According to this embodiment, the process described above with reference to the flowchart can be implemented as a computer software program. For example, this embodiment includes a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowchart, which can be downloaded and installed from a network via a communication device 990, or installed from a storage device 930, or installed from a read-only memory 920. When the computer program is executed by the processor 910, it performs the functions defined in the method of this embodiment.

[0140] It should be noted that the aforementioned computer-readable storage medium can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. Computer-readable storage media include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. A computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this embodiment, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. The propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0141] The aforementioned computer-readable storage medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0142] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0143] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.

[0144] In cases involving remote computers, the remote computer can be connected to the user's computer through any type of network—including a local area network (LAN) or a wide area network (WAN)—or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0145] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. Each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0146] The units involved in the embodiments of this disclosure can be implemented in software or in hardware. The names of the units are not, in some cases, limiting the scope of the unit itself.

[0147] The functions described above can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0148] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0149] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

[0150] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An information fusion method, characterized in that: include: Collect data; Extract features from the data; Similarity is determined based on the aforementioned features; A similarity fusion model is constructed based on the aforementioned similarity, the first weight coefficient, and the second weight coefficient. Train the similarity fusion model; Aggregate groups based on a trained similarity fusion model; The training of the similarity fusion model includes: Based on the distribution characteristics of the information, a labeled sample set similar to the information distribution is constructed; Initialize the first weight coefficient, the second weight coefficient, and set the convergence condition; The first and second weight coefficients are iteratively updated. After each iteration, the loss function value is determined and the parameter direction is adjusted until the convergence condition is met, and the first optimal weight parameter and the second optimal weight coefficient are obtained.

2. The information fusion method according to claim 1, characterized in that: The process of obtaining the first optimal weight parameter and the second optimal weight coefficient includes: Based on the combined loss function consisting of cross-entropy loss function and pairing loss function, the deviation between the similarity predicted by the similarity fusion model and the true label of the sample is measured, and the first optimal weight coefficient and the second optimal weight coefficient are solved iteratively.

3. The information fusion method according to claim 2, characterized in that: The aggregation grouping based on the similarity fusion model includes: The entire set of information is paired up in pairs. The final similarity of each pair of information is determined by the trained similarity fusion model. All pairwise similarity results are stored in matrix form to form an information similarity matrix. A test set was constructed by extracting 10% of the full information and testing the clustering performance of the trained similarity fusion model under different similarity thresholds. The F1 score was used as the evaluation index, and the threshold with the highest F1 score was selected as the basis for clustering. Information that meets the conditions is merged into the same cluster based on the highest threshold of the F1 value, while information that does not meet the threshold requirement is classified into different clusters.

4. The information fusion method according to claim 1, characterized in that: The similarity determination based on the features includes: unique identifier similarity, account feature similarity, and name feature similarity; When the unique identifiers are identical, account feature similarity and name feature similarity are not calculated. When the similarity of unique identifiers is inconsistent or missing, the final similarity is determined by weighted fusion of the similarity of account features and the similarity of name features.

5. The information fusion method according to claim 4, characterized in that: The account feature similarity includes: Extract customer account information from different business systems and construct a set of account features for each customer record; The account overlap of the two records is calculated based on the account feature set, the number of intersection elements of the two customer account sets, and the number of union elements of the two customer account sets, and is used as the account feature similarity. The name feature similarity includes: For product category customer names that cannot be segmented normally, N-gram encoding is used to generate feature vectors. For institutional customer names that can be segmented normally, feature vectors are generated using TF-IDF-based word segmentation encoding. Determine the cosine similarity of the feature vectors, take the average cosine similarity as the initial name similarity, and transform it to the [0,1] interval through a normalization formula to obtain the name feature similarity.

6. The information fusion method according to claim 1, characterized in that: The labeled sample set includes: positive samples from different systems recordings of the same real customer and negative samples from different customer records that have no relation to each other; The ratio of positive samples to negative samples was 1:1.

2.

7. An information fusion device, characterized in that, include: The data acquisition module is used to collect data. The extraction module is used to extract features of information from the data; A similarity determination module is used to determine similarity based on the features; A similarity fusion model construction module is used to construct a similarity fusion model based on the similarity, the first weight coefficient, and the second weight coefficient. The training module is used to train the similarity fusion model; The grouping module is used to aggregate groups based on a trained similarity fusion model; The training module includes: The labeled sample set construction module is used to construct a labeled sample set that is similar to the information distribution based on the distribution characteristics of the information. The initialization module is used to initialize the first weight coefficient, the second weight coefficient, and set the convergence conditions. The iterative update module is used to iteratively update the first weight coefficient and the second weight coefficient. After each iteration, the loss function value is determined and the parameter direction is adjusted until the convergence condition is met, and the first optimal weight parameter and the second optimal weight coefficient are obtained.

8. An electronic device, characterized in that, include: Processor and memory; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium for storing non-transitory computer-readable instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.