Customer data processing method and device, computer equipment and storage medium

Abstract

The invention relates to a client data processing method and device, computer equipment and a storage medium, and the method comprises the following steps: analyzing a client information data set, and obtaining a client record; performing multi-dimensional vectorization processing on the information items in the customer records based on the language processing model to obtain information item vector representation; calculating the cosine similarity between the information item recorded by the client and the information item recorded by other clients to obtain an information item similarity score; performing weighted calculation on the information item similarity score to obtain a total similarity score; judging whether the total similarity score is greater than or equal to a preset similarity threshold; and if not, determining the corresponding customer record pair as a potential same customer record, and executing a data merging operation to generate a unified customer information record. The method and the device can be applied to application scenes of a financial service system and a digital medical system, the client data in the system can be effectively merged and processed, and the processing accuracy and efficiency are improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology and can be applied to the financial and medical fields, and in particular to a customer data processing method, device, computer equipment and storage medium. Background Art

[0002] In the fintech and digital healthcare sectors, customer information datasets contain numerous duplicate or similar records. These records may originate from different channels or at different points in time. In the fintech sector, for example, customer A applies for a debit card at bank branch A and then applies for a personal loan at branch B. Because the systems are not synchronized in real time, branch B's customer record may differ from branch A's in details such as phone number and home address. Similarly, in digital healthcare, patient A may visit the internal medicine department for hypertension and then the surgical department for knee pain. The patient's basic information in the electronic medical record systems of the internal medicine and surgical departments may be inconsistent due to different input from different departments, such as updated contact information and discrepancies in medical history. This data redundancy and inconsistency severely impacts the accuracy and usability of customer information, creating difficulties in subsequent customer credit assessments, targeted marketing, medical diagnosis, and treatment plan development.

[0003] Traditional customer information processing methods rely on manual review, but this approach presents numerous problems. In the fintech sector, banks' manual review teams spend a significant amount of time comparing customer information from various channels, verifying each record individually for duplicates. This is not only inefficient but also prone to fatigue and negligence when processing large amounts of data. In digital healthcare, medical staff must manually organize and compare patient records from different departments, which not only distracts them from patient care but can also lead to delays in manual processing, impacting the timeliness of diagnosis and treatment. Therefore, both traditional methods and manual review struggle to efficiently and accurately identify and merge duplicate records when faced with the diversity and complexity of customer information. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a customer data processing method, apparatus, computer equipment and storage medium to solve the problem of low efficiency and low accuracy in the merging processing of customer data in the system.

[0005] In a first aspect, an embodiment of the present application provides a method for processing customer data, which adopts the following technical solution:

[0006] Obtaining a customer information data set, parsing the customer information data set, and obtaining customer records;

[0007] Performing multi-dimensional vectorization processing on the information items in the customer record based on a pre-trained language processing model to obtain a vector representation of the information item;

[0008] Calculating the cosine similarity between the information item of the customer record and the information item of other customer records based on the information item vector representation to obtain an information item similarity score;

[0009] Performing weighted calculation on the similarity scores of the information items based on a preset weight configuration table to obtain a total similarity score corresponding to each pair of customer records;

[0010] Determining whether the total similarity score is greater than or equal to a preset similarity threshold;

[0011] If the total similarity score is greater than or equal to the preset similarity threshold, the customer record pair corresponding to the total similarity score is determined as a potential identical customer record, and a data merging operation is performed on the potential identical customer record to generate a unified customer information record.

[0012] In a second aspect, an embodiment of the present application further provides a client data processing device, which adopts the following technical solution:

[0013] An information acquisition module is used to acquire a customer information data set, parse the customer information data set, and obtain customer records;

[0014] An information processing module, configured to perform multi-dimensional vectorization processing on the information items in the customer records based on a pre-trained language processing model to obtain vector representations of the information items;

[0015] A similarity calculation module, configured to calculate the cosine similarity between the information item of the customer record and the information items of other customer records based on the information item vector representation, to obtain an information item similarity score;

[0016] a score calculation module, configured to perform weighted calculation on the similarity scores of the information items based on a preset weight configuration table to obtain a total similarity score corresponding to each pair of customer records;

[0017] A threshold judgment module is used to judge whether the total similarity score is greater than or equal to a preset similarity threshold;

[0018] The data operation module is used to determine the customer record pair corresponding to the total similarity score as a potential identical customer record if the total similarity score is greater than or equal to the preset similarity threshold, and perform a data merging operation on the potential identical customer record to generate a unified customer information record.

[0019] In a third aspect, an embodiment of the present application further provides a computer device that adopts the following technical solution:

[0020] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of any one of the above customer data processing methods when executing the computer-readable instructions.

[0021] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0022] A computer-readable storage medium stores computer-readable instructions, wherein the computer-readable instructions, when executed by a processor, implement the steps of any of the above methods for processing customer data.

[0023] Compared with the prior art, the embodiments of the present application have the following main beneficial effects: the embodiment obtains a customer information data set, parses the customer information data set, and obtains customer records; performs multi-dimensional vectorization processing on the information items in the customer records based on a pre-trained language processing model to obtain an information item vector representation; calculates the cosine similarity between the information items of the customer record and the information items of other customer records based on the information item vector representation to obtain an information item similarity score; performs weighted calculation on the information item similarity scores based on a preset weight configuration table to obtain a total similarity score corresponding to each pair of customer records; determines whether the total similarity score is greater than or equal to a preset similarity threshold; if the total similarity score is greater than or equal to the preset similarity threshold, the customer record pair corresponding to the total similarity score is determined as a potential identical customer record, and performs a data merge operation on the potential identical customer records to generate a unified customer information record. This effectively realizes the effective merging of customer data, saves system memory, and improves the efficiency of subsequent customer data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the solutions in this application, a brief introduction will be given below to the drawings required for use in the description of the embodiments of this application. Obviously, the drawings described below are some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0025] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0026] Figure 2 A flowchart of an embodiment of a method for processing customer data according to the present application;

[0027] Figure 3 is a structural diagram of an embodiment of a client data processing device according to the present application;

[0028] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0030] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to unrelated or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0031] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0032] like Figure 1 As shown, system architecture 100 may include a terminal device 101, a network 102, and a server 103. Terminal device 101 may be a laptop computer 1011, a tablet computer 1012, or a mobile phone 1013. Network 102 is a medium for providing a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0033] The user can use the terminal device 101 to interact with the server 103 via the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0034] The terminal device 101 can be various electronic devices with a display screen and supporting web browsing. In addition to the laptop computer 1011, tablet computer 1012 or mobile phone 1013, the terminal device 101 can also be an e-book reader, MP3 player (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer IV) player, laptop computer and desktop computer, etc.

[0035] The server 103 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal device 101 .

[0036] It should be noted that the client data processing method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the client data processing device is generally set in the server / terminal device.

[0037] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0038] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for processing customer data according to the present application. The customer data processing method includes the following steps:

[0039] Step S10, obtaining a customer information data set, parsing the customer information data set, and obtaining customer records;

[0040] In this embodiment, a customer information dataset includes customer records corresponding to multiple different customers. This customer information dataset can be obtained by collecting customer data information through different business systems. In the field of FinTech, financial institutions (such as banks and securities companies) collect various customer information. This information may include basic customer information (name, ID number, contact information, etc.), transaction records (transaction amount, transaction time, transaction type, etc.), credit scores, investment preferences, etc. For example, banks collect this data from their core business systems, online banking platforms, mobile banking applications, and other channels, forming a large customer information dataset. In the field of digital healthcare, medical institutions (such as hospitals and clinics) collect various patient information. This information includes basic patient information (name, ID number, contact information, etc.), medical records (disease diagnosis, treatment process, test results, etc.), drug allergy history, family medical history, and health monitoring data (such as blood pressure, blood sugar, etc.). For example, hospitals collect this data from their electronic medical record systems and medical Internet of Things devices (such as smart bracelets and remote blood pressure monitors) to form patient information datasets.

[0041] Step S20, performing multi-dimensional vectorization processing on the information items in the customer record based on the pre-trained language processing model to obtain a vector representation of the information item;

[0042] In this embodiment, the collected customer information dataset is parsed and broken down into individual customer records. Each customer record contains information about different aspects of the customer. For example, in a financial business system, a customer record may contain information such as customer A's name is "Zhang San", ID number is "1234xxxxxxxxxx5678", the most recent transaction amount is "5,000 yuan", the transaction time is "2024-12-25", and the investment preference is "conservative". In a digital healthcare system, a patient record may contain information such as patient A's name is "Zhou Qi", ID number is "3456xxxxxxxxxx7890", the most recent disease diagnosis is "hypertension", the treatment process is "taking antihypertensive medication", the drug allergy history is "none", and the most recent blood pressure measurement result is "140 / 90 mmHg". A pre-trained language processing model (such as a BERT model) is used to vectorize the text information in the customer record (such as customer feedback, investment preference description, etc.). At the same time, numerical information (such as transaction amount and credit score) is normalized and converted into vectors. Categorical information (such as transaction type: transfer, consumption, financial management, etc.) is one-hot encoded and converted into vectors. In this way, each information item is converted into a vector representation. For example, customer A's investment preference of "conservative" is converted into a specific text vector, the transaction amount of "5,000 yuan" is normalized and converted into a numerical vector, and the transaction type of "consumption" is one-hot encoded into a categorical vector.

[0043] Step S30, calculating the cosine similarity between the information item of the customer record and the information items of other customer records based on the information item vector representation to obtain an information item similarity score;

[0044] In this embodiment, the cosine similarity of the same information item between different customer records is calculated based on the vector representation of the information item. For example, the cosine similarity between the transaction amount vectors of customer A and customer B, the cosine similarity between the investment preference text vectors, etc. are calculated to obtain the similarity score of each information item. If the transaction amount vectors of customer A and customer B are a and b respectively, the cosine similarity calculation formula is

[0045]

[0046] Where "a·b" represents the dot product of vectors a and b, "||a||" and "||b||" represent the moduli of vectors a and b, respectively.

[0047] Step S40, performing weighted calculation on the similarity scores of the information items based on a preset weight configuration table to obtain a total similarity score corresponding to each pair of customer records;

[0048] In this embodiment, the similarity score for each information item is weighted according to a preset weighting table. Different information items have different importance. For example, assuming the transaction amount similarity score has a weight of 0.4, the investment preference similarity score has a weight of 0.3, and the contact information similarity score has a weight of 0.3, then the total similarity score between customers A and B = transaction amount similarity score × 0.4 + investment preference similarity score × 0.3 + contact information similarity score × 0.3.

[0049] Step S50, determining whether the total similarity score is greater than or equal to a preset similarity threshold;

[0050] In this embodiment, by comparing the total similarity score with the preset similarity threshold, the difference between the total similarity score and the preset similarity threshold is determined, thereby providing a reliable logical judgment basis for subsequent processing.

[0051] Step S60: If the total similarity score is greater than or equal to the preset similarity threshold, the customer record pair corresponding to the total similarity score is determined as a potential identical customer record, and a data merging operation is performed on the potential identical customer records to generate a unified customer information record;

[0052] In this embodiment, potentially identical customer records refer to identical customer records. For example, if the total similarity score is greater than or equal to a preset similarity threshold, the records of Customer A and Customer B are identified as potentially identical customer records, and a data merge operation is performed to generate a unified customer information record. In this embodiment, the preset similarity threshold is set to 0.8, which can be adjusted accordingly based on actual circumstances.

[0053] Step S70: If the total similarity score is less than the preset similarity threshold, the customer record pair corresponding to the total similarity score is determined as a non-potential identical customer record.

[0054] In this embodiment, non-potentially identical customer records refer to non-identical customer records. For example, if the total similarity score is less than a threshold, the records of customer A and customer B are determined to be non-potentially identical customer records.

[0055] This embodiment obtains a customer information dataset, parses the customer information dataset, and obtains customer records. Based on a pre-trained language processing model, multi-dimensional vectorization is performed on the information items in the customer records to obtain an information item vector representation. Based on the information item vector representation, the cosine similarity between the information items in the customer record and those in other customer records is calculated to obtain an information item similarity score. Based on a preset weight configuration table, the information item similarity scores are weighted to obtain a total similarity score corresponding to each pair of customer records. It is determined whether the total similarity score is greater than or equal to a preset similarity threshold. If the total similarity score is greater than or equal to the preset similarity threshold, the customer record pair corresponding to the total similarity score is determined as a potential identical customer record, and a data merge operation is performed on the potential identical customer records to generate a unified customer information record. If the total similarity score is less than the preset similarity threshold, the customer record pair corresponding to the total similarity score is determined as a non-potential identical customer record. This effectively achieves efficient merging of customer data, saves system memory, and improves the efficiency of subsequent customer data processing.

[0056] The method of this embodiment can be applied to client data processing in digital medical systems. Hospitals collect patient information from the electronic medical record systems of various departments, including basic information (name, ID number, contact information, etc.), medical records (disease diagnosis, treatment process, test results, etc.), drug allergy history, family medical history, and health monitoring data (such as blood pressure, blood sugar, etc.). For example, in Department A's system, there is patient record 1: name "Li Si", ID number "9876xxxxxxxxxx5432", contact information "139xxxxx000", most recent disease diagnosis "hypertension", treatment process "taking antihypertensive drugs", drug allergy history "none", and most recent blood pressure measurement result "140 / 90 mmHg". In Department B's system, there is patient record 2: name "Li Si", ID number "9876xxxxxxxxxx5432", contact information "139xxxxx001", most recent disease diagnosis "hypertension", treatment process "taking antihypertensive drugs", drug allergy history "none", and most recent blood pressure measurement result "142 / 92 mmHg". The hospital parses the collected patient information dataset to obtain specific information items for each patient record. It then uses a pre-trained language processing model to vectorize the text information in the patient records. The disease diagnosis "hypertension" is converted into a text vector. Numerical information is normalized and converted into vectors, with the blood pressure measurements "140 / 90 mmHg" and "142 / 92 mmHg" normalized and converted into numeric vectors. Categorical information is one-hot encoded and converted into vectors, with the disease diagnosis "hypertension" converted into a categorical vector. Ultimately, each information item in each patient record is converted into a vector representation. Based on these vector representations, the hospital calculates the cosine similarity of the same information item between patient records 1 and 2. Assume the cosine similarity of the blood pressure measurement results is 0.92, the cosine similarity of the disease diagnoses is 0.98, and the cosine similarity of the contact information is 0.88. The hospital weights the similarity scores for each information item according to a pre-set weighting table. Assume that the blood pressure measurement similarity score has a weight of 0.4, the disease diagnosis similarity score has a weight of 0.3, and the contact information similarity score has a weight of 0.3. The total similarity score between Patient Record 1 and Patient Record 2 is: 0.92 × 0.4 + 0.98 × 0.3 + 0.88 × 0.3 = 0.926. The hospital compares the calculated total similarity score of 0.926 with the preset similarity threshold (assuming it is 0.8). Because the total similarity score is greater than or equal to the threshold, the bank identifies Patient Record 1 and Patient Record 2 as potentially identical customer records and performs a data merge operation to generate a unified patient information record.

[0057] The method of this embodiment can be applied to customer data processing in financial business systems. Banks collect customer information from the customer management systems of various branches, including basic information (name, ID number, contact information, etc.), transaction records (transaction amount, transaction time, transaction type, etc.), credit scores, investment preferences, and so on. For example, Branch A's system contains customer record 1: name "Zhang San", ID number "1234xxxxxxxxxx5678", contact information "138xxxxx000", most recent transaction amount "5000 yuan", transaction type "transfer", credit score "750", and investment preference "conservative". Branch B's system contains customer record 2: name "Zhang San", ID number "1234xxxxxxxxxx5678", contact information "138xxxxx001", most recent transaction amount "5100 yuan", transaction type "transfer", credit score "755", and investment preference "conservative". The bank uses a pre-trained language processing model to vectorize the text information in the customer record. For example, a BERT model customized for the financial field is used to convert the investment preference "conservative" into a text vector. At the same time, the numerical information is normalized and converted into vectors. The transaction amounts "5,000 yuan" and "5,100 yuan" are normalized and converted into numerical vectors. Categorical information is one-hot encoded and converted into vectors, and the transaction type "transfer" is converted into a categorical vector. Ultimately, each information item in each customer record is converted into a vector representation. Based on the vector representations of these information items, the bank calculates the cosine similarity of the same information item between customer records 1 and 2. For example, the bank calculates the cosine similarity between transaction amount vectors and investment preference text vectors to obtain a similarity score for each information item. Assume the cosine similarity for transaction amounts is 0.95, the cosine similarity for investment preferences is 0.98, and the cosine similarity for contact information is 0.85. The bank then weights the similarity scores for each information item according to a pre-set weighting table. Assume the weight for the transaction amount similarity score is 0.4, the weight for the investment preference similarity score is 0.3, and the weight for the contact information similarity score is 0.3. The total similarity score between Customer Record 1 and Customer Record 2 is: 0.95 × 0.4 + 0.98 × 0.3 + 0.85 × 0.3 = 0.931. The bank compares the calculated total similarity score of 0.931 with the preset similarity threshold (assuming it is 0.8). Because the total similarity score is greater than or equal to the threshold, the bank identifies Customer Record 1 and Customer Record 2 as potentially identical customer records and performs a data merge operation to generate a unified customer information record.

[0058] In some optional implementations of this embodiment, obtaining a customer information dataset, parsing the customer information dataset, and obtaining customer records includes the following steps:

[0059] Obtaining a customer information extraction identifier, and extracting a customer information dataset from a database according to the customer information extraction identifier;

[0060] In this embodiment, the customer information extraction identifier is a unique identifier corresponding to the customer information data set. The customer information extraction identifier is used as a query condition to perform a traversal query in the database to extract the corresponding customer information data set.

[0061] Preprocessing the customer information dataset to obtain a standard customer information dataset;

[0062] In this embodiment, preprocessing operations include removing duplicate data, correcting erroneous information (such as correcting incorrectly formatted phone numbers and dates), supplementing missing values ​​(for example, filling in missing customer address information based on business rules or statistical methods), and standardizing data formats (converting dates in different formats to a standard format, such as "YYYY-MM-DD"). Preprocessing the customer information dataset ensures data quality and consistency, laying a solid foundation for subsequent information extraction and processing.

[0063] Key information is extracted from the customer information data set according to preset information extraction rules to obtain the customer record.

[0064] In this embodiment, key information is extracted from a preprocessed standard customer information dataset based on preset information extraction rules. These preset rules are developed based on business needs and data characteristics to identify key information that requires focus and extraction. For example, within customer information, name, contact information, transaction amount, purchase date, and other key information items may be key information items. By applying data mining, text processing, pattern matching, and other technical means, these key information items are identified and extracted from the standard customer information dataset according to the preset rules, ultimately obtaining customer records containing key customer information.

[0065] This embodiment obtains a customer information extraction identifier, extracts a customer information dataset from a database based on the identifier, preprocesses the customer information dataset to obtain a standard customer information dataset, and extracts key information from the customer information dataset based on preset information extraction rules to obtain the customer record. This effectively obtains customer records and provides reliable data support for subsequent processing steps.

[0066] In some optional implementations of this embodiment, the information items include: text information, numerical information, categorical information, and geographic location information, and performing multidimensional vectorization processing on the information items in the customer records based on the pre-trained language processing model to obtain vector representations of the information items includes the following steps:

[0067] Extracting information items from the customer record to obtain the text information, the numerical information, the category information, and the geographic location information;

[0068] In this embodiment, information items are extracted from customer records and divided into textual information, numerical information, categorical information, and geographic location information. Textual information typically includes descriptive content, such as customer feedback and descriptions of interests and hobbies; numerical information involves quantifiable data, such as transaction amount, age, and consumption frequency; categorical information refers to data with limited classification options, such as gender (male, female, other), membership level (regular member, senior member, VIP member), and education level (high school, undergraduate, master's, doctoral); geographic location information is primarily data related to geographic location, such as home address, detailed address of workplace, and latitude and longitude of store locations. The above process can be implemented through technologies such as text analysis and pattern recognition, separating different types of information such as text, numerical values, categories, and geographic location from customer records to facilitate subsequent targeted feature extraction.

[0069] Inputting the text information, the numerical information, the category information, and the geographic location information into the language processing model for feature extraction to obtain text information features, numerical information features, category information features, and location information features;

[0070] In this embodiment, the language processing model can adopt the BERT model, and the extracted text information, numerical information, category information, and geographic location information are respectively input into the language processing model for feature extraction. For text information, the pre-trained language processing model can extract semantically rich text information features based on the semantics and contextual relationships of the text; for numerical information, the model can learn its numerical size, distribution and other features, such as the high and low transaction amount, the age distribution, etc., and then form numerical information features; after the category information is processed, the model can extract its features under different categories, such as the distribution characteristics of gender in different categories (male, female, other), reflecting the contribution of gender category to customer characteristics; geographic location information can extract features related to geographic location, such as the distribution characteristics of customers in different regions, the distance characteristics between geographic locations, etc.

[0071] Feature vector fusion is performed on the text information feature, the numerical information feature, the category information feature, and the position information feature to obtain the information item vector representation.

[0072] In this embodiment, feature vector fusion is performed on the extracted text information features, numerical information features, category information features, and location information features. Feature vector fusion can be achieved through various methods, such as simple concatenation, where each feature vector is connected in a certain order to form a long vector; or weighted fusion, where each feature is assigned a different weight based on its importance and relevance, and then a weighted sum is performed to obtain a fused vector. More complex neural network structures, such as multi-layer perceptrons (MLPs), can also be used to automatically learn the associations and fusion methods between different features, so that the fused vector can more comprehensively and accurately represent the comprehensive characteristics of the customer record. The information item vector representation obtained after fusion integrates various types of information features to form a unified vector representation that comprehensively reflects the customer record. This unified vector representation is the information item vector representation.

[0073] This embodiment extracts information items from the customer records to obtain the textual information, numerical information, categorical information, and geographic location information; inputs the textual information, numerical information, categorical information, and geographic location information into the language processing model for feature extraction to obtain textual information features, numerical information features, categorical information features, and location information features; and performs feature vector fusion on the textual information features, numerical information features, categorical information features, and location information features to obtain the information item vector representation. This effectively implements multi-dimensional vectorization of the information items in the customer records to obtain the corresponding information item vector representation, facilitating subsequent calculation of information item similarity scores.

[0074] In some optional implementations of this embodiment, calculating the cosine similarity between the information item of the customer record and the information items of other customer records based on the information item vector representation to obtain the information item similarity score includes the following steps:

[0075] Extracting information items from a current customer record and information items from a target customer record, and using the information items from the current customer record and the information items from the target customer record as a customer information item pair;

[0076] In this embodiment, information items are extracted from the current customer record and the target customer record. These information items may include textual information (such as name, address, and description of hobbies and interests), numerical information (such as age and transaction amount), categorical information (such as gender and membership level), and geographic location information (such as home address coordinates and workplace coordinates). The information items in the current customer record are paired with the corresponding information items in the target customer record to form customer information item pairs. For example, the name in the current customer record is paired with the name in the target customer record, the transaction amount in the current customer record is paired with the transaction amount in the target customer record, and so on, preparing for subsequent similarity calculations.

[0077] The cosine similarity of the corresponding information items between the pairs of customer information items is calculated to obtain an information item similarity value, and the information item similarity value is used as the information item similarity score.

[0078] In this embodiment, the cosine similarity between each pair of customer information items is calculated. Cosine similarity is used to measure the similarity between two vectors, and its value ranges from -1 to 1, with values ​​closer to 1 indicating more similar vectors. For textual information item pairs, the text can be first converted into vectors (e.g., using methods such as a bag-of-words model, TF-IDF, or a pre-trained language model), and then cosine similarity is calculated. For numerical information item pairs, the numerical values ​​can be directly treated as vector elements to calculate cosine similarity. Categorical information item pairs can be one-hot encoded and converted into vectors before cosine similarity is calculated. For geographic location information item pairs, the coordinates can be converted into vectors to calculate cosine similarity. The calculated cosine similarity value is the similarity score for the information item pair, reflecting the degree of similarity between the current customer record and the target customer record in the corresponding information item. For example, if the cosine similarity between the name information item pair of the current customer and the target customer is 0.9, it indicates that they have a high degree of similarity in terms of name; if the cosine similarity between the transaction amount information item pair is 0.7, it indicates that they also have a certain degree of similarity in terms of transaction amount. In this way, the similarity score of each information item pair can be obtained separately, providing a quantitative basis for the subsequent comprehensive evaluation of the overall similarity of customer records.

[0079] This embodiment extracts information items from the current customer record and target customer record, treating them as a customer information item pair; calculates the cosine similarity of corresponding information items between the customer information item pairs to obtain an information item similarity value, and uses this information item similarity value as the information item similarity score. This effectively calculates the similarity between information items in different customer records, obtaining the corresponding information item similarity score, and facilitating the subsequent calculation of the total similarity score.

[0080] In some optional implementations of this embodiment, performing weighted calculation on the information item similarity scores based on a preset weight configuration table to obtain a total similarity score corresponding to each pair of customer records includes the following steps:

[0081] Obtaining a configuration table extraction identifier, and extracting the preset weight configuration table from a database according to the configuration table extraction identifier;

[0082] In this embodiment, the configuration table extraction identifier is a unique identifier corresponding to the preset weight configuration table, and the configuration table extraction identifier is used as a query condition to perform a traversal query in the database to extract the corresponding preset weight configuration table.

[0083] Obtaining a corresponding information item weight in the preset weight configuration table according to the information item;

[0084] In this embodiment, for each information item for which similarity needs to be calculated, its name or identifier is matched with the "Information Item Name" column in the preset weight configuration table. For example, if the information item currently being processed is "name", the row corresponding to "name" is found in the weight configuration table. Once the match is successful, the corresponding information item weight is obtained from the "Weight Value" column of the row. This weight value indicates the importance of the information item in calculating the total similarity score. For example, if the weight value of "name" in the weight configuration table is 0.3, it means that in the calculation of the total similarity score, the similarity score of the information item "name" will be multiplied by 0.3 as its contribution to the total score.

[0085] The total similarity score is obtained by performing a weighted summation based on the information item weight and the information item similarity score.

[0086] In this embodiment, for each customer record pair, after calculating the similarity score of each information item (such as the name similarity score, age similarity score, etc.), each information item similarity score is multiplied by its corresponding information item weight. For example, if the name similarity score is 0.8 and the weight is 0.3, the name portion contributes 0.8 × 0.3 = 0.24 to the total similarity score; if the age similarity score is 0.6 and the weight is 0.2, the age portion contributes 0.6 × 0.2 = 0.12. Then, the contribution values ​​of all information items are added together, that is, 0.24 + 0.12 + ... (the contribution of other information items) to obtain the total similarity score. This total similarity score comprehensively considers the similarity of each information item and their importance, thereby more accurately reflecting the overall similarity between two customer records.

[0087] This embodiment obtains a configuration table extraction identifier, extracts the preset weight configuration table from the database based on the configuration table extraction identifier, obtains the corresponding information item weight from the preset weight configuration table based on the information item, and performs a weighted summation of the information item weight and the information item similarity score to obtain the total similarity score. This effectively calculates the total similarity score representing the similarity between two overall customer records, facilitating subsequent threshold determination processing.

[0088] In some optional implementations of this embodiment, determining whether the total similarity score is greater than or equal to a preset similarity threshold includes the following steps:

[0089] Obtaining a threshold extraction identifier, and extracting the preset similarity threshold from a database according to the threshold extraction identifier;

[0090] In this embodiment, the threshold extraction identifier is a unique identifier corresponding to a preset similarity threshold, and a traversal query is performed in the database using the threshold extraction identifier as a query condition to extract the corresponding preset similarity threshold.

[0091] The total similarity score is numerically compared with the preset similarity threshold.

[0092] In this embodiment, the calculated total similarity score is directly compared to a preset similarity threshold. This process determines whether a pair of customer records is considered potentially identical. If the total similarity score is greater than or equal to the threshold, the two customer records are considered to belong to the same customer; conversely, if the total similarity score is less than the threshold, they are considered to belong to different customers.

[0093] This embodiment obtains a threshold extraction identifier, extracts the preset similarity threshold from a database based on the threshold extraction identifier, and compares the total similarity score with the preset similarity threshold. This effectively determines the difference between the total similarity score and the preset similarity threshold, facilitating subsequent processing based on the judgment result.

[0094] In some optional implementations of this embodiment, the customer record pair includes a current customer record and a target customer record, and determining the customer record pair corresponding to the total similarity score as a potential identical customer record, and performing a data merging operation on the potential identical customer records to generate a unified customer information record includes the following steps:

[0095] Identifying the current customer record and the target customer record corresponding to the total similarity score;

[0096] In this embodiment, after calculating the total similarity score, it is necessary to identify the current customer record and target customer record to which the score corresponds. This step is to accurately locate which two customer records have the highest similarity so that they can be further processed. By tracking and identifying record identifiers (such as record numbers, customer IDs, etc.), it is possible to ensure that subsequent operations target the correct customer record pair.

[0097] Determining the current customer record and the target customer record as the potential identical customer records, and generating a merging operation strategy based on the potential identical customer records;

[0098] In this embodiment, once the total similarity score meets certain similarity conditions (meaning the total similarity score is greater than or equal to a preset similarity threshold), the corresponding current customer record and target customer record are determined as potential identical customer records. At this point, the system believes that the two records are likely to belong to the same customer, and generates a merge operation strategy based on these potential identical customer records. The merge operation strategy is to guide how to integrate the information in the two records to form a unified customer information record. The strategy includes determining which information items need to be retained, which information items need to be updated, how to handle conflicting information (such as when the contact information in different records is inconsistent, which one to select as the final contact information), and the order and method of merging. For example, the strategy may stipulate that when merging, more complete address information should be retained first, or newer contact information should be selected.

[0099] A data merging operation is performed on the potential identical customer records based on the merging operation strategy to generate the unified customer information record.

[0100] In this embodiment, based on the merge operation strategy generated above, a data merge operation is performed on potentially identical customer records. Specifically, the various information items in the two records are integrated according to the rules specified in the strategy. For consistent information items, their values ​​are directly retained; for inconsistent information items, they are processed according to the conflict resolution rules in the strategy, such as taking an average, selecting the latest, most complete, or highest priority information item, etc. Ultimately, through the merge operation, a unified customer information record is generated. This record integrates the valid information from the two potentially identical customer records, eliminates duplicate and inconsistent information, and forms a complete, accurate, and unique customer information record, so that customer data can be used and managed uniformly in subsequent business scenarios such as customer management and data analysis.

[0101] This embodiment identifies the current customer record and the target customer record corresponding to the total similarity score; determines the current customer record and the target customer record as potential identical customer records; generates a merge operation strategy based on the potential identical customer records; and performs a data merge operation on the potential identical customer records based on the merge operation strategy to generate the unified customer information record. This effectively achieves unified merging of information between different customer records, conserves system memory, and effectively improves the efficiency of subsequent customer data processing.

[0102] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes in the above-described method embodiments. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0103] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0104] Further references Figure 3 , as a response to the above Figure 1 In order to realize the method shown in FIG, the present application provides an embodiment of a client data processing device. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0105] like Figure 3 As shown, the customer data processing device 700 described in this embodiment includes: an information acquisition module 801, an information processing module 802, a similarity calculation module 803, a score calculation module 804, a threshold judgment module 805, a data operation module 806, and a record determination module 807.

[0106] in:

[0107] The information acquisition module 801 is used to acquire a customer information data set, parse the customer information data set, and obtain customer records;

[0108] An information processing module 802 is configured to perform multi-dimensional vectorization processing on the information items in the customer record based on a pre-trained language processing model to obtain a vector representation of the information items;

[0109] A similarity calculation module 803 is configured to calculate the cosine similarity between the information item of the customer record and the information items of other customer records based on the information item vector representation to obtain an information item similarity score;

[0110] A score calculation module 804 is configured to perform weighted calculation on the similarity scores of the information items based on a preset weight configuration table to obtain a total similarity score corresponding to each pair of customer records;

[0111] A threshold determination module 805 is configured to determine whether the total similarity score is greater than or equal to a preset similarity threshold;

[0112] A data operation module 806 is configured to determine the customer record pair corresponding to the total similarity score as a potential identical customer record if the total similarity score is greater than or equal to the preset similarity threshold, and perform a data merging operation on the potential identical customer record to generate a unified customer information record;

[0113] The record determination module 807 is configured to determine the customer record pair corresponding to the total similarity score as a non-potential identical customer record if the total similarity score is less than the preset similarity threshold.

[0114] By employing the above-mentioned customer data processing device, this embodiment can obtain a customer information dataset, parse the customer information dataset to obtain customer records; perform multidimensional vectorization processing on the information items in the customer records based on a pre-trained language processing model to obtain an information item vector representation; calculate the cosine similarity between the information items in the customer record and those in other customer records based on the information item vector representation to obtain an information item similarity score; perform weighted calculation on the information item similarity scores based on a preset weight configuration table to obtain a total similarity score corresponding to each pair of customer records; determine whether the total similarity score is greater than or equal to a preset similarity threshold; if the total similarity score is greater than or equal to the preset similarity threshold, determine the customer record pair corresponding to the total similarity score as a potential identical customer record, and perform a data merge operation on the potential identical customer records to generate a unified customer information record. This effectively implements efficient merging of customer data, saves system memory, and improves the efficiency of subsequent customer data processing.

[0115] In some optional implementations of this embodiment, the information acquisition module 801 includes: a data set extraction unit, a data set preprocessing unit, and a key information extraction unit.

[0116] The data set extraction unit is used to obtain a customer information extraction identifier and extract a customer information data set from a database according to the customer information extraction identifier;

[0117] The data set preprocessing unit is used to preprocess the customer information data set to obtain a standard customer information data set;

[0118] The key information extraction unit is configured to extract key information from the customer information dataset according to preset information extraction rules to obtain the customer record.

[0119] This embodiment provides an information acquisition module 801 including a data set extraction unit, a data set pre-processing unit, and a key information extraction unit, thereby achieving effective acquisition of customer records to provide reliable data support for subsequent processing steps.

[0120] In some optional implementations of this embodiment, the information processing module 802 includes: an information item extraction unit, a feature extraction unit, and a feature vector fusion unit.

[0121] The information item extraction unit is used to extract information items from the customer record to obtain the text information, the numerical information, the category information, and the geographic location information;

[0122] The feature extraction unit is used to input the text information, the numerical information, the category information, and the geographic location information into the language processing model for feature extraction to obtain text information features, numerical information features, category information features, and location information features;

[0123] The feature vector fusion unit is used to perform feature vector fusion on the text information feature, the numerical information feature, the category information feature, and the position information feature to obtain the information item vector representation.

[0124] This embodiment effectively implements multi-dimensional vectorization processing of information items in customer records by setting up an information processing module 802 including an information item extraction unit, a feature extraction unit, and a feature vector fusion unit to obtain corresponding information item vector representations, thereby facilitating subsequent information item similarity score calculations.

[0125] In some optional implementations of this embodiment, the similarity calculation module 803 includes: an information item pair determination unit and a similarity value calculation unit.

[0126] The information item pair determination unit is configured to extract information items from a current customer record and information items from a target customer record, and use the information items from the current customer record and the information items from the target customer record as a customer information item pair;

[0127] The similarity value calculation unit is configured to calculate the cosine similarity of corresponding information items between the pair of customer information items to obtain an information item similarity value, and use the information item similarity value as the information item similarity score.

[0128] This embodiment provides a similarity calculation module 803 including an information item pair determination unit and a similarity value calculation unit, thereby effectively calculating the similarity between information items of different customer records and obtaining corresponding information item similarity scores to facilitate subsequent calculation of the total similarity score.

[0129] In some optional implementations of this embodiment, the score calculation module 804 includes: a configuration table extraction unit, a weight acquisition unit, and a weighted calculation unit.

[0130] The configuration table extraction unit is configured to obtain a configuration table extraction identifier and extract the preset weight configuration table from a database according to the configuration table extraction identifier;

[0131] The weight obtaining unit is configured to obtain the corresponding information item weight in the preset weight configuration table according to the information item;

[0132] The weighted calculation unit is configured to perform weighted summation based on the information item weight and the information item similarity score to obtain the total similarity score.

[0133] This embodiment provides a score calculation module 804 including a configuration table extraction unit, a weight acquisition unit, and a weighted calculation unit, thereby effectively calculating the total similarity score representing the similarity between two overall customer records, to facilitate subsequent threshold judgment processing.

[0134] In some optional implementations of this embodiment, the threshold determination module 805 includes: a preset threshold extraction unit and a numerical value comparison unit.

[0135] The preset threshold extraction unit is used to obtain a threshold extraction identifier and extract the preset similarity threshold from a database according to the threshold extraction identifier;

[0136] The numerical comparison unit is used to perform numerical comparison between the total similarity score and the preset similarity threshold.

[0137] This embodiment provides a threshold judgment module 805 including a preset threshold extraction unit and a numerical comparison unit, thereby effectively realizing the judgment of the size between the total similarity score and the preset similarity threshold, so as to facilitate subsequent corresponding processing operations based on the judgment result.

[0138] In some optional implementations of this embodiment, the data operation module 806 includes: a customer record identification unit, a strategy generation unit, and a data merging operation unit.

[0139] The customer record identification unit is configured to identify the current customer record and the target customer record corresponding to the total similarity score;

[0140] The strategy generating unit is configured to determine the current customer record and the target customer record as the potential identical customer records, and generate a merging operation strategy based on the potential identical customer records;

[0141] The data merging operation unit is configured to perform a data merging operation on the potential identical customer records based on the merging operation strategy to generate the unified customer information record.

[0142] This embodiment provides a data operation module 806 including a customer record identification unit, a policy generation unit, and a data merging operation unit, thereby effectively merging information of different customer records to save system memory and effectively improve the efficiency of subsequent customer data processing.

[0143] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0144] The computer device 9 includes a memory 91, a processor 92, and a network interface 93 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 9 with components 91-93, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0145] The computer device may be a desktop computer, notebook computer, PDA, cloud server, etc. The computer device may interact with the user via a keyboard, mouse, remote control, touchpad, or voice control device.

[0146] The memory 91 includes at least one type of readable storage medium, including flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, a magnetic disk, an optical disk, etc. In some embodiments, the memory 91 may be an internal storage unit of the computer device 9, such as the hard disk or memory of the computer device 9. In other embodiments, the memory 91 may also be an external storage device of the computer device 9, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the computer device 9. Of course, the memory 91 may also include both the internal storage unit of the computer device 9 and its external storage device. In this embodiment, the memory 91 is generally used to store the operating system and various application software installed on the computer device 9, such as computer-readable instructions of the customer data processing method. In addition, the memory 91 can also be used to temporarily store various types of data that have been output or are to be output.

[0147] In some embodiments, the processor 92 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 92 is generally used to control the overall operation of the computer device 9. In this embodiment, the processor 92 is used to execute computer-readable instructions stored in the memory 91 or process data, such as computer-readable instructions for executing the customer data processing method.

[0148] The network interface 93 may include a wireless network interface or a wired network interface. The network interface 93 is generally used to establish a communication connection between the computer device 9 and other electronic devices.

[0149] By using the above-mentioned computer device, this embodiment can obtain a customer information dataset, parse the customer information dataset to obtain customer records; perform multidimensional vectorization processing on the information items in the customer records based on a pre-trained language processing model to obtain an information item vector representation; calculate the cosine similarity between the information items of the customer record and the information items of other customer records based on the information item vector representation to obtain an information item similarity score; perform weighted calculation on the information item similarity scores based on a preset weight configuration table to obtain a total similarity score corresponding to each pair of customer records; determine whether the total similarity score is greater than or equal to a preset similarity threshold; if the total similarity score is greater than or equal to the preset similarity threshold, determine the customer record pair corresponding to the total similarity score as a potential identical customer record, and perform a data merge operation on the potential identical customer records to generate a unified customer information record. This effectively realizes the effective merging of customer data, saves system memory, and improves the efficiency of subsequent customer data processing.

[0150] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions. The computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the customer data processing method as described above.

[0151] By using the computer-readable storage medium, this embodiment can obtain a customer information dataset, parse the customer information dataset to obtain customer records, perform multi-dimensional vectorization processing on the information items in the customer records based on a pre-trained language processing model to obtain an information item vector representation, calculate the cosine similarity between the information items in the customer record and the information items in other customer records based on the information item vector representation to obtain an information item similarity score, perform weighted calculation on the information item similarity scores based on a preset weight configuration table to obtain a total similarity score corresponding to each pair of customer records, determine whether the total similarity score is greater than or equal to a preset similarity threshold, and if the total similarity score is greater than or equal to the preset similarity threshold, determine the customer record pair corresponding to the total similarity score as a potential identical customer record, and perform a data merge operation on the potential identical customer records to generate a unified customer information record. This effectively achieves effective merging of customer data, saves system memory, and improves the efficiency of subsequent customer data processing.

[0152] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0153] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.

[0154] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A customer data processing method, characterized in that: The steps include: Obtaining a customer information data set, parsing the customer information data set, and obtaining customer records; Performing multi-dimensional vectorization processing on the information items in the customer record based on a pre-trained language processing model to obtain a vector representation of the information item; Calculating the cosine similarity between the information item of the customer record and the information item of other customer records based on the information item vector representation to obtain an information item similarity score; Performing weighted calculation on the similarity scores of the information items based on a preset weight configuration table to obtain a total similarity score corresponding to each pair of customer records; Determining whether the total similarity score is greater than or equal to a preset similarity threshold; If the total similarity score is greater than or equal to the preset similarity threshold, the customer record pair corresponding to the total similarity score is determined as a potential identical customer record, and a data merging operation is performed on the potential identical customer record to generate a unified customer information record.

2. The customer data processing method according to claim 1, characterized in that: The step of obtaining a customer information data set, parsing the customer information data set, and obtaining customer records specifically includes: Obtaining a customer information extraction identifier, and extracting a customer information dataset from a database according to the customer information extraction identifier; Preprocessing the customer information dataset to obtain a standard customer information dataset; Key information is extracted from the customer information data set according to preset information extraction rules to obtain the customer record.

3. The customer data processing method according to claim 1, characterized in that: The information items include: text information, numerical information, category information, and geographic location information. The step of performing multi-dimensional vectorization processing on the information items in the customer records based on the pre-trained language processing model to obtain vector representations of the information items specifically includes: Extracting information items from the customer record to obtain the text information, the numerical information, the category information, and the geographic location information; Inputting the text information, the numerical information, the category information, and the geographic location information into the language processing model for feature extraction to obtain text information features, numerical information features, category information features, and location information features; Feature vector fusion is performed on the text information feature, the numerical information feature, the category information feature, and the position information feature to obtain the information item vector representation.

4. The customer data processing method according to claim 1, characterized in that: The step of calculating the cosine similarity between the information item of the customer record and the information item of other customer records based on the information item vector representation to obtain the information item similarity score specifically includes: Extracting information items from a current customer record and information items from a target customer record, and using the information items from the current customer record and the information items from the target customer record as a customer information item pair; The cosine similarity of the corresponding information items between the pairs of customer information items is calculated to obtain an information item similarity value, and the information item similarity value is used as the information item similarity score.

5. The customer data processing method according to claim 1, characterized in that: The step of performing weighted calculation on the similarity scores of the information items based on a preset weight configuration table to obtain a total similarity score corresponding to each pair of customer records specifically includes: Obtaining a configuration table extraction identifier, and extracting the preset weight configuration table from a database according to the configuration table extraction identifier; Obtaining a corresponding information item weight in the preset weight configuration table according to the information item; The total similarity score is obtained by performing a weighted summation based on the information item weight and the information item similarity score.

6. The customer data processing method according to claim 1, characterized in that: The step of determining whether the total similarity score is greater than or equal to a preset similarity threshold specifically includes: Obtaining a threshold extraction identifier, and extracting the preset similarity threshold from a database according to the threshold extraction identifier; The total similarity score is numerically compared with the preset similarity threshold.

7. The method for processing customer data according to claim 1, wherein: The customer record pair includes a current customer record and a target customer record. The steps of determining the customer record pair corresponding to the total similarity score as a potential identical customer record, and performing a data merging operation on the potential identical customer records to generate a unified customer information record specifically include: Identifying the current customer record and the target customer record corresponding to the total similarity score; Determining the current customer record and the target customer record as the potential identical customer records, and generating a merging operation strategy based on the potential identical customer records; A data merging operation is performed on the potential identical customer records based on the merging operation strategy to generate the unified customer information record.

8. A customer data processing device, characterized in that: include: An information acquisition module is used to acquire a customer information data set, parse the customer information data set, and obtain customer records; An information processing module, configured to perform multi-dimensional vectorization processing on the information items in the customer records based on a pre-trained language processing model to obtain vector representations of the information items; A similarity calculation module, configured to calculate the cosine similarity between the information item of the customer record and the information items of other customer records based on the information item vector representation, to obtain an information item similarity score; a score calculation module, configured to perform weighted calculation on the similarity scores of the information items based on a preset weight configuration table to obtain a total similarity score corresponding to each pair of customer records; A threshold judgment module is used to judge whether the total similarity score is greater than or equal to a preset similarity threshold; The data operation module is used to determine the customer record pair corresponding to the total similarity score as a potential identical customer record if the total similarity score is greater than or equal to the preset similarity threshold, and perform a data merging operation on the potential identical customer record to generate a unified customer information record.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the customer data processing method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the customer data processing method according to any one of claims 1 to 7.

Images