Entity alignment method based on heterogeneous data source and related device

Abstract

The invention discloses an entity alignment method based on a heterogeneous data source and a related device, and relates to the technical field of big data, and the method comprises the steps: obtaining the structural similarity of a first financial entity and a second financial entity based on a structural attribute set and a corresponding attribute value set; and based on the text attribute set of each financial entity in each financial entity library, obtaining the text similarity of the first financial entity and the second financial entity. And fusing the structural similarity and the text similarity of the first financial entity and the second financial entity, and calculating a comprehensive similarity. On the basis of the comprehensive similarity of the first financial entity and the second financial entity, alignment data of the financial entity library of the first data source and the financial entity library of the second data source are determined, and the alignment data are to-be-aligned entity pairs. Comprehensive similarity calculation is carried out on the financial entities in the financial entity library of the heterogeneous data source, and the accuracy and reliability of entity alignment are improved.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to an entity alignment method based on heterogeneous data sources and related devices. Background Art

[0002] With the continuous expansion of Internet finance, the data in the financial field has increased dramatically. The financial system needs to build a financial knowledge base and formulate risk control measures based on the financial risks in the financial knowledge base. The financial knowledge base includes a large number of financial entities and the relationships between entities. For example, there are legal person, shareholding, and controlling relationships between institutional entities and individual entities.

[0003] Due to the complexity and diversity of business in the financial field, the amount of data from financial data sources is huge. In the process of building a financial knowledge base, the existing financial knowledge is usually fused to obtain a complete and reliable financial knowledge base. There are differences in data structures between different financial data sources, and the reliability of financial knowledge base fusion is poor, which leads to the difficulty and low accuracy of financial knowledge base construction. Summary of the invention

[0004] In view of the above problems, the present application provides an entity alignment method based on heterogeneous data sources and related devices to achieve the purpose of improving the reliability of entity alignment based on heterogeneous data sources. The specific scheme is as follows:

[0005] The first aspect of the present application provides an entity alignment method based on heterogeneous data sources.

[0006] Extracting financial entities from heterogeneous data sources to obtain a financial entity library of the heterogeneous data sources, wherein the heterogeneous data sources include a first heterogeneous data source and a second heterogeneous data source;

[0007] Based on the structured attribute set and the corresponding attribute value set of each financial entity in each financial entity library, the structured similarity between the first financial entity and the second financial entity is obtained, the first financial entity is any financial entity in the financial entity library of the first data source, and the second financial entity is any financial entity in the financial entity library of the second data source; the structured attribute set includes a plurality of structured attributes, and the attribute value set includes the attribute value of each structured attribute.

[0008] Based on a text attribute set of each financial entity in each financial entity library, obtaining a text similarity between the first financial entity and the second financial entity, wherein the text attribute set includes a plurality of text attributes;

[0009] The structural similarity and text similarity of the first financial entity and the second financial entity are integrated to calculate the comprehensive similarity of the first financial entity and the second financial entity; based on the comprehensive similarity of the first financial entity and the second financial entity, the alignment data of the financial entity library of the first data source and the financial entity library of the second data source are determined, and the alignment data are entity pairs to be aligned, and the entity pairs to be aligned include two financial entities whose comprehensive similarity is greater than a preset alignment threshold.

[0010] In a possible implementation, financial entities are extracted from heterogeneous data sources to obtain a financial entity library of the heterogeneous data sources, including:

[0011] Acquiring initial data from the heterogeneous data source and performing data preprocessing to obtain data source data, wherein the data preprocessing includes grammatical normalization and data normalization;

[0012] Financial entities are identified from the data source data of the heterogeneous data sources, and financial entity information is extracted, wherein the financial entity information includes structured information and unstructured information, wherein the structured information includes structured attributes and attribute values, and the unstructured information includes text.

[0013] In a possible implementation, obtaining the structural similarity between the first financial entity and the second financial entity based on the structured attribute set and the corresponding attribute value set of each financial entity in each financial entity library includes:

[0014] Acquire common structured attribute data of the structured attribute sets of the first financial entity and the second financial entity, wherein the common structured attribute data includes a common structured attribute set and an attribute value group set, and the common structured attribute set includes at least one common structured attribute;

[0015] For each attribute value group corresponding to the common structured attribute, calculating the string edit distance between two attribute values ​​in the attribute value group;

[0016] Calculating the attribute similarity of each of the common structural attributes based on the string length and the string edit distance;

[0017] Based on the attribute similarity of each of the common structural attributes of the first financial entity and the second financial entity, the structural similarity of the first financial entity and the second financial entity is calculated.

[0018] In a possible implementation, based on the text attribute sets of each financial entity in each financial entity library, obtaining the text similarity between the first financial entity and the second financial entity includes:

[0019] Performing word segmentation on the text of the first financial entity and the text of the second financial entity to obtain a text attribute set of the first financial entity and the second financial entity;

[0020] Inputting the text attribute sets of the first financial entity and the second financial entity into a text feature extraction model respectively, obtaining a feature vector set of the first financial entity and the second financial entity output by the text feature extraction model, wherein the feature vector set includes a plurality of feature vectors;

[0021] Calculating text similarity between a first feature vector of the first financial entity and a feature vector of the second financial entity;

[0022] If the text similarity between the first feature vector and the second feature vector is greater than a preset similarity threshold, the text attribute corresponding to the first feature vector and the text attribute corresponding to the second feature vector are taken as a similar text attribute pair;

[0023] The average value of the text similarities of the similar text attribute pairs of the first financial entity and the second financial entity is used as the text similarity between the first financial entity and the second financial entity.

[0024] In a possible implementation, integrating the structural similarity and text similarity of the first financial entity and the second financial entity to calculate the comprehensive similarity of the first financial entity and the second financial entity includes:

[0025] Based on a preset weight coefficient, the structural similarity and the text similarity of the first financial entity and the second financial entity are weightedly added to obtain a comprehensive similarity.

[0026] In a possible implementation, determining the alignment data of the financial entity library of the first data source and the financial entity library of the second data source based on the comprehensive similarity between the first financial entity and the second financial entity includes:

[0027] If the comprehensive similarity between the first financial entity and the second financial entity is greater than a preset alignment threshold, taking the first financial entity and the second financial entity as a candidate entity pair;

[0028] If multiple candidate entity pairs include the same financial entity, the candidate entity pair with the highest comprehensive similarity is retained as the entity pair to be aligned.

[0029] A second aspect of the present application provides an entity alignment device based on heterogeneous data sources, comprising:

[0030] An entity library construction unit is used to extract financial entities from heterogeneous data sources to obtain a financial entity library of the heterogeneous data sources, wherein the heterogeneous data sources include a heterogeneous first data source and a second data source;

[0031] The structured similarity calculation unit is used to obtain the structured similarity of a first financial entity and a second financial entity based on a structured attribute set and a corresponding attribute value set of each financial entity in each financial entity library, wherein the first financial entity is any financial entity in the financial entity library of the first data source, and the second financial entity is any financial entity in the financial entity library of the second data source; the structured attribute set includes a plurality of structured attributes, and the attribute value set includes an attribute value of each structured attribute.

[0032] A text similarity calculation unit, configured to obtain the text similarity between the first financial entity and the second financial entity based on a text attribute set of each financial entity in each financial entity library, wherein the text attribute set includes a plurality of text attributes;

[0033] A similarity fusion unit, configured to fuse the structural similarity and text similarity of the first financial entity and the second financial entity, and calculate the comprehensive similarity of the first financial entity and the second financial entity;

[0034] The entity alignment unit is used to determine the alignment data of the financial entity library of the first data source and the financial entity library of the second data source based on the comprehensive similarity between the first financial entity and the second financial entity, and the alignment data is an entity pair to be aligned, and the entity pair to be aligned includes two financial entities whose comprehensive similarity is greater than a preset alignment threshold.

[0035] A third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the entity alignment method based on heterogeneous data sources according to the first aspect or any implementation of the first aspect.

[0036] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0037] The memory is used to store computer programs;

[0038] The processor is used to execute the computer program so that the electronic device can implement the entity alignment method based on heterogeneous data sources according to the above-mentioned first aspect or any implementation manner of the first aspect.

[0039] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the entity alignment method based on heterogeneous data sources according to the above-mentioned first aspect or any implementation of the first aspect.

[0040] By means of the above technical solution, the present application provides an entity alignment method based on heterogeneous data sources and a related device, extracting financial entities from heterogeneous data sources to obtain a financial entity library of the heterogeneous data sources, wherein the heterogeneous data sources include a heterogeneous first data source and a second data source. Based on the structured attribute set and the corresponding attribute value set of each financial entity in each financial entity library, the structured similarity between the first financial entity and the second financial entity is obtained, wherein the first financial entity is any financial entity in the financial entity library of the first data source, and the second financial entity is any financial entity in the financial entity library of the second data source. The structured attribute set includes a plurality of structured attributes, and the attribute value set includes the attribute values ​​of each structured attribute. Based on the text attribute set of each financial entity in each financial entity library, the text similarity between the first financial entity and the second financial entity is obtained, wherein the text attribute set includes a plurality of text attributes. The structured similarity and the text similarity of the first financial entity and the second financial entity are integrated to calculate the comprehensive similarity between the first financial entity and the second financial entity. Based on the comprehensive similarity between the first financial entity and the second financial entity, the alignment data of the financial entity library of the first data source and the financial entity library of the second data source are determined, and the entity pairs to be aligned are aligned. The entity pairs to be aligned include two financial entities whose comprehensive similarity is greater than a preset alignment threshold. The present application combines the unstructured data and structured data of the financial entities, calculates the comprehensive similarity of the financial entities in the financial entity libraries of heterogeneous data sources, and determines whether the financial entities in the financial entity libraries of the heterogeneous data sources can be aligned based on the comprehensive similarity, thereby improving the accuracy and reliability of entity alignment. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0042] Figure 1 A schematic diagram of an entity alignment system architecture based on heterogeneous data sources provided in this application;

[0043] Figure 2 A schematic diagram of the hardware structure of a terminal provided in this application;

[0044] Figure 3 A schematic diagram of the hardware structure of a server provided for this application;

[0045] Figure 4 A schematic diagram of a flow chart of an entity alignment method based on heterogeneous data sources provided in an embodiment of the present application;

[0046] Figure 5 A specific implementation process of an entity alignment method based on heterogeneous data sources provided in an embodiment of the present application;

[0047] Figure 6 A specific structural diagram of an entity alignment device based on heterogeneous data sources provided in an embodiment of the present application;

[0048] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0050] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0051] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0052] The present application can be applied in the field of big data technology, and specifically can be applied to a scenario where abnormal data is obtained in an entity alignment scenario based on heterogeneous data sources. An entity alignment method based on heterogeneous data sources provided in an embodiment of the present application can be applied to an entity alignment system based on heterogeneous data sources. The entity alignment method based on heterogeneous data sources is implemented through a terminal or a server in an entity alignment system based on heterogeneous data sources, or the entity alignment method based on heterogeneous data sources is implemented through the interaction between a terminal and a server in an entity alignment system based on heterogeneous data sources.

[0053] See also Figure 1 , Figure 1 A schematic diagram of an entity alignment system architecture based on heterogeneous data sources is shown. The system may include a terminal 100 and a server 200. The server 200 may include one or more servers ( Figure 1 In the example, a server is included, and the server 200 can provide the method provided in the embodiment of the present application for one or more terminals.

[0054] Among them, a knowledge base fusion application can be installed on the terminal 100. The above application and web page can provide an interface. The terminal 100 can receive relevant parameters entered by the user on the knowledge base fusion interface and send the above parameters to the server 200. The server 200 can obtain processing results based on the received parameters and return the processing results to the terminal 100.

[0055] It should be understood that in some optional implementations, the terminal 100 can also complete the action of obtaining the processing result based on the received parameters by itself without the cooperation of the server, and the embodiments of the present application are not limited to this.

[0056] Next describe Figure 1 The product form of the mid-terminal 100;

[0057] The terminal 100 in the embodiment of the present application can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiment of the present application does not impose any limitation on this.

[0058] Figure 2 An optional hardware structure diagram of the terminal 100 is shown.

[0059] refer to Figure 2 As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), an earphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190 and other components. Those skilled in the art will appreciate that Figure 2 These are merely examples of terminals or multi-function devices and do not constitute limitations on the terminals or multi-function devices, which may include more or fewer components than those shown in the figures, or combinations of certain components, or different components.

[0060] The input unit 130 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the portable multifunctional device. Specifically, the input unit 130 may include a touch screen 131 (optional) and / or other input devices 132. The touch screen 131 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 170, and can receive and execute the command sent by the processor 170; the touch signal at least includes the touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0061] Among them, the input device 132 can receive input data and the like.

[0062] The display unit 140 may be used to display information input by the user or provided to the user, various menus of the terminal 100, interactive interfaces, file displays, and / or playback of any multimedia file. In an embodiment of the present application, the display unit 140 may be used to display an interface, processing results, etc. of entity alignment based on heterogeneous data sources.

[0063] The memory 120 can be used to store instructions and data. The memory 120 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc.; the instruction storage area can store software units such as operating systems, applications, instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; provide the processor 170 with hardware, software and data resources including management of computing and processing equipment, and support control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0064] The processor 170 is the control center of the terminal 100. It uses various interfaces and lines to connect various parts of the entire terminal 100. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it executes various functions of the terminal 100 and processes data, thereby controlling the terminal device as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 170. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 170 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0065] Among them, the memory 120 can be used to store software codes related to the entity alignment method based on heterogeneous data sources, the processor 170 can execute the steps of the entity alignment method based on heterogeneous data sources, and can also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to implement corresponding functions.

[0066] The radio frequency unit 110 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 170 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0067] In this embodiment of the present application, the RF unit 110 can send data to the server 200 and receive processing results sent by the server 200.

[0068] It should be understood that the radio frequency unit 110 is optional and can be replaced by other communication interfaces, such as a network port.

[0069] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0070] The terminal 100 further includes an external interface 180 , which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal 100 to communicate with other devices, or to connect a charger to charge the terminal 100 .

[0071] Although not shown, the terminal 100 may also include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which are not described in detail here. Some or all of the methods described below may be applied in the following embodiments. Figure 2 In the terminal 100 shown.

[0072] Next describe Figure 1 The product form of the server 200;

[0073] Figure 3 A structural diagram of a server 200 is provided, such as Figure 3 As shown, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0074] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0075] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0076] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0077] The memory 204 may be used to store software codes related to the entity alignment method based on heterogeneous data sources, and the processor 202 may execute the steps of the entity alignment method based on heterogeneous data sources of the chip, and may also schedule other units to implement corresponding functions.

[0078] It should be understood that the above-mentioned terminal 100 and server 200 can be centralized or distributed devices, and the processors in the above-mentioned terminal 100 and server 200 (such as processor 170 and processor 202) can be hardware circuits (such as application specific integrated circuit (ASIC), field-programmable gate array (FPGA), general-purpose processor, digital signal processor (DSP), microprocessor or microcontroller, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0079] The construction of a financial knowledge base requires the fusion of data from different sources. Knowledge is used to represent entities and the relationships between entities. One of the keys to the fusion process is entity alignment. Financial knowledge bases generally have an ontology layer and a data layer. The financial ontology layer is generally constructed by professionals and has a high degree of completeness. Entity alignment is mainly completed at the data layer. Its goal is to find the same substance in real life in different data sources, clean up and merge duplicate and redundant entities, and deduplicate and complement the attributes between the same entity pairs, so as to solve the problems of entity reuse and attribute missing in the knowledge base, thereby creating a large-scale unified financial knowledge base.

[0080] Existing entity alignment methods include measuring the similarity of entity-related texts through vector calculation to achieve entity alignment. In the field of financial knowledge base, this method lacks semantic information of entities and relationships and related structured attribute information.

[0081] In addition, existing entity alignment methods can also use rule extraction, supervised learning, unsupervised learning and other methods to align and map entities. Supervised learning requires manual semantic annotation, which has high accuracy, but requires a large amount of data and consumes a lot of manpower. Unsupervised learning embeds the triples in the knowledge graph into low-dimensional vectors.

[0082] With the rise of knowledge representation learning, entity alignment algorithms based on representation learning have also developed rapidly and achieved good results. Based on the representation learning method, entities and relationships in the knowledge graph are mapped into dense vectors in a low-dimensional space through representation learning, which fully obtains the deep features of the data and directly uses mathematical expressions to calculate the similarity between entities, thus solving the shortcoming of highly relying on complete text information in the entity alignment process.

[0083] However, methods based on similarity theory and representation learning can achieve good results for structured financial entity alignment. However, for heterogeneous financial data sources, there are still data quality challenges and technical shortcomings in the entity alignment process that cannot comprehensively utilize entity structure information and semantic information.

[0084] In summary, the existing indication fusion methods have the following technical disadvantages:

[0085] 1. For financial entity alignment, the lack of structured attribute information related to the entity can easily lead to poor data fusion effects.

[0086] 2. Using deep learning technology to embed word vectors for entity non-structural information and using Manhattan distance, Euclidean distance, etc. to calculate similarity may cause a semantic gap and poor data fusion effect.

[0087] 3. Only part of the entity information is used, and the entity structured attributes and unstructured information are not fully utilized, resulting in a deviation in the entity alignment effect.

[0088] In order to solve the above problems, the embodiment of the present application provides an entity alignment method based on heterogeneous data sources. The entity alignment method based on heterogeneous data sources in the embodiment of the present application is described in detail below with reference to the accompanying drawings. Figure 4 , Figure 4 The process diagram of an entity alignment method based on heterogeneous data sources provided in an embodiment of the present application is as follows: Figure 4 As shown, an entity alignment method based on heterogeneous data sources provided in an embodiment of the present application may include the following steps:

[0089] S401. Extract financial entities from heterogeneous data sources to obtain a financial entity library of the heterogeneous data sources.

[0090] In this embodiment, the heterogeneous data source includes a first heterogeneous data source and a second heterogeneous data source.

[0091] S402: Based on the structured attribute sets and corresponding attribute value sets of each financial entity in the financial entity library, obtain the structured similarity between the first financial entity and the second financial entity, where the financial entity library includes a plurality of financial entities.

[0092] In this embodiment, the first financial entity is any financial entity in the financial entity library of the first data source, and the second financial entity is any financial entity in the financial entity library of the second data source.

[0093] In this embodiment, the structured attribute set includes multiple structured attributes, and the attribute value set includes the attribute value of each structured attribute. The structured attributes include structured data items such as area, quantity, and transaction amount.

[0094] S403: Based on the text attribute set of each financial entity in the financial entity database, obtain the text similarity between the first financial entity and the second financial entity.

[0095] In this embodiment, the text attribute set of the financial entity includes multiple text attributes of the financial entity. The text attributes are obtained by parsing the text information of the financial entity, and the parsing method includes standardization and word segmentation. Among them, the text information includes text such as name, abstract, and body.

[0096] S404: Integrate the structural similarity and text similarity of the first financial entity and the second financial entity to calculate the comprehensive similarity between the first financial entity and the second financial entity.

[0097] In this embodiment, the comprehensive similarity is positively correlated with the structural similarity and the text similarity, respectively, and the method of fusing the structural similarity and the text similarity of two financial entities includes weighted addition.

[0098] S405: Determine alignment data of the financial entity library of the first data source and the financial entity library of the second data source based on the comprehensive similarity between the first financial entity and the second financial entity.

[0099] In this embodiment, the alignment data includes a pair of entities to be aligned. The pair of entities to be aligned includes two financial entities whose comprehensive similarity is greater than a preset alignment threshold.

[0100] It can be seen from the above technical solutions that the embodiment of the present application provides an entity alignment method based on heterogeneous data sources. Combining the unstructured data and structured data of financial entities, the financial entities in the financial entity library of the heterogeneous data source are comprehensively similarly calculated, and based on the comprehensive similarity, it is determined whether the financial entities in the financial entity library of the heterogeneous data source can be aligned, thereby improving the accuracy and reliability of entity alignment.

[0101] Furthermore, the embodiment of the present application provides a specific implementation method of an entity alignment method based on heterogeneous data sources, and provides an entity alignment algorithm based on weighted information fusion based on financial entities and their structured information and unstructured information to perform financial entity alignment. Figure 5 A specific implementation flow chart of an entity alignment method based on heterogeneous data sources provided in an embodiment of the present application is as follows: Figure 5 As shown, this method specifically includes:

[0102] S501: Acquire original data from multiple data sources, perform data preprocessing on the original data, and obtain an initial knowledge base of each data source.

[0103] In this embodiment, the multiple data sources include multiple heterogeneous data sources. For example, the data sources include various types of encyclopedia data on encyclopedia web pages. The initial knowledge base corresponding to any data source, i.e., the target data source, includes unstructured attributes and structured text of financial entities extracted from the original data of the target data source.

[0104] In this embodiment, data preprocessing of the original data of the target data source specifically includes:

[0105] Through ontology analysis, the representation and storage methods of financial information are specified.

[0106] Due to the professionalism of the field, it is difficult to achieve ideal results using ordinary word segmentation tools. First, special terms in the financial field are collected and put into the knowledge base of the word segmentation tool, which can better improve the accuracy of word segmentation and thus improve the accuracy of the entire solution.

[0107] Financial entity information is extracted from different data sources. The financial entity information includes structured information and unstructured information. The structured information includes unstructured text. For example, the financial entity information includes the name of the financial entity, the summary of the financial entity, the main text of the financial entity, etc. The structured information includes structured attributes and attribute values.

[0108] It should be noted that different data sources have inconsistent descriptions of the same financial entity. In order to improve the efficiency of entity fusion, a preliminary review of the entity information is conducted. Specifically, the entity information is reviewed in combination with crowdsourcing to improve the data quality of entity fusion and build a unified resource description framework.

[0109] This step preprocesses the data from various financial data sources, including grammar normalization and data normalization, to improve data quality and build a unified expression standard.

[0110] S502: Acquire a structured attribute set and a corresponding attribute value set of the first financial entity and the second financial entity.

[0111] In this embodiment, the first financial entity is any financial entity in the financial entity library of the first data source, the second financial entity is any financial entity in the financial entity library of the second data source, the structured attribute set includes multiple structured attributes, and the attribute value set includes attribute values ​​of each structured attribute.

[0112] S503: For the first financial entity and the second financial entity, obtain common structured attribute data of the structured attribute set of the first financial entity and the second financial entity.

[0113] In this embodiment, the public structured attribute data includes a public structured attribute set and an attribute value group set. The public structured attribute set includes at least one public structured attribute, the public structured attribute is a structured attribute included in the structured attribute sets of the first financial entity and the second financial entity, the attribute value group set includes an attribute value group corresponding to each public structured attribute in the public structured attribute set, and the attribute value group includes the attribute values ​​of the first financial entity and the second financial entity.

[0114] In this embodiment, a union of the structured attribute sets of the first financial entity and the second financial entity is obtained as a common structured attribute set, and an attribute value group is obtained from the attribute value set.

[0115] Assume that entity E a is the financial entity of data source A, E a The structured attribute set P a Including k a structured attributes, and the corresponding attribute value set is V a , entity E b is the financial entity of data source B, E b The structured attribute set P b Including k b structured attributes, and the corresponding attribute value set is V b .

[0116] E a With E b The public structured attribute set of V a and V b The union CP of t public structured attributes, CP includes t public structured attributes, where the i-th (i∈[1, t]) public structured attribute is recorded as CPi, and the attribute value group set corresponding to CP includes the attribute value groups of t public structured attributes [V ai , V bi ], where V ai is the E of the ith public structured attribute CPi in the public structured attribute set a The attribute value of V bi is the E of the ith public structured attribute CPi in the public structured attribute set b The attribute value of .

[0117] S504: For each attribute value group corresponding to a common structured attribute, calculate the string edit distance between two attribute values ​​in the attribute value group.

[0118] In this embodiment, the attribute value is a string, and the string edit distance is the minimum number of operations performed when converting two strings. Taking the first string and the second string as an example, the string edit distance between the first string and the second string is the minimum number of operations performed when converting the first string to the second string, where the operations include insertion, deletion, modification, etc. The operations increase the similarity between the two strings. If the two strings are exactly the same, the string edit distance is 0.

[0119] In this embodiment, edit[m][n] represents the string edit distance between a first string L1 having a length of m and a second string L2 having a length of n. The method for calculating edit[m][n] includes:

[0120] If the last character (end character) of L1 and L2 is the same, edit[m][n]=edit[m-1][n-1].

[0121] If the last characters of L1 and L2 are different, operate on L1 or L2 and count, that is, the operand is increased by 1 each time an operation is performed.

[0122] For example, after replacing the last characters, the two strings are exactly the same: edit[m][n] = edit[m-1][n-1]+1;

[0123] After deleting the last character of L1, the two strings are exactly the same: edit[m][n] = edit[m-1][n] + 1;

[0124] After deleting the last character of L2, the two strings are exactly the same: edit[m][n] = edit[m][n-1] + 1;

[0125] After adding the last character of L2 to the end of L1, the two strings are exactly the same: edit[m][n] = edit[m][n-1] + 1;

[0126] After adding the last character of L1 to the end of L2, the two strings are exactly the same: edit[m][n] = edit[m-1][n] + 1;

[0127] When the L1 string is empty, edit[0][n] = n;

[0128] When the L2 string is empty, edit[m][0] = m.

[0129] In summary, the calculation formula of edit[m][n] is as follows:

[0130]

[0131] In formula 1, the flag function is used to determine whether two strings are equal, as shown in formula (2):

[0132]

[0133] S505: Calculate the attribute similarity of the common structural attributes based on the character string lengths and character string edit distances of the two attribute values ​​in the attribute value group.

[0134] In this embodiment, using the idea of ​​dynamic programming, the attribute similarity of the common structural attribute is equal to the ratio of the string edit distance of two attribute values ​​corresponding to the common structural attribute to the maximum length of the attribute value.

[0135] Continuing with the above example, the calculation formula for the attribute similarity SimCP(CPi) of the common structured attribute CPi is:

[0136]

[0137] Among them, len(V ai ) means E a The attribute value V of the public structured attribute CPi ai The string length, len(V bi ) means E b The attribute value V of the public structured attribute CPi bi The string length, edit(V ai , V bi ) indicates V ai and V bi The string edit distance of .

[0138] S506: Calculate the structural similarity between the first financial entity and the second financial entity based on the attribute similarity of each common structural attribute of the first financial entity and the second financial entity.

[0139] In this embodiment, the attribute similarities of all common structural attributes of the first financial entity and the second financial entity are normalized to obtain a normalized mean of the attribute similarities as the structural similarity between the first financial entity and the second financial entity.

[0140] Continuing from the previous example, E a and E b The structural similarity SimZ(E a , E b ) is calculated as:

[0141]

[0142] S507: Based on the texts of each financial entity, obtain a text attribute set of the financial entity.

[0143] In this embodiment, the improved word segmentation tool and domain dictionary are used to perform word segmentation processing on the text of the financial entity, and the word segmentation is used as a text attribute to obtain a text attribute set of the financial entity. The text attribute set includes multiple text attributes, wherein the text attribute in the text attribute set is the word segmentation.

[0144] It should be noted that for text, it is necessary to perform word segmentation on the text. This step can be performed using the Jieba word segmentation tool. The existing word segmentation tools have cross-cutting effects on domain knowledge. Therefore, in the data preprocessing stage, this application organizes a large number of financial entity-specific terms and constructs a financial domain dictionary. This step directly performs word segmentation on the text based on the pre-constructed financial domain dictionary to obtain the word segmentation result and improve the accuracy of word segmentation.

[0145] Still using entity E from data source A a and entity E from data source B b For example, the improved word segmentation tool and domain dictionary are used to improve E a Perform word segmentation to obtain a The text attribute set E of the word segmentation a , and including E b The text attribute set of the word segmentation is F b , where the text attribute set F a Any text attribute in is denoted as F am , text attribute set F b Any text attribute in is denoted as F bn , where m∈[1,M], M is F a The number of text attributes in F, n∈[1,N], where N is b The number of text attributes in .

[0146] S508: Input the text attribute sets of the first financial entity and the second financial entity into a text feature extraction model to obtain feature vector sets of the first financial entity and the second financial entity output by the text feature extraction model.

[0147] In this embodiment, the feature vector set includes the feature vector of each text attribute in the text attribute set.

[0148] In this embodiment, the text feature extraction model is a Doc2vec model. The Doc2Vec model is an unsupervised algorithm based on converting a word sequence into a vector representation of a fixed length, extracting the feature vector of each text attribute in an input text attribute set, and obtaining a feature vector set corresponding to the input text attribute set.

[0149] Continuing from the previous example, the text attribute set F a The corresponding eigenvector set is Va , V a Medium F am The eigenvector of am , F b The corresponding eigenvector set is V b , F b Medium F bn The eigenvector of bn .

[0150] It should be noted that the Doc2Vec model extracts feature vectors based on an unsupervised algorithm Doc2Vec, which can represent a fixed-length sentence, paragraph, etc. as a fixed-length vector. This vector is used as its feature vector. Compared with the Word2vec model, the Doc2vec model combines sentence vector information to better express the semantic information of the text.

[0151] The Doc2Vec model is trained using the sentence vector distributed memory model (PV-DW). As a model training method of Doc2vec, PV-DW slides and samples words of a fixed length from a sentence each time, takes one of the words as the predicted word, and uses the others as input words. It adds a paragraph vector, which makes up for the shortcomings of the bag-of-words model to a certain extent.

[0152] In this example, we first set the basic parameters of the model. The vector dimension is generally 200 dimensions, and the sliding window is 5 to better capture information.

[0153] S509: Calculate the text similarity between the first feature vector and the second feature vector.

[0154] In this embodiment, the first feature vector is any feature vector in a feature vector set of the first financial entity, the second feature vector is any feature vector in a feature vector set of the second financial entity, and the text similarity includes cosine similarity.

[0155] Continuing from the previous example, the first eigenvector is V ai , the second eigenvector is V bi , the module lengths are |V am | and |V bn |, calculate the eigenvector V ai and V bi The cosine similarity Sim(F ai , F bi ) is as shown in formula (5):

[0156]

[0157] S510: If the text similarity between the first feature vector and the second feature vector is greater than a preset similarity threshold, the text attribute corresponding to the first feature vector and the text attribute corresponding to the second feature vector are taken as a similar text attribute pair.

[0158] In this embodiment, the similarity threshold is set to 0.8, and z similar text attribute pairs greater than the similarity threshold are obtained through continuous iterative comparison.

[0159] S511 . Taking the average value of the text similarities of each similar text attribute pair of the first financial entity and the second financial entity as the text similarity of the first financial entity and the second financial entity.

[0160] In this embodiment, the average value includes the arithmetic mean, then, E is calculated a and E b The text similarity method is as follows:

[0161]

[0162] S512: The structural similarity and text similarity of the first financial entity and the second financial entity are integrated to calculate the comprehensive similarity between the first financial entity and the second financial entity.

[0163] In this embodiment, based on a preset weight coefficient, the structural similarity and the text similarity are weighted and added to obtain the comprehensive similarity.

[0164] Continuing with the previous example, calculate E a and E b The comprehensive similarity SimE(E a , E b ) is as shown in formula (7):

[0165] Sim E(E a , E b )=w1·Sim Z(E a , E b )+w2·Sim F(E a , E b ) (7).

[0166] Among them, w1 and w2 represent the weight coefficients of structural similarity and text similarity respectively.

[0167] In this embodiment, the weight coefficients of the structural similarity and the text similarity are flexibly configured based on the type of the financial entity to improve the accuracy of the comprehensive similarity measurement of the financial entity.

[0168] S513: The first financial entity and the second financial entity whose comprehensive similarity is greater than a preset alignment threshold are taken as candidate entity pairs.

[0169] In this embodiment, the candidate entity pair includes two alignable financial entities, and the comprehensive similarity of the alignable financial entities is greater than the alignment threshold.

[0170] S514: If the comprehensive similarity between a first financial entity in the first data source and multiple second financial entities in the first data source is greater than an alignment threshold, determine the first financial entity and a second financial entity having the greatest comprehensive similarity to the first financial entity as an entity pair to be aligned.

[0171] It can be seen from the above technical scheme that the entity alignment method based on heterogeneous data sources provided by the embodiment of the present application obtains the structured information of the financial entity, and then uses the dynamic programming algorithm to calculate the attribute similarity based on the string edit distance, thereby mining the structured similarity between the financial entities. After obtaining the unstructured attribute information of the financial entity, that is, the text information, the text is represented by a feature vector using a deep learning algorithm model, and the vector similarity is calculated, thereby mining the text similarity of the deep semantic information of the text of the first financial entity and the second financial entity. Further, the text similarity and the structural similarity are integrated to obtain the comprehensive similarity, and the comprehensive similarity is used as the similarity metric to determine the financial entities that can be aligned, and the structured information and unstructured information related to the financial entity are used, combined with the weight coefficient, and the information of each part is comprehensively utilized to improve the accuracy of entity alignment.

[0172] Specifically, S501 provides an optional method for extracting financial entities from heterogeneous data sources to obtain a financial entity library of heterogeneous data sources, performs ontology analysis on the characteristic set of financial entity information, formulates a unified resource description format, and constructs a professional term dictionary, i.e., a financial field dictionary, to improve the accuracy of word segmentation.

[0173] S502-505 provides an optional specific method for obtaining the structural similarity between the first financial entity and the second financial entity based on the structured attribute set and the corresponding attribute value set of each financial entity in the financial entity library. The similarity is calculated using the string edit distance for the structured attributes of the financial entities to obtain the structural similarity, which represents the similarity between different financial entities in the structured data.

[0174] S506-511 provides an optional specific method for obtaining the text similarity between the first financial entity and the second financial entity based on the text attribute set of each financial entity in the financial entity library. The method uses the Doc2vec model built based on deep learning technology to mine the feature vectors of the financial entity text and uses cosine similarity to calculate the vector similarity to obtain the text similarity, which represents the similarity of different financial entities on unstructured data.

[0175] S512 provides an optional specific method for fusing the structural similarity and text similarity of the first financial entity and the second financial entity to calculate the comprehensive similarity of the first financial entity and the second financial entity. The method uses a weighted information fusion method to fuse the structural similarity and text similarity, and the obtained comprehensive similarity improves the similarity accuracy.

[0176] S513-514 provides an optional specific method for determining the alignment data of the financial entity library of the first data source and the financial entity library of the second data source based on the comprehensive similarity between the first financial entity and the second financial entity. The pre-configured threshold and comprehensive similarity are used to determine and screen the alignable entities, thereby improving the reliability of entity alignment.

[0177] It should be noted that the present application does not limit the execution order of obtaining the structural similarity of the first financial entity and the second financial entity based on the structural attribute set and the corresponding attribute value set of each financial entity in the financial entity library, and obtaining the text similarity of the first financial entity and the second financial entity based on the text attribute set of each financial entity in the financial entity library. In an optional embodiment, the similarity stage tasks of structural similarity and text similarity can be executed in parallel to improve the efficiency of entity alignment.

[0178] The above describes an entity alignment method based on heterogeneous data sources provided in an embodiment of the present application. The following describes a device for executing the above entity alignment method based on heterogeneous data sources.

[0179] See also Figure 7 , Figure 7 The present invention provides a schematic diagram of a structure of an entity alignment device based on heterogeneous data sources. Figure 7 As shown, the entity alignment device 700 based on heterogeneous data sources includes:

[0180] 7. An entity alignment device based on heterogeneous data sources, characterized by comprising:

[0181] An entity library construction unit 601 is used to extract financial entities from heterogeneous data sources to obtain a financial entity library of the heterogeneous data sources, wherein the heterogeneous data sources include a heterogeneous first data source and a second data source;

[0182] The structured similarity calculation unit 602 is used to obtain the structured similarity of the first financial entity and the second financial entity based on the structured attribute set and the corresponding attribute value set of each financial entity in each financial entity library, wherein the first financial entity is any financial entity in the financial entity library of the first data source, and the second financial entity is any financial entity in the financial entity library of the second data source; the structured attribute set includes a plurality of structured attributes, and the attribute value set includes the attribute value of each structured attribute.

[0183] A text similarity calculation unit 603, configured to obtain the text similarity between the first financial entity and the second financial entity based on a text attribute set of each financial entity in each financial entity library, wherein the text attribute set includes a plurality of text attributes;

[0184] A similarity fusion unit 604, configured to fuse the structural similarity and text similarity of the first financial entity and the second financial entity, and calculate the comprehensive similarity of the first financial entity and the second financial entity;

[0185] The entity alignment unit 605 is used to determine the alignment data of the financial entity library of the first data source and the financial entity library of the second data source based on the comprehensive similarity between the first financial entity and the second financial entity, wherein the alignment data is an entity pair to be aligned, and the entity pair to be aligned includes two financial entities whose comprehensive similarity is greater than a preset alignment threshold.

[0186] In a possible implementation, the entity library construction unit is used to extract financial entities from heterogeneous data sources to obtain the financial entity library of the heterogeneous data sources, and is specifically used to:

[0187] Acquiring initial data from the heterogeneous data source and performing data preprocessing to obtain data source data, wherein the data preprocessing includes grammatical normalization and data normalization;

[0188] Financial entities are identified from the data source data of the heterogeneous data sources, and financial entity information is extracted, wherein the financial entity information includes structured information and unstructured information, wherein the structured information includes structured attributes and attribute values, and the unstructured information includes text.

[0189] In a possible implementation, the structured similarity calculation unit is used to obtain the structured similarity between the first financial entity and the second financial entity based on the structured attribute set and the corresponding attribute value set of each financial entity in each financial entity library, specifically to:

[0190] Acquire common structured attribute data of the structured attribute sets of the first financial entity and the second financial entity, wherein the common structured attribute data includes a common structured attribute set and an attribute value group set, and the common structured attribute set includes at least one common structured attribute;

[0191] For each attribute value group corresponding to the common structured attribute, calculating the string edit distance between two attribute values ​​in the attribute value group;

[0192] Calculating the attribute similarity of each of the common structural attributes based on the string length and the string edit distance;

[0193] Based on the attribute similarity of each of the common structural attributes of the first financial entity and the second financial entity, the structural similarity of the first financial entity and the second financial entity is calculated.

[0194] In a possible implementation, the text similarity calculation unit is used to obtain the text similarity between the first financial entity and the second financial entity based on the text attribute set of each financial entity in each financial entity library, specifically to:

[0195] Performing word segmentation on the text of the first financial entity and the text of the second financial entity to obtain a text attribute set of the first financial entity and the second financial entity;

[0196] Inputting the text attribute sets of the first financial entity and the second financial entity into a text feature extraction model respectively, obtaining a feature vector set of the first financial entity and the second financial entity output by the text feature extraction model, wherein the feature vector set includes a plurality of feature vectors;

[0197] Calculating text similarity between a first feature vector of the first financial entity and a feature vector of the second financial entity;

[0198] If the text similarity between the first feature vector and the second feature vector is greater than a preset similarity threshold, the text attribute corresponding to the first feature vector and the text attribute corresponding to the second feature vector are taken as a similar text attribute pair;

[0199] The average value of the text similarities of the similar text attribute pairs of the first financial entity and the second financial entity is used as the text similarity between the first financial entity and the second financial entity.

[0200] In a possible implementation, the similarity fusion unit is used to fuse the structural similarity and text similarity of the first financial entity and the second financial entity, and when calculating the comprehensive similarity of the first financial entity and the second financial entity, specifically to:

[0201] Based on a preset weight coefficient, the structural similarity and the text similarity of the first financial entity and the second financial entity are weightedly added to obtain a comprehensive similarity.

[0202] In a possible implementation, when the entity alignment unit is used to determine the alignment data of the financial entity library of the first data source and the financial entity library of the second data source based on the comprehensive similarity between the first financial entity and the second financial entity, it is specifically used to:

[0203] If the comprehensive similarity between the first financial entity and the second financial entity is greater than a preset alignment threshold, taking the first financial entity and the second financial entity as a candidate entity pair;

[0204] If multiple candidate entity pairs include the same financial entity, the candidate entity pair with the highest comprehensive similarity is retained as the entity pair to be aligned.

[0205] The present application also provides an electronic device in an embodiment. Figure 7 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic device in the embodiments of the present application may include but is not limited to fixed terminals such as mobile phones, notebook computers, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 7 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0206] like Figure 6 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0207] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a memory card, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0208] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any entity alignment method based on heterogeneous data sources provided in the embodiment of the present application.

[0209] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any entity alignment method based on heterogeneous data sources provided in an embodiment of the present application.

[0210] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0211] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0212] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0213] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. An entity alignment method based on heterogeneous data sources, characterized in that: include: Extracting financial entities from heterogeneous data sources to obtain a financial entity library of the heterogeneous data sources, wherein the heterogeneous data sources include a first heterogeneous data source and a second heterogeneous data source; Based on the structured attribute set and the corresponding attribute value set of each financial entity in each financial entity library, obtaining the structured similarity between the first financial entity and the second financial entity, wherein the first financial entity is any financial entity in the financial entity library of the first data source, and the second financial entity is any financial entity in the financial entity library of the second data source; the structured attribute set includes a plurality of structured attributes, and the attribute value set includes the attribute value of each structured attribute; Based on a text attribute set of each financial entity in each financial entity library, obtaining a text similarity between the first financial entity and the second financial entity, wherein the text attribute set includes a plurality of text attributes; integrating the structural similarity and the textual similarity of the first financial entity and the second financial entity to calculate the comprehensive similarity of the first financial entity and the second financial entity; Based on the comprehensive similarity between the first financial entity and the second financial entity, alignment data of the financial entity library of the first data source and the financial entity library of the second data source are determined, and the alignment data is an entity pair to be aligned, and the entity pair to be aligned includes two financial entities whose comprehensive similarity is greater than a preset alignment threshold.

2. The entity alignment method based on heterogeneous data sources according to claim 1 is characterized in that: Extract financial entities from heterogeneous data sources to obtain a financial entity library of heterogeneous data sources, including: Acquiring initial data from the heterogeneous data source and performing data preprocessing to obtain data source data, wherein the data preprocessing includes grammatical normalization and data normalization; Financial entities are identified from the data source data of the heterogeneous data sources, and financial entity information is extracted, wherein the financial entity information includes structured information and unstructured information, wherein the structured information includes structured attributes and attribute values, and the unstructured information includes text.

3. The entity alignment method based on heterogeneous data sources according to claim 1 is characterized in that: Based on the structured attribute sets and corresponding attribute value sets of each financial entity in each financial entity library, the structured similarity between the first financial entity and the second financial entity is obtained, including: Acquire common structured attribute data of the structured attribute sets of the first financial entity and the second financial entity, wherein the common structured attribute data includes a common structured attribute set and an attribute value group set, and the common structured attribute set includes at least one common structured attribute; For each attribute value group corresponding to the common structured attribute, calculating the string edit distance between two attribute values ​​in the attribute value group; Calculating the attribute similarity of each of the common structural attributes based on the string length and the string edit distance; Based on the attribute similarity of each of the common structural attributes of the first financial entity and the second financial entity, the structural similarity of the first financial entity and the second financial entity is calculated.

4. The entity alignment method based on heterogeneous data sources according to claim 1 is characterized in that: Based on the text attribute sets of each financial entity in each financial entity library, obtaining the text similarity between the first financial entity and the second financial entity includes: Performing word segmentation on the text of the first financial entity and the text of the second financial entity to obtain a text attribute set of the first financial entity and the second financial entity; Inputting the text attribute sets of the first financial entity and the second financial entity into a text feature extraction model respectively, obtaining a feature vector set of the first financial entity and the second financial entity output by the text feature extraction model, wherein the feature vector set includes a plurality of feature vectors; Calculating text similarity between a first feature vector of the first financial entity and a feature vector of the second financial entity; If the text similarity between the first feature vector and the second feature vector is greater than a preset similarity threshold, the text attribute corresponding to the first feature vector and the text attribute corresponding to the second feature vector are taken as a similar text attribute pair; The average value of the text similarities of the similar text attribute pairs of the first financial entity and the second financial entity is used as the text similarity between the first financial entity and the second financial entity.

5. The entity alignment method based on heterogeneous data sources according to claim 1 is characterized in that: Combining the structural similarity and text similarity of the first financial entity and the second financial entity to calculate the comprehensive similarity of the first financial entity and the second financial entity includes: Based on a preset weight coefficient, the structural similarity and the text similarity of the first financial entity and the second financial entity are weightedly added to obtain a comprehensive similarity.

6. The entity alignment method based on heterogeneous data sources according to claim 1 is characterized in that: Determining alignment data of a financial entity library of the first data source and a financial entity library of the second data source based on the comprehensive similarity between the first financial entity and the second financial entity includes: If the comprehensive similarity between the first financial entity and the second financial entity is greater than a preset alignment threshold, taking the first financial entity and the second financial entity as a candidate entity pair; If multiple candidate entity pairs include the same financial entity, the candidate entity pair with the highest comprehensive similarity is retained as the entity pair to be aligned.

7. An entity alignment device based on heterogeneous data sources, characterized in that: include: An entity library construction unit is used to extract financial entities from heterogeneous data sources to obtain a financial entity library of the heterogeneous data sources, wherein the heterogeneous data sources include a heterogeneous first data source and a second data source; a structured similarity calculation unit, configured to obtain a structured similarity between a first financial entity and a second financial entity based on a structured attribute set and a corresponding attribute value set of each financial entity in each financial entity library, wherein the first financial entity is any financial entity in the financial entity library of the first data source, and the second financial entity is any financial entity in the financial entity library of the second data source; the structured attribute set includes a plurality of structured attributes, and the attribute value set includes an attribute value of each structured attribute; A text similarity calculation unit, configured to obtain the text similarity between the first financial entity and the second financial entity based on a text attribute set of each financial entity in each financial entity library, wherein the text attribute set includes a plurality of text attributes; A similarity fusion unit, configured to fuse the structural similarity and text similarity of the first financial entity and the second financial entity, and calculate the comprehensive similarity of the first financial entity and the second financial entity; The entity alignment unit is used to determine the alignment data of the financial entity library of the first data source and the financial entity library of the second data source based on the comprehensive similarity between the first financial entity and the second financial entity, and the alignment data is an entity pair to be aligned, and the entity pair to be aligned includes two financial entities whose comprehensive similarity is greater than a preset alignment threshold.

8. A computer program product, characterized in that It includes computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the entity alignment method based on heterogeneous data sources as described in any one of claims 1 to 6.

9. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the entity alignment method based on heterogeneous data sources as described in any one of claims 1 to 6.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the entity alignment method based on heterogeneous data sources as described in any one of claims 1 to 6.

Images