Method for encoding multi-modal business entities, entity matching method and device

Abstract

The embodiments of the present specification provide an encoding method, a matching method and corresponding devices for multi-modal business entities. The encoding method comprises: obtaining a plurality of sub-encoding vectors corresponding to multi-modal features of a target business entity; generating a target encoding vector according to the plurality of sub-encoding vectors, the target encoding vector comprising an index field and a data field, the index field comprising metadata of each of the plurality of sub-encoding vectors; and the data field comprising the plurality of sub-encoding vectors. When matching two business entities, the respective encoding vectors corresponding to the two business entities can be obtained, which have the format of the above target encoding vector. Then, according to the metadata in the index field, the respective plurality of sub-encoding vectors are extracted, and the distance between the business entities is determined by using each sub-encoding vector according to the metadata.

Description

Technical Field

[0001] This specification relates to knowledge graph platforms through one or more embodiments, and more particularly to a method for encoding multimodal business entities, an entity matching method, and an apparatus. Background Technology

[0002] In many scenarios, business entities to be analyzed, such as users, merchants, and operational events, possess multimodal feature information. For example, in a multimodal knowledge graph, the information of a business entity not only includes text but can also include feature information from various modalities such as voice, images, video, geographic information, and behavioral information. This feature information sometimes also includes privacy-related information. For instance, when the business entity is a user, the entity information may include textual feature information such as user ID and user profile, as well as image-based facial feature information, frequently used device information, and recent operational behavior sequences. When the business entity is a merchant, the entity information may include textual information such as merchant ID and merchant category description, as well as image-based scanned copies of the merchant's business license, video-based information about the merchant's surrounding environment, and geographic coordinates of the merchant's location, and so on.

[0003] In knowledge graph platforms, it is often necessary to reason about relationships between entities and perform data analysis, especially entity linking operations, which analyze whether any two entities in the graph have the same characteristics. Therefore, for multimodal business entities, there is a need to integrate information from their various modalities for more effective entity data analysis, such as performing linking operations more efficiently. Summary of the Invention

[0004] In view of the above problems, this specification provides an encoding method and a matching method for multimodal business entities. By uniformly encoding multimodal feature information into a comprehensive encoding vector, it is easier to calculate the distance for entity matching, thereby improving the efficiency of entity data analysis.

[0005] According to the first aspect, an encoding method for multimodal business entities is provided, including:

[0006] Obtain multiple sub-encoding vectors corresponding to the multimodal features of the target business entity;

[0007] A target encoding vector is generated based on the plurality of sub-encoding vectors. The target encoding vector includes an index field and a data field. The index field includes metadata of each of the plurality of sub-encoding vectors. The data field includes the plurality of sub-encoding vectors.

[0008] In one embodiment, obtaining multiple sub-encoding vectors corresponding to the multimodal features of a target business entity specifically includes: obtaining the multimodal features of the target business entity; and encoding the multimodal features into the multiple sub-encoding vectors according to the encoding method of each modality.

[0009] In different examples, multimodal features may include multiple of the following: text features, speech features, image features, geolocation features, device features, and behavioral features; correspondingly, multiple sub-encoding vectors may include multiple of the following: text embedding vector, voiceprint feature vector, graph feature encoding vector, LBS encoding vector, device type encoding vector, and behavioral sequence vector.

[0010] According to one embodiment, the aforementioned metadata may include the vector type and vector length of the corresponding sub-encoded vector.

[0011] In one implementation, the target encoding vector is used to determine the distance between business entities, and then to match the business entities; the metadata includes the weight of the corresponding sub-encoding vector when calculating the distance.

[0012] According to one example, the target encoding vector also includes a header field indicating the number of the plurality of sub-encoding vectors.

[0013] In one implementation, the target business entity is a business entity in a knowledge graph; the target encoding vector is used for entity chaining.

[0014] Furthermore, in one example, the target encoding vector can also be associated with the target business entity and stored in the database corresponding to the knowledge graph.

[0015] According to the second aspect, an entity matching method for multimodal business entities is provided, including:

[0016] Obtain the first and second encoding vectors corresponding to the first business entity and the second business entity, respectively. Each encoding vector includes an index field and a data field. The index field includes metadata of each of the multiple sub-encoding vectors corresponding to the multimodal features of the corresponding business entity. The data field includes the multiple sub-encoding vectors.

[0017] Based on the metadata in the index field, extract multiple first sub-encoding vectors and multiple second sub-encoding vectors from the first and second encoding vectors, respectively;

[0018] Based on the metadata, the entity distance between the first business entity and the second business entity is determined using the plurality of first sub-encoding vectors and the plurality of second sub-encoding vectors, wherein the entity distance corresponds to the degree of matching between the entities.

[0019] In one embodiment, the metadata includes the vector length of each sub-encoded vector; thus, multiple first sub-encoded vectors can be extracted from the data field of the first encoded vector based on the vector length of each sub-encoded vector, and multiple second sub-encoded vectors can be extracted from the data field of the second encoded vector.

[0020] In one implementation, determining the entity distance between the first business entity and the second business entity specifically includes: determining multiple sub-distances corresponding to multiple sub-vector groups formed by corresponding sub-coding vectors in the plurality of first sub-coding vectors and the plurality of second sub-coding vectors; and combining the multiple sub-distances to obtain the entity distance.

[0021] In one example of the above implementation, the metadata includes the vector type of each sub-encoded vector; thus, multiple sub-distances can be determined as follows: based on the target vector type of the sub-encoded vectors contained in any target sub-vector group, the corresponding target distance calculation method is determined; based on the target distance calculation method, the distance between the first sub-encoded vector and the second sub-encoded vector included in the target sub-vector group is calculated as the sub-distance corresponding to the target sub-vector group.

[0022] Furthermore, when the target vector type is a text embedding vector, voiceprint feature vector, or graph feature encoding vector, the target distance is calculated by calculating the cosine distance; when the target vector type is an LBS encoding vector, the target distance is calculated by calculating the spherical space distance; when the target vector type is a device type encoding vector, the target distance is calculated by calculating the Jaccard similarity coefficient.

[0023] In one example of the above implementation, the metadata includes the weights of the corresponding sub-encoding vectors when calculating the entity distance; thus, the entity distance can be obtained as follows: the entity distance is obtained by weighted summing of the multiple sub-distances according to the weights of the sub-encoding vectors in each sub-vector group.

[0024] In one embodiment, each encoding vector further includes a header field indicating the number of the plurality of sub-encoding vectors; the method further includes: parsing a plurality of first metadata from the first encoding vector and parsing a plurality of second metadata from the second encoding vector based on the header field.

[0025] According to one implementation, the first business entity and the second business entity are business entities in a knowledge graph; the method further includes: determining whether the first business entity and the second business entity are matching entities for the target characteristics based on the entity distance.

[0026] According to a third aspect, an encoding apparatus for multimodal business entities is provided, comprising:

[0027] The acquisition unit is configured to acquire multiple sub-encoding vectors corresponding to the multimodal features of the target business entity;

[0028] The encoding unit is configured to generate a target encoding vector based on the plurality of sub-encoding vectors. The target encoding vector includes an index field and a data field. The index field includes metadata of each of the plurality of sub-encoding vectors. The data field includes the plurality of sub-encoding vectors.

[0029] According to the fourth aspect, an entity matching device for multimodal business entities is provided, comprising:

[0030] The encoding acquisition unit is configured to acquire the first and second encoding vectors corresponding to the first business entity and the second business entity, respectively. Each encoding vector includes an index field and a data field. The index field includes metadata of each of the multiple sub-encoding vectors corresponding to the multimodal features of the corresponding business entity. The data field includes the multiple sub-encoding vectors.

[0031] The sub-vector extraction unit is configured to extract multiple first sub-encoded vectors and multiple second sub-encoded vectors from the first and second encoded vectors respectively, based on the metadata in the index field;

[0032] The determining unit is configured to determine the entity distance between the first business entity and the second business entity based on the metadata, using the plurality of first sub-encoding vectors and the plurality of second sub-encoding vectors, wherein the entity distance corresponds to the degree of matching between entities.

[0033] According to a fifth aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method described in the first or second aspect.

[0034] According to a sixth aspect, a computing device is provided, including a memory and a processor, characterized in that the memory stores executable code, and when the processor executes the executable code, it implements the method described in the first aspect or the second aspect.

[0035] In the embodiments of this specification, for multimodal business entities, the multimodal feature information can be uniformly encoded into a comprehensive encoding vector. This encoding vector can cover multimodal data such as voice, image, video, geography, and behavior. Furthermore, during the matching analysis process, based on this unified comprehensive encoding vector, the matching degree between entities can be quickly calculated, improving the analysis efficiency of business entities. Attached Figure Description

[0036] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This diagram illustrates an implementation scenario of an embodiment described in the application specification.

[0038] Figure 2 A flowchart illustrating a method for encoding multimodal business entities according to one embodiment is shown.

[0039] Figure 3 This illustrates a specific example of the multimodal characteristics of a business entity;

[0040] Figure 4 This illustrates the structure of a target encoding vector according to one embodiment;

[0041] Figure 5 This shows a specific example of a target encoding vector;

[0042] Figure 6 A flowchart illustrating an entity matching method for multimodal business entities according to one embodiment is shown;

[0043] Figure 7 A schematic block diagram of an encoding apparatus for encoding multimodal business entities according to one embodiment is shown;

[0044] Figure 8 A schematic block diagram of a matching apparatus for matching multimodal business entities is shown according to one embodiment. Detailed Implementation

[0045] The solution provided in this specification will now be described with reference to the accompanying drawings.

[0046] Figure 1 A schematic diagram illustrating an implementation scenario of an embodiment described in the application specification is shown. Figure 1 The example illustrates the process of performing entity linking operations on business entities within a knowledge graph platform.

[0047] As is known to those skilled in the art, a knowledge graph is a semantic network that describes various entities and their relationships in the real world in a graph pattern. Nodes in this semantic network correspond to entities, and edges correspond to the relationships between entities. By combining expert experience and prior data, the knowledge graph can be used to explain the correctness of the relationships and rules in the graph, as well as infer the relationships and rules that do not appear in the graph. The knowledge graph platform is a middle platform with knowledge graph as its core capability, which provides capabilities of knowledge management, knowledge reasoning and knowledge services for various businesses, as well as graph solutions supporting these capabilities. Entity linking refers to inferring, from the perspective of business applications, whether any two entities in the graph have the same characteristics, for example, whether two users (as business entities) belong to the same family, whether two payment collection codes (as business entities) belong to the same store, whether two accounts (as business entities) belong to the same natural person, and so on.

[0048] In Figure 1 in the example, the knowledge graph platform maintains a knowledge graph that includes users as entities. In response to an external request or according to reasoning needs, the knowledge graph platform can obtain the entity nodes corresponding to user A and user B respectively, and perform an entity linking operation on them, that is, determine whether entity A and entity B match for a certain characteristic. More specifically, in the process of the linking operation, it is generally necessary to embed entity A and entity B into an N-dimensional vector space respectively to obtain their encoded vector representations, and then measure the similarity between the entities by calculating the distance between the vectors. This similarity serves as a representation of the matching degree of the two entities.

[0049] On the other hand, with the enrichment of big data, the feature information of each entity presents the characteristics of multi-modality. For example, a business entity in the knowledge graph can have information of multiple modalities such as text, speech, image, video, geographic information, and behavior. Generally, feature information of different modalities has different encoding methods. For example, for text features, an embedding layer such as a BERT model is often used to obtain a text embedding vector, and for images, a feature extraction network based on convolution operation is often used to obtain an image feature vector, and so on. Therefore, multi-modal business entities pose challenges to the analysis of association relationships between entities, such as entity linking operations.

[0050] In one related technology, for multimodal business entities, their feature information is broken down into multiple parts such as speech, image, video, geography, and behavior. Each part is independently encoded using a corresponding encoding method, resulting in independent encoding vectors. In business entity matching analysis such as entity chain indexing, recall is performed separately for each encoding vector, thus recalling multiple result sets. Then, in the fine-ranking stage, the large number of recalled results in multiple result sets are scored to determine the final matching recall result. In this scheme, encoding and storing each modal feature separately and recalling it independently leads to a large amount of total recalled data, increasing the data volume in the fine-ranking stage and increasing the time consumption of entity chain indexing.

[0051] In view of this, this specification provides an encoding scheme for multimodal entities and a corresponding matching analysis scheme in its embodiments. In this encoding scheme, multimodal feature information can be uniformly encoded into a comprehensive encoding vector, which can cover multimodal data such as speech, image, video, geographic, and behavioral data. Furthermore, during the matching analysis process, based on this unified comprehensive encoding vector, the matching degree between entities can be quickly calculated, improving the analysis efficiency of business entities. The implementation of this improved scheme is described in detail below.

[0052] Figure 2 This diagram illustrates a method flowchart for encoding multimodal business entities according to one embodiment. This method can be executed through a computing platform, which can be any computing unit, server, device, device cluster, etc., with computing and processing capabilities. In a specific example, the computing platform is a knowledge graph platform. Figure 2 As shown, the encoding method may include the steps described in detail below.

[0053] First, in step 21, multiple sub-encoding vectors corresponding to the multimodal features of the target business entity are obtained.

[0054] It is understandable that the target business entity is a business entity with multimodal characteristics, which may include various features such as text features, voice features, image features, geographic location features, device features, and behavioral features.

[0055] In one embodiment, in step 21, the multimodal features of the target business entity are first obtained; then, according to the encoding method of each modality, each modal feature is encoded into a sub-encoding vector, thereby obtaining multiple sub-encoding vectors corresponding to the aforementioned multimodal features. Generally, for a feature of a certain modality, there are already multiple encoding methods for that modality feature in the prior art. Any one of them can be used for the above encoding. For example, for text features, an embedding layer such as a BERT model can be used to encode them, or a word2vec tool can be used to encode them, obtaining a text embedding vector as a sub-encoding vector. For image features, a feature extraction network based on convolutional operations can be used to encode them, obtaining a graph feature vector as a sub-encoding vector. Other modal features can also be encoded using corresponding existing encoding methods, thus encoding the features of each modality into corresponding sub-encoding vectors. Therefore, the multiple sub-encoding vectors obtained can include many of the following: text embedding vector, voiceprint feature vector, graph feature encoding vector, LBS encoding vector, device type encoding vector, behavior sequence vector, etc.

[0056] In another embodiment, if the multimodal features have been encoded into multiple sub-encoding vectors by other devices prior to step 21, then in step 21 the computing platform only needs to read or receive these multiple sub-encoding vectors respectively. For example, the business entity can be a user, and the multimodal features include biometric information in image form. The user device can then directly encode the biometric image into sub-encoding vectors after acquisition and submit them to the computing platform. In this case, the computing platform only needs to receive or read the pre-received and stored sub-encoding vectors.

[0057] Figure 3 This example illustrates a specific instance of multimodal features of a business entity. In this example, the business entity is a transaction user (whose user ID is shown in the figure), possessing the following five modal attribute features: username (text feature), geographic location (lbs geographic location feature), device used (device type feature), facial recognition feature for transaction verification (image feature), and unlock voice (voice feature). Encoding these five modal features separately yields the following results: Figure 3 The five modality features shown correspond to five sub-encoding vectors, including: a text embedding vector representing the user's name, an LBS encoding vector representing the location, a device type encoding vector representing the user's device, a graph feature encoding vector representing facial features, and a voiceprint feature vector representing speech.

[0058] Based on obtaining multiple sub-encoding vectors corresponding to the multimodal features of the target business entity, in step 22, a target encoding vector can be generated according to the multiple sub-encoding vectors. The target encoding vector includes an index field and a data field. The index field includes the metadata of each of the multiple sub-encoding vectors. The data field includes the multiple sub-encoding vectors.

[0059] Figure 4 The diagram illustrates the structure of a target encoding vector according to one embodiment. Specifically, for n sub-encoding vectors corresponding to n modal features, the index field of the target encoding vector may include n metadata items, such as metadata 1 to metadata n shown in the diagram. Each metadata item describes the characteristics of the corresponding sub-encoding vector. The data field may include n sub-encoding vectors arranged in the same order as the n metadata items. Fields may be separated by a preset symbol.

[0060] Specifically, in one embodiment, the metadata may include the vector length (or dimension) of the corresponding sub-encoded vector; used to extract each sub-encoded vector from the data field.

[0061] In one example, the metadata may also include the vector type of the corresponding sub-encoded vector, such as a text embedding vector, a graph feature encoding vector, a LBS location encoding vector, etc.; this vector type can be used to determine the distance calculation method for the corresponding sub-encoded vector when calculating entity distance / similarity.

[0062] In one embodiment, the metadata may also include the weights of the corresponding sub-encoded vectors when calculating entity distance.

[0063] In other examples, the metadata may optionally include other data describing the characteristics of the sub-coded vector, such as the encoding method, the numerical range of the vector elements, the position range of the sub-coded vector within the entire encoded vector, and so on.

[0064] exist Figure 4 In the example, the target encoding vector also includes a header field indicating the number n of sub-encoding vectors. This field can be omitted if the number n is pre-defined or fixed.

[0065] Figure 5 This example illustrates a specific instance of a target encoding vector. The example utilizes... Figure 4 The structure shown is for Figure 3 The sub-encoding vectors shown are synthesized to obtain the result. As mentioned earlier, Figure 3 The diagram shows five sub-encoding vectors corresponding to five modality features: V, representing the text embedding vector of the name. name The LBS encoding vector V represents the location. lbs V represents the device type encoding vector of the user equipment.device V represents the voiceprint feature vector of speech. voice , representing the graph feature encoding vector V of facial features face Based on the above 5 sub-encoding vectors, generate Figure 5 The target encoded vector shown in the figure includes a header field, an index field, and a data field. The header field has a value of 5, corresponding to 5 sub-encoded vectors. The index field contains 5 pieces of metadata, each including the vector type, vector weight, and vector length of the corresponding sub-encoded vector. For example, the first piece of metadata is V. name The metadata includes the text embedding vector type (represented by 1), a vector weight of 0.2, and a vector length of 512 dimensions. The second piece of metadata is V. lbs The metadata includes the LBS location feature type (represented by 2), a vector weight of 0.3, and a vector length of 2 dimensions. Other metadata structures and meanings are similar and will not be described individually. Data fields are formed by sequentially concatenating the aforementioned five sub-encoding vectors. Fields are distinguished and separated from each other, as well as from different sub-encoding vectors, using the symbol "#". Thus, based on... Figure 3 The multiple autoencode vectors of the multimodal features shown form a comprehensive encoding vector.

[0066] In the context of knowledge graphs, after generating comprehensive target encoding vectors for business entities using the methods described above, these encoding vectors can be associated with the business entities and stored in the corresponding database of the knowledge graph, facilitating subsequent reading and implementation of entity linking operations.

[0067] The following describes the process of performing entity matching analysis based on the target encoding vector generated in the above manner.

[0068] Figure 6 A flowchart illustrating an entity matching method for multimodal business entities according to one embodiment is shown. This method can be executed by a computing platform, which may specifically be a knowledge graph platform. Figure 6 As shown, the matching analysis method may include the steps described in detail below.

[0069] In step 61, the first and second encoding vectors corresponding to the first business entity and the second business entity are obtained respectively. Each encoding vector includes an index field and a data field. The index field includes the metadata of each of the multiple sub-encoding vectors corresponding to the multimodal features of the corresponding business entity. The data field includes the multiple sub-encoding vectors.

[0070] It can be understood that the first business entity and the second business entity are any two business entities to be analyzed for matching degree. For ease of description, the first and second encoding vectors corresponding to the first and second business entities are represented as vector X and vector Y, respectively. Both vector X and vector Y have the vector structure of the aforementioned target encoding vector, for example... Figure 4 The vector structure shown. More specifically, in one example, both vector X and vector Y have Figure 5 The vector form shown is different only in the specific values ​​of each field.

[0071] In one embodiment, each encoded vector also includes (e.g.) Figure 5 As shown, a header field indicates the number n of multiple sub-encoded vectors. In this case, multiple first metadata can be parsed from the first encoded vector and multiple second metadata can be parsed from the second encoded vector based on this header field. In another embodiment, the number n of sub-encoded vectors contained in each encoded vector is predetermined or fixed. In this case, the corresponding metadata can be extracted directly from the index field of the first and second encoded vectors based on the predetermined or fixed number n.

[0072] In step 62, based on the metadata in the index field, multiple first sub-encoding vectors and multiple second sub-encoding vectors are extracted from the first and second encoding vectors, respectively.

[0073] In one embodiment, the metadata includes the vector length of each sub-coded vector. Thus, based on the vector length of each sub-coded vector, multiple first sub-coded vectors can be extracted from the data field of the first coded vector X, and multiple second sub-coded vectors can be extracted from the data field of the second coded vector.

[0074] continue Figure 5 For example, assume that the first and second encoding vectors X and Y both have Figure 5 In the form of [formula missing], for vector X, its index field includes 5 segments of metadata. Based on the vector length indicated in each of the 5 segments of metadata, 5 sub-encoded vectors (the first sub-encoded vector) can be extracted from the data field, for example, X[formula missing]. name ,X lbs ,X device ,X voice ,X face A similar operation is performed on vector Y, extracting five sub-encoded vectors (second sub-encoded vectors) from its data fields, for example, Y. name ,Y lbs ,Y device ,Y voice ,Y face .

[0075] In another embodiment, the metadata can directly indicate the position range of the sub-coded vector within the entire coded vector. In this case, each sub-coded vector can be extracted directly based on the aforementioned position range, thereby obtaining the aforementioned plurality of first sub-coded vectors and plurality of second sub-coded vectors.

[0076] Next, in step 63, based on the metadata, the entity distance between the first business entity and the second business entity is determined using the plurality of first sub-encoding vectors and the plurality of second sub-encoding vectors. This entity distance corresponds to the degree of matching between the entities.

[0077] To determine the entity distance mentioned above, in one embodiment, firstly, multiple sub-distances are determined corresponding to multiple sub-vector groups formed by the corresponding sub-coding vectors in multiple first sub-coding vectors and multiple second sub-coding vectors; then, the multiple sub-distances are combined to obtain the entity distance.

[0078] Continuing with the previous example, multiple first sub-encoding vectors include X name ,X lbs ,X device ,X voice ,X face Multiple second sub-encoded vectors include Y name ,Y lbs ,Y device ,Y voice ,Y face The corresponding sub-encoding vectors form 5 sub-vector groups, (X name ,Y name ),(X lbs ,Y lbs ), (X device ,Y device ),(X voice ,Y voice ),(X face Y face We can first determine the five sub-distances d1(X) corresponding to the five sub-vector groups. name ,Y name ),d2(X lbs ,Y lbs ), d3(X device ,Y device ),d4(X voice ,Y voice ),d5(X face Y face Then, these are combined to obtain the final entity distance.

[0079] In one embodiment, since different sub-coded vectors correspond to different vector types, different distance calculation methods are required to determine the corresponding sub-distances. On the other hand, as mentioned earlier, the metadata can include a field indicating the vector type. Thus, based on the vector type of the sub-coded vectors contained in any target sub-vector group, the corresponding distance calculation method can be determined, and based on this method, the sub-distance corresponding to the target sub-vector group, i.e., the distance between the first and second sub-coded vectors included in the target sub-vector group, can be calculated.

[0080] Specifically, when the vector type is a text embedding vector, voiceprint feature vector, or graph feature encoding vector, the corresponding distance calculation method can be cosine distance; when the vector type is an LBS encoded vector, the corresponding distance calculation method can be spherical spatial distance; and when the vector type is a device type encoded vector, the corresponding distance calculation method can be Jaccard similarity coefficient. The distance calculation method for different vector types can be flexibly configured as needed, and no specific limitations are made here.

[0081] In one embodiment, as described above, the metadata may include the weights of the corresponding sub-coded vectors when calculating the entity distance. In such a case, when combining multiple sub-distances, the entity distance can be obtained by weighted summation of the multiple sub-distances according to the weights of the sub-coded vectors in each sub-vector group.

[0082] In a specific example, the entity distance D corresponding to the first encoding vector X and the second encoding vector Y can be calculated using the following formula:

[0083] D(X,Y)=0.2*f cos (X name ,Y name )+0.3*f sphere (X lbs ,Y lbs )+0.1*f jaccard (X device ,Y device )+0.1*f cos (X voice ,Y voice )+0.1*f cos (X face ,Y face (1)

[0084] In the above formula (1), f cos f represents the calculation of cosine distance. sphere f represents the calculation of the spatial distance of the sphere; jaccardThis indicates the calculation of the Jaccard similarity coefficient. According to formula (1), for any set of sub-coded vectors (Xi, Yi), the sub-distance is determined by the distance calculation method corresponding to the vector type of the sub-coded vector i; then, according to the weight of the sub-coded vector i indicated in the metadata, it is weighted, and the weighted sum of the sub-distances of each sub-vector group is used as the final entity distance D.

[0085] It is understandable that the entity distance determined above can characterize the matching degree between entities, and thus determine whether two business entities are matching entities for the target characteristic based on this matching degree. For example, if the matching degree represented by the entity distance between two business entities is greater than a certain threshold, it can be determined that the two business entities match each other for the target characteristic. Specifically, in the case of entity chaining operations on a knowledge graph platform, the above matching degree can be used to determine whether two entities in the graph have the same characteristics, such as whether two users belong to the same family, whether two payment codes belong to the same store, whether two accounts belong to the same natural person, and so on.

[0086] As can be seen, in the above entity matching analysis process, the entity distance, which represents the entity matching degree, can be directly calculated based on the encoding vector of the business entity, thus directly obtaining the result of whether a match is obtained. This matching analysis scheme simplifies the technical implementation of multi-path independent recall based on each modality feature, and reduces the time consumption of the fine ranking stage by reducing the number of recalls, thereby improving the efficiency of entity matching analysis.

[0087] According to another embodiment, with Figure 2 Corresponding to the encoding method shown, this specification also provides an apparatus for encoding multimodal business entities. This apparatus can be deployed in any computing unit, platform, server, or device with computing and processing capabilities, and in particular, it can be deployed in a knowledge graph platform. Figure 7 A schematic block diagram of an encoding apparatus for encoding multimodal business entities according to one embodiment is shown. Figure 7 As shown, the encoding device 700 includes:

[0088] Acquisition unit 71 is configured to acquire multiple sub-encoding vectors corresponding to the multimodal features of the target business entity;

[0089] Encoding unit 72 is configured to generate a target encoding vector based on the plurality of sub-encoding vectors. The target encoding vector includes an index field and a data field. The index field includes metadata of each of the plurality of sub-encoding vectors. The data field includes the plurality of sub-encoding vectors.

[0090] For details on the execution process of this encoding device and its various units, please refer to the combined... Figure 2The specific implementation process of the described encoding method will not be repeated here.

[0091] According to yet another embodiment, with Figure 6 Corresponding to the matching method shown, this specification also provides an apparatus for entity matching of multimodal business entities. This apparatus can be deployed in any computing unit, platform, server, or device with computing and processing capabilities, and in particular, it can be deployed in a knowledge graph platform. Figure 8 This diagram illustrates a matching apparatus for matching multimodal business entities according to one embodiment. Figure 8 As shown, the matching device 800 includes:

[0092] The encoding acquisition unit 81 is configured to acquire the first and second encoding vectors corresponding to the first business entity and the second business entity, respectively. Each encoding vector includes an index field and a data field. The index field includes metadata of each of the multiple sub-encoding vectors corresponding to the multimodal features of the corresponding business entity. The data field includes the multiple sub-encoding vectors.

[0093] The sub-vector extraction unit 82 is configured to extract multiple first sub-encoded vectors and multiple second sub-encoded vectors from the first and second encoded vectors respectively, based on the metadata in the index field.

[0094] The determining unit 83 is configured to determine the entity distance between the first business entity and the second business entity based on the metadata, using the plurality of first sub-encoding vectors and the plurality of second sub-encoding vectors, wherein the entity distance corresponds to the degree of matching between entities.

[0095] Each unit of the above matching device can be configured to perform... Figure 6 The method and steps, and the specific execution process, can be referred to in conjunction with the previous steps. Figure 6 The description is omitted.

[0096] According to other embodiments, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed in a computer, causes the computer to perform the aforementioned combination. Figure 2 or Figure 6 The described method and process.

[0097] According to another embodiment, a computing device is also provided, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to achieve the foregoing combination. Figure 2 or Figure 6 The described method and process.

[0098] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0099] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A coding method for multimodal business entities, comprising: Obtain multiple sub-encoding vectors corresponding to the multimodal features of the target business entity; the multimodal features include multiple of the following: text features, speech features, image features, geographic location features, device features, and behavioral features; the multiple sub-encoding vectors include multiple of the following: text embedding vector, voiceprint feature vector, graph feature encoding vector, LBS encoding vector, device type encoding vector, and behavioral sequence vector; A target encoding vector is generated based on the plurality of sub-encoding vectors. The target encoding vector includes an index field and a data field. The index field includes metadata for each of the plurality of sub-encoding vectors. The metadata includes the vector type of the corresponding sub-encoding vector, and the vector type is used to determine the distance calculation method for the corresponding sub-encoding vector. The data field includes the plurality of sub-encoding vectors.

2. The method according to claim 1, wherein, Obtain multiple sub-encoding vectors corresponding to the multimodal features of the target business entity, including: Obtain the multimodal features of the target business entity; Based on the respective encoding methods of each modality, the multimodal features are encoded into the multiple sub-encoding vectors.

3. The method according to claim 1, wherein, The metadata includes the vector length of the corresponding sub-encoded vector.

4. The method according to claim 1, wherein, The target encoding vector is used to determine the distance between business entities, and then to match the business entities; the metadata includes the weight of the corresponding sub-encoding vector when calculating the distance.

5. The method according to claim 1, wherein, The target encoding vector also includes a header field indicating the number of the plurality of sub-encoding vectors.

6. The method according to claim 1, wherein, The target business entity is a business entity in a knowledge graph; the target encoding vector is used for entity chaining.

7. The method according to claim 6, further comprising: The target encoding vector is associated with the target business entity and stored in the database corresponding to the knowledge graph.

8. An entity matching method for multimodal business entities, comprising: Obtain the first and second encoding vectors corresponding to the first and second business entities, respectively. Each encoding vector includes an index field and a data field. The index field includes metadata for each of the multiple sub-encoding vectors corresponding to the multimodal features of the corresponding business entity. The metadata includes the vector type of the corresponding sub-encoding vector, which is used to determine the distance calculation method of the corresponding sub-encoding vector. The data field includes the multiple sub-encoding vectors. The multimodal features include multiples of the following: text features, speech features, image features, geographic location features, device features, and behavioral features. The multiple sub-encoding vectors correspond to multiples of the following: text embedding vector, voiceprint feature vector, graph structure feature encoding vector, LBS encoding vector, device type encoding vector, and behavioral sequence vector. Based on the metadata in the index field, extract multiple first sub-encoding vectors and multiple second sub-encoding vectors from the first and second encoding vectors, respectively; Based on the metadata, the entity distance between the first business entity and the second business entity is determined using the plurality of first sub-encoding vectors and the plurality of second sub-encoding vectors, wherein the entity distance corresponds to the degree of matching between the entities.

9. The method according to claim 8, wherein, The metadata includes the vector length of each sub-encoded vector; Based on the metadata in the index field, extracting multiple first sub-encoded vectors and multiple second sub-encoded vectors from the first and second encoded vectors respectively includes: Based on the vector length of each sub-encoded vector, multiple first sub-encoded vectors are extracted from the data field of the first encoded vector, and multiple second sub-encoded vectors are extracted from the data field of the second encoded vector.

10. The method according to claim 8, wherein, Determining the entity distance between the first business entity and the second business entity includes: Determine multiple sub-distances corresponding to multiple sub-vector groups formed by corresponding sub-coding vectors in the plurality of first sub-coding vectors and the plurality of second sub-coding vectors; The entity distance is obtained by combining the multiple sub-distances.

11. The method according to claim 10, wherein, The plurality of sub-vector groups includes any target sub-vector group; Determining multiple sub-distances corresponding to multiple sub-vector groups formed by the multiple first sub-coding vectors and the multiple second sub-coding vectors includes: Based on the target vector type of the sub-encoded vectors contained in the target sub-vector group, determine the corresponding target distance calculation method; According to the target distance calculation method, the distance between the first sub-encoding vector and the second sub-encoding vector included in the target sub-vector group is calculated, and it is used as the sub-distance corresponding to the target sub-vector group.

12. The method according to claim 11, wherein, When the target vector type is a text embedding vector, a voiceprint feature vector, or a graph feature encoding vector, the target distance is calculated by calculating the cosine distance; When the target vector type is an LBS encoded vector, the target distance is calculated by calculating the spherical spatial distance; When the target vector type is a device type encoding vector, the target distance is calculated by calculating the Jaccard similarity coefficient.

13. The method according to claim 10, wherein, The metadata includes the weights of the corresponding sub-encoding vectors when calculating the entity distance; The step of combining the multiple sub-distances to obtain the entity distance includes: The entity distance is obtained by weighting and summing the multiple sub-distances according to the weights of the sub-encoding vectors in each sub-vector group.

14. The method according to claim 8, wherein, Each encoded vector also includes a header field indicating the number of the plurality of sub-encoded vectors; the method further includes: Based on the header field, multiple first metadata are parsed from the first encoding vector, and multiple second metadata are parsed from the second encoding vector.

15. The method according to claim 8, wherein, The first business entity and the second business entity are business entities in a knowledge graph; the method further includes: Based on the entity distance, determine whether the first business entity and the second business entity are matching entities for the target characteristics.

16. An encoding apparatus for multimodal business entities, comprising: The acquisition unit is configured to acquire multiple sub-encoding vectors corresponding to the multimodal features of the target business entity; the multimodal features include multiple of the following: text features, speech features, image features, geographic location features, device features, and behavioral features; the multiple sub-encoding vectors correspond to multiple of the following: text embedding vector, voiceprint feature vector, graph structure feature encoding vector, LBS encoding vector, device type encoding vector, and behavioral sequence vector; The encoding unit is configured to generate a target encoding vector based on the plurality of sub-encoding vectors. The target encoding vector includes an index field and a data field. The index field includes metadata of each of the plurality of sub-encoding vectors. The data field includes the plurality of sub-encoding vectors.

17. An entity matching device for multimodal business entities, comprising: The encoding acquisition unit is configured to acquire the first and second encoding vectors corresponding to the first business entity and the second business entity, respectively. Each encoding vector includes an index field and a data field. The index field includes metadata of each of the multiple sub-encoding vectors corresponding to the multimodal features of the corresponding business entity. The data field includes the multiple sub-encoding vectors. The multimodal features include multiple of the following: text features, speech features, image features, geographic location features, device features, and behavioral features. The multiple sub-encoding vectors include multiple of the following: text embedding vector, voiceprint feature vector, graph structure feature encoding vector, LBS encoding vector, device type encoding vector, and behavioral sequence vector. The sub-vector extraction unit is configured to extract multiple first sub-encoded vectors and multiple second sub-encoded vectors from the first and second encoded vectors respectively, based on the metadata in the index field; The determining unit is configured to determine the entity distance between the first business entity and the second business entity based on the metadata, using the plurality of first sub-encoding vectors and the plurality of second sub-encoding vectors, wherein the entity distance corresponds to the degree of matching between entities.

18. A computing device, comprising a memory and a processor, characterized in that, The memory stores executable code, and when the processor executes the executable code, it implements the method of any one of claims 1-15.

Images