An information extraction method and device

Abstract

The embodiment of the application provides a kind of information extraction method and device, example is related to artificial intelligence technical field, the method includes: obtaining the original data of target object including static attribute and dynamic attribute, and based on the original data, with target object as root node, dynamic attribute and static attribute are non-root node Global hierarchical attribute graph is constructed, global hierarchical attribute graph is mapped to uniform embedding space, information aggregation is carried out in the space, and basic representation information is obtained;Using selective aggregation mechanism, secondary information aggregation is carried out, enhanced representation information is obtained, and then based on the basic representation information and the enhanced representation information, the target representation information of the target object is obtained, effectively utilizes the high-order interaction relationship between static and dynamic attributes, which is implicit and beneficial to the task, eliminates the semantic gap, so that the whole feature extraction process can be automated, and the performance of the model on complex decision tasks is significantly improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of artificial intelligence technology, and in particular to an information extraction method and device. BACKGROUND

[0002] In the era of big data, the data generated and collected by information systems is increasingly complex and diverse. These data are often stored in semi-structured form and exhibit inherent hierarchical structure. How to efficiently extract valuable representation information from such mixed data containing multiple attributes to support downstream tasks such as user classification, risk assessment, and information recommendation is a great challenge.

[0003] Related technologies use a separate modeling method to process such data; specifically, different representation information is obtained by independently processing static attributes and dynamic records using completely different models; then the multiple representation information is fused by simple weighting or rules to obtain the final result.

[0004] However, in the above data processing process, the representation information of the static attributes and the dynamic records is located in different distribution spaces, resulting in a semantic gap; thus, directly fusing different representation information cannot effectively capture the implicit association between static attributes and dynamic records, resulting in low accuracy of information extraction, which in turn affects the accuracy of downstream tasks. SUMMARY

[0005] Embodiments of the present application provide an information extraction method and device for improving the accuracy of information extraction on complex heterogeneous data.

[0006] In one aspect, the present application provides an information extraction method, comprising:

[0007] obtaining original data of a target object, the original data comprising at least one static attribute and at least one dynamic attribute of the target object;

[0008] constructing a global hierarchical attribute graph based on the original data, wherein the global hierarchical attribute graph comprises a root node and a plurality of non-root nodes, the root node representing the target object, and each non-root node representing the static attribute or the dynamic attribute; each non-root node is directly or indirectly connected to the root node;

[0009] mapping the global hierarchical attribute graph to a unified embedding space, and aggregating the attributes represented by the plurality of non-root nodes to the root node in the embedding space to obtain basic representation information;

[0010] adopting a selective aggregation mechanism to perform secondary information aggregation in the global hierarchical attribute graph to obtain enhanced representation information;

[0011] obtain target representation information of the target object based on the basic representation information and the enhanced representation information.

[0012] In an aspect, an embodiment of the present application provides an information extraction device, comprising:

[0013] a composition module, configured to obtain original data of a target object, the original data comprising at least one static attribute and at least one dynamic attribute of the target object;

[0014] construct a global hierarchical attribute graph based on the original data, wherein the global hierarchical attribute graph comprises a root node and a plurality of non-root nodes, the root node representing the target object, and each non-root node representing the static attribute or the dynamic attribute; and each non-root node being directly or indirectly connected to the root node;

[0015] an information acquisition module, configured to map the global hierarchical attribute graph to a unified embedding space, and aggregate attributes represented by the plurality of non-root nodes respectively to the root node in the embedding space, to obtain basic representation information;

[0016] adopt a selective aggregation mechanism to perform secondary information aggregation in the global hierarchical attribute graph, to obtain enhanced representation information;

[0017] obtain target representation information of the target object based on the basic representation information and the enhanced representation information.

[0018] Optionally, the non-root nodes in the global hierarchical attribute graph comprise static nodes representing the static attributes and dynamic nodes representing the dynamic attributes.

[0019] connection edges between the static nodes represent hierarchical relationships between the static attributes, and connection edges between the dynamic nodes represent time sequences of the dynamic attributes.

[0020] Optionally, each node in the global hierarchical attribute graph comprises a text description of an entity represented by the node, the entity being any one of the target object, the static attribute, and the dynamic attribute.

[0021] The composition module is specifically configured to:

[0022] for each node in the global hierarchical attribute graph, encode a text description contained in the node into a text feature to obtain a corresponding text feature vector;

[0023] construct an initial node feature matrix corresponding to the global hierarchical attribute graph based on the obtained plurality of text feature vectors;

[0024] map the initial node feature matrix to a unified embedding space.

[0025] Optionally, the information obtaining module is specifically configured to:

[0026] determine a plurality of leaf nodes in the plurality of non-root nodes;

[0027] aggregate information from each of the leaf nodes to the root node in the embedding space to obtain base representation information.

[0028] Optionally, the embedding space comprises a static aggregation subspace and a dynamic aggregation subspace; the static aggregation subspace and the dynamic aggregation subspace are in an orthogonal relationship; the plurality of leaf nodes comprises static leaf nodes representing the static attributes and dynamic leaf nodes representing the dynamic attributes;

[0029] the information obtaining module is specifically configured to:

[0030] statically aggregate information along each of the static leaf nodes to the root node in the static aggregation subspace and dynamically aggregate information along each of the dynamic leaf nodes to the root node in the dynamic aggregation subspace to obtain base representation information.

[0031] Optionally, the static aggregation subspace and the dynamic aggregation subspace are obtained by constraining through an orthogonal loss function, and the orthogonal loss function is used to minimize the norm of the inner product between the static aggregation subspace and the dynamic aggregation subspace.

[0032] Optionally, the information obtaining module is specifically configured to:

[0033] determine an attention weight of an attribute represented by each non-root node relative to the target object;

[0034] map the global hierarchical attribute graph to a unified embedding space, and aggregate attributes represented by the plurality of non-root nodes to the root node in the embedding space according to the attention weights corresponding to the plurality of non-root nodes respectively to obtain enhanced representation information.

[0035] Optionally, the information obtaining module is specifically configured to:

[0036] determine an attention weight of an attribute represented by each non-root node relative to the target object based on an attention score calculation function comprising a truncation hyperparameter, wherein the truncation hyperparameter is used to truncate information flow of non-root nodes irrelevant to a downstream classification task.

[0037] Optionally, the information obtaining module is further configured to:

[0038] perform a downstream classification task on the target object based on the target representation information to obtain a classification result.

[0039] On one hand, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described information extraction method.

[0040] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the information extraction method described above.

[0041] On one hand, embodiments of this application provide a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the above-described information extraction method.

[0042] In this embodiment, a global hierarchical attribute graph is constructed based on heterogeneous raw data containing static and dynamic attributes. This global hierarchical attribute graph is then mapped to a unified embedding space, resulting in a basic representation information at the root node that comprehensively reflects all attributes of the entity. Next, a selective aggregation mechanism is used to perform secondary information aggregation on the global hierarchical attribute graph to obtain enhanced representation information. Finally, the basic representation information and the enhanced representation information are combined to form the final target representation information for various downstream tasks. This application eliminates the semantic gap by uniformly modeling the static and dynamic attributes of data in a shared representation space, enabling the entire information extraction process to be automated and significantly reducing manual design costs. It effectively discovers and utilizes the implicit, task-beneficial high-order interaction relationships between static and dynamic attributes, significantly improving the accuracy of information extraction and the model's performance on complex decision-making tasks. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 A schematic diagram of a system architecture provided in an embodiment of this application;

[0045] Figure 2 A flowchart illustrating an information extraction method provided in an embodiment of this application;

[0046] Figure 3 A schematic diagram of a global hierarchical attribute graph provided for an embodiment of the present application;

[0047] Figure 4 A structural schematic diagram of an information extraction device provided for an embodiment of the present application;

[0048] Figure 5 A structural schematic diagram of a computer device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and beneficial effects of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0050] For the convenience of understanding, the terms involved in the embodiments of the present application are explained below.

[0051] Selective Semantic Aggregation (SSAgg), a technology that selectively focuses and aggregates key semantic information in the input sequence through attention mechanisms and other methods, is widely used in natural language processing tasks to improve model performance.

[0052] The system architecture diagram to which the technical solutions of the embodiments of the present application are applicable is briefly introduced below. It should be noted that the system architecture diagram described below is only used to illustrate the embodiments of the present application and is not limiting.

[0053] Reference Figure 1 It is a system architecture diagram to which the embodiments of the present application are applicable. The system architecture at least includes a terminal device 101 and a server 102. The number of terminal devices 101 can be one or more, and the number of servers 102 can also be one or more. The present application does not specifically limit the number of terminal devices 101 and servers 102.

[0054] The terminal device 101 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart home appliance, a smart voice interaction device, a smart vehicle device, etc., but is not limited thereto.

[0055] The server 102 can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms, but is not limited thereto.

[0056] The information extraction method in the embodiments of the present application can be executed by the terminal device 101, or by the server 102, or by the terminal device 101 and the server 102 in interaction.

[0057] When the information extraction method in the embodiments of the present application is executed by the terminal device 101 or the server 102 alone, the following steps are executed: obtaining original data of a target object, constructing a global hierarchical attribute graph based on the original data, mapping the global hierarchical attribute graph to a unified embedding space, and aggregating attributes represented by multiple non-root nodes to a root node in the embedding space to obtain basic representation information; using a selective aggregation mechanism to perform secondary information aggregation on the global hierarchical attribute graph to obtain enhanced representation information; and obtaining target representation information of the target object based on the basic representation information and the enhanced representation information.

[0058] Of course, the terminal device 101 and the server 102 can also jointly execute the information extraction method in the present application, and the present application does not make specific limitations here.

[0059] The server 102 and the terminal device 101 can each include one or more processors, memories, and I / O interfaces for interaction, etc. In addition, the server 102 can also be configured with a database, which can be used to store target page parameters, etc. The memories of the server 102 and the terminal device 101 can also store program instructions required for execution of the information extraction method provided in the embodiments of the present application, and these program instructions can be executed by the processors to implement the information extraction process provided in the embodiments of the present application.

[0060] It should be noted that when the information extraction method provided in the embodiments of the present application is executed by the server 102 or the terminal device 101 alone, the system architecture of the present application can only include a single device of the server 102 or the terminal device 101, or the server 102 and the terminal device 101 can be considered as the same device. Of course, in actual application, when the information extraction method provided in the embodiments of the present application is executed by the server 102 and the terminal device 101 jointly, the server 102 and the terminal device 101 can also be the same device, i.e., the server 102 and the terminal device 101 can be different functional modules of the same device, or virtual devices virtualized by the same physical device.

[0061] In the embodiments of the present application, the terminal device 101 and the server 102 can be directly or indirectly connected through one or more networks. The network can be a wired network or a wireless network, for example, the wireless network can be a mobile cellular network or a wireless fidelity (WIFI) network, and of course, other possible networks are also possible, which are not limited in the embodiments of the present application.

[0062] Based on the system architecture diagram shown in the figure, the embodiments of the present application provide a flow of an information extraction method, as shown in the figure, the flow of the method is executed by a computer device, which can be the terminal device 101 and / or the server 102 shown in the figure, including the following steps: Figure 1 Figure 2 The flow of the method is executed by a computer device, which can be the terminal device 101 and / or the server 102 shown in the figure, including the following steps: Figure 1 The flow of the method is executed by a computer device, which can be the terminal device 101 and / or the server 102 shown in the figure, including the following steps:

[0063] Step S201, obtaining original data of a target object, the original data including at least one static attribute and at least one dynamic attribute of the target object.

[0064] Specifically, the original data is usually heterogeneous data in semi-structured formats such as JSON. The static attribute keeps unchanged during the running period, and is usually used to describe the fixed features of the target object. For example, "occupation: enterprise executive", "credit rating: good", "permanent address: XX city, XX district", etc.

[0065] The dynamic attribute changes its value during the program running period, and is usually used to represent the state of the object, for example, the user's loan record, etc.

[0066] Step S202, constructing a global hierarchical attribute graph based on the original data, wherein the global hierarchical attribute graph includes a root node and a plurality of non-root nodes, the root node represents the target object, and each non-root node represents a static attribute or a dynamic attribute; each non-root node is directly or indirectly connected to the root node;

[0067] Specifically, the target object is taken as the root node, which is the collection point of all information; the static attribute and the dynamic attribute of the target object are taken as the non-root nodes, which are directly or indirectly connected to the root node, forming a directed acyclic graph with the target object as the root, which represents the global hierarchical attribute graph of the target object.

[0068] For example, taking the scene of "an enterprise executive with good credit record, who made multiple small game top-ups at high frequency in the late night during the workday" as an example.

[0069] ​In the global hierarchical attribute graph, the root node represents an enterprise executive; the non-root nodes are used to represent dynamic attributes such as "Profession: Enterprise Executive", "Credit Rating: Good", "Income Level: High", and dynamic attributes such as "Transaction 1 (Type: Game Recharge, Amount: 10 Yuan, Time: 23:05)".

[0070] In some embodiments, the non-root nodes in the global hierarchical attribute graph include static nodes representing static attributes and dynamic nodes representing dynamic attributes; the connection edges between the static nodes represent the hierarchical relationship between the static attributes, and the connection edges between the dynamic nodes represent the time sequence between the dynamic attributes.

[0071] Specifically, two types of directed links, static links and dynamic links, are constructed according to the relationship between attributes. Static links are established between static nodes to reflect the inherent hierarchy or ownership (parent-child) relationship of data, and a directed edge is established from a child node to a parent node. For example, directed edges are established from the three static nodes "Profession", "Credit Rating", and "Income Level" to the root node "User". Among them, there is no link relationship between static nodes that have no hierarchical or ownership (parent-child) relationship, i.e., no directed edges are established between the three static nodes "Profession", "Credit Rating", and "Income Level".

[0072] Dynamic links are only established between dynamic nodes of the same attribute to capture the time sequence relationship, and a dynamic link points from an earlier record node to the next immediate record node. A dynamic link is usually represented by a directed edge. For example, a directed edge is established from the dynamic node "Record 1" to the dynamic node "Record 2", and a directed edge is established from the dynamic node "Record 2" to the dynamic node "Record 3".

[0073] In step S203, the global hierarchical attribute graph is mapped to a unified embedding space, and the attributes represented by each of the plurality of non-root nodes are aggregated to the root node in the embedding space to obtain basic representation information.

[0074] The embedding space is a low-dimensional, dense real vector space used to capture the semantic features of the original data. Through embedding technology, discrete symbols (such as words, sentences, images, etc.) are mapped to mathematical representations in a continuous vector space, so that entities with similar semantics are closer in the space.

[0075] The basic representation information refers to the most basic and numerical form that can be calculated by a model in data processing and machine learning models, which is used to represent the original input (such as text, image, category, etc.), and is the premise of subsequent feature extraction and semantic understanding.

[0076] In some embodiments, for each node in the global hierarchical attribute graph, a text feature code is obtained for a text description contained by the node, to obtain a corresponding text feature vector; based on the obtained multiple text feature vectors, an initial node feature matrix corresponding to the global hierarchical attribute graph is constructed; and the initial node feature matrix is mapped to a unified embedding space.

[0077] Specifically, each node in the global hierarchical attribute graph includes a text description of an entity represented by the node, the entity being any one of: a target object, a static attribute, and a dynamic attribute.

[0078] Text features are initialized for all nodes in the global hierarchical attribute graph. For a leaf node, the original text is "attribute name: attribute value"; for an intermediate node, it is "attribute name: [attribute identifier]"; and for a root node, it is a special "[root identifier]". Then, a pre-trained text encoder (e.g., a Sentence-BERT model) is used to uniformly encode these texts into initial text feature vectors, to finally form an initial node feature matrix where k is the feature dimension.

[0079] The obtained initial node feature matrix X is mapped to a higher-dimensional embedding space through a learnable linear transformation matrix to obtain an embedding matrix H = X·W .

[0080] For example, first, an original text description is added for each node in the global hierarchical attribute graph. For a root node, the text description is "[user root node]". For a static node, e.g., a node representing a profession, the text description is "profession: enterprise executive"; and for a node representing a credit rating, the text description is "credit rating: good". For a dynamic node, the same format is used, e.g., a node representing a first transaction, the text description is "transaction type: game top-up, amount: 10 yuan".

[0081] Next, a pre-trained text encoder (e.g., a Sentence-BERT model) is used to encode the above text content into a vector in a high-dimensional space, i.e., a text feature vector of the text.

[0082] Finally, a feature matrix is constructed using the above-obtained text feature vectors, and the feature vectors obtained by encoding the text information of all nodes in the graph are integrated (e.g., stacked by rows) to form an initial node feature matrix where N is the total number of nodes, and k is the dimension of each feature.

[0083] In some embodiments, multiple leaf nodes are determined from the multiple non-root nodes; and in the embedding space, information is aggregated from each leaf node to the root node to obtain basic representation information.

[0084] The plurality of leaf nodes include static leaf nodes representing static attributes and dynamic leaf nodes representing dynamic attributes.

[0085] In some embodiments, the embedding space includes a static aggregated subspace and a dynamic aggregated subspace; the static aggregated subspace and the dynamic aggregated subspace are in an orthogonal relationship. Static aggregation of information is performed along each static leaf node to the root node in the static aggregated subspace, and dynamic aggregation of information is performed along each dynamic leaf node to the root node in the dynamic aggregated subspace, to obtain basic representation information.

[0086] Specifically, two learnable linear transformation matrices and are defined, respectively, for defining the static aggregated subspace and the dynamic aggregated subspace.

[0087] The static aggregated subspace can be understood as a part specially used for interpreting and learning static attributes, and when information flows through a static link (for example, from a "degree" node to an "education experience" node), the feature transformation thereof will be mainly performed through this subspace.

[0088] The dynamic aggregated subspace can be regarded as a part specially used for capturing and understanding time-dependent relationships, and when information flows through a dynamic link (for example, from a previous transaction record to a subsequent transaction record), the feature transformation thereof is mainly performed through this dynamic subspace.

[0089] All static links together constitute a static initial node feature matrix (N is the total number of nodes), wherein if a node i is a sub-attribute of a node j , then , otherwise 0; all dynamic links constitute a dynamic initial node feature matrix (N is the total number of nodes), wherein if a node i is a sub-attribute of a node j , then , otherwise 0.

[0090] For example, as shown in Figure 3 , it is assumed that original data (JSON) in a semi-structured format with static attributes and dynamic attributes of a user is obtained, wherein the static attributes include occupation, permanent address (city, district), credit rating, and the dynamic attributes include transaction records (record 1, record 2, record 3...).

[0091] A global hierarchical attribute graph is constructed for the user, and the nodes in the graph are:

[0092] "User (root node)", “Profession”, “Credit rating”, “Residential address”,

[0093] “City”, “District”, “Transaction record” (manages dynamic sequences), “Record 1” (dynamic node), “Record 2” (dynamic node), “Record 3” (dynamic node), wherein, the earliest record in the dynamic sequence is connected to the node by means of static linkage.

[0094] Therefore, for the static initial node feature matrix , the items with value 1 (the rest are 0) include:

[0095] (linkage: → ), (linkage: → ), (linkage: → ), (linkage: → ), (linkage: → ), (linkage: → ), (linkage: → ).

[0096] For the dynamic initial node feature matrix , the items with value 1 (the rest are 0) include:

[0097] (linkage: → ), (linkage: → ).

[0098] The model adopts a bottom-up approach, starting from the leaf nodes and aggregating information layer by layer to the root node. For any non-leaf node in the global hierarchical attribute graph, the updated representation is updated according to its one or more child nodes is calculated, and the specific calculation method depends on the link type between and , which is divided into the following two cases:

[0099] Static aggregation in static aggregation subspace: if the child node is connected to by static link, its information aggregation formula is shown in the following formula (1):

[0100] (1)

[0101] Dynamic reasoning in dynamic aggregation subspace: if the child node is connected to by dynamic link (i.e., they are adjacent records of the same dynamic attribute), a gated recurrent unit (GRU) network is used for time series modeling, and its information aggregation formula is shown in the following formula (2):

[0102] (2)

[0103] In formula (1) and formula (2), represents the initial node representation of the current node before updating;

[0104] represents the updated (i.e., processed) node representation of the child node of the current node (in static aggregation) or the previous time node (in dynamic reasoning);

[0105] and are linear transformation matrices of static aggregation subspace and dynamic aggregation subspace, respectively, and are pseudo-inverses of and , respectively;

[0106] and (only used for static aggregation) are learnable attention weights for aggregating its own information and the information from its child node ;

[0107] n (only used for static aggregation) represents the total number of static child nodes possessed by the node ;

[0108] ​GRU (only for dynamic inference) represents the gate recurrent unit. GRU (only for dynamic inference) represents the hidden state passed from the previous time step (node ) to the GRU.

[0109] When the information propagation finally converges to the root node r, the updated representation of r is obtained . This representation is the basic representation information of the target object, denoted as = .

[0110] Specifically, the latest record in the dynamic record is connected to a node summarizing the properties of the dynamic node through static linking, and the root node uses the static aggregation formula (1) described above to calculate the final output representation. For example, the latest record node in multiple loan records is linked to the loan record node, and then this node is connected to the root node through static linking, thereby ensuring that all nodes under the root node are connected to it through static linking.

[0111] In the embodiments of the present application, static aggregation is mainly performed in the subspace (main term), while the component in the subspace is retained (secondary term) to maintain the unity of the representation. Dynamic aggregation is mainly performed in the subspace, while the component in the subspace is retained. In this way, regardless of whether the information path is static or dynamic, the representation of all nodes is always located in the same unified semantic space.

[0112] In some embodiments, the static aggregation subspace and the dynamic aggregation subspace are in an orthogonal relationship, which is obtained by constraint through an orthogonal loss function, which is used to minimize the norm of the inner product between the static aggregation subspace and the dynamic aggregation subspace.

[0113] To ensure that the two information processing paths are independent at the parameter level, an orthogonality loss term is added to the final loss function to achieve this, as shown in the following formula (3):

[0114] (3)

[0115] wherein represents the square of the norm of the matrix F.

[0116] This loss term is minimized during model training, thereby forcing the basis vectors of the two subspaces to be orthogonal to each other. If the degree of orthogonality of the two subspaces is not large enough, then The value of the loss function will be larger, thus generating a "penalty" signal, and the model will automatically adjust the parameters of the two subspaces to minimize the loss until the two subspaces are mathematically approximately orthogonal, thereby forcing the basis vectors of the two subspaces to be independent of each other.

[0117] wherein the total loss function of the model is l defined as shown in the following formula (4):

[0118] (4)

[0119] wherein, is the total loss function of the model;

[0120] is the loss function of the downstream task, for example, the cross-entropy loss of the classification task or the mean absolute error of the regression task, which is calculated by comparing the predicted output of the model with the "true label" of the downstream task;

[0121] is the orthogonality loss, which calculates the square of the F-norm of the inner product between the static subspace matrix and the dynamic subspace matrix to measure the degree of orthogonality between the two;

[0122] γ is a weighting hyperparameter for balancing the two losses, which are minimized during the model training process, so that the term is also minimized.

[0123] Step S204, using a selective aggregation mechanism, performing secondary information aggregation on the global hierarchical attribute graph to obtain enhanced representation information;

[0124] wherein the selective semantic aggregation is a technology for selectively focusing and aggregating key semantic information in the input sequence through attention mechanisms and the like, which assigns a learnable weight to different parts of the input sequence, highlights important semantic units, and suppresses irrelevant or redundant information to achieve efficient modeling of the context.

[0125] In some embodiments, the attention weight of each non-root node representation with respect to the target object is determined based on an attention score calculation function containing a truncated hyperparameter, wherein the truncated hyperparameter is used to truncate the information flow of the non-root node irrelevant to the downstream classification task.

[0126] Specifically, the attention weight of the neighbor node to the center node is calculated, which is specifically shown in the following formula (5):

[0127] (5)

[0128] where, is the attention score computed by a learnable scoring function , is a set of neighbor nodes of ; the key difference from standard attention mechanisms is the addition of a positive term in the denominator where is a learnable or pre-defined hyper-parameter, acts as a “cut-off” or “gate”.

[0129] By learning the scoring function and the hyper-parameter during training, the model can autonomously and selectively “turn off” certain information propagation paths. For example, if the model considers the information from node to as noise or irrelevant to the task, it can learn a very small attention score , making the corresponding weight tend to zero.

[0130] Taking the center node in Figure 3 as an example, when the model calculates the representation information of the root node (“user” node): the center node is , the neighbor nodes are (“occupation”), (“credit rating”), (“permanent address”), (“transaction record”); the model will calculate the attention weights of the links → , → , etc.

[0131] In the embodiments of the present application, by adjusting and

[0132] , this mechanism can functionally simulate any node selection and edge pruning, equivalent to information propagation on an optimal, task-related superstructure, efficiently capturing high-order interaction information across static and dynamic attributes.

[0133] In some embodiments, the attention weight of each non-root node representation attribute relative to the target object is determined; the global hierarchical attribute graph is mapped to a unified embedding space, and in the embedding space, the attributes represented by the multiple non-root nodes are aggregated into the root node according to the respective attention weights of the multiple non-root nodes, to obtain enhanced representation information. any of the non-leaf "center nodes" in the tree defined by the function The model will use the resulting attention weights to perform a weighted sum of the base representations of all "neighbor nodes" (i.e. the nodes of the tree rooted at to compute a new representation at the second stage of information aggregation, as shown in the following equation (6):

[0134] (6)

[0135] where is a newly defined learnable linear transformation matrix for the SSAgg second aggregation stage.

[0136] After the second bottom-up information propagation via the SSAgg mechanism, when the information propagation finally converges at the root node r, a new representation is obtained. This representation is the enhanced representation information of the target object, denoted as = .

[0137] In the embodiments of the present application, the SSAgg mechanism functionally simulates arbitrary node selection and edge pruning, which is equivalent to performing information propagation on an optimal, task-related superstructure, efficiently capturing complex high-order interaction information across static and dynamic attributes.

[0138] In step S205, the target representation information of the target object is obtained based on the base representation information and the enhanced representation information.

[0139] Specifically, the obtained base representation information and the enhanced representation information are combined to obtain the target representation information of the target object , as shown in the following equation (7):

[0140] (7)

[0141] In some embodiments, a downstream classification task is performed on the target object based on the target representation information to obtain a classification result.

[0142] The target representation information is input into a simple fully connected layer classifier or regressor to perform a downstream classification task (such as user churn prediction, credit risk assessment, etc.).

[0143] In the embodiment of the present application, a global layered attribute graph is constructed according to heterogeneous raw data containing static attributes and dynamic attributes, to obtain initial node representation of nodes; the initial node representation is mapped to a unified embedding space, to obtain a basic representation of an entity at a root node, which can comprehensively reflect all attributes of the entity; then, complex interaction between static and dynamic attributes is further captured, to obtain an enhanced entity representation at the root node; finally, the basic representation information and the enhanced representation information are combined to form final target representation information, which is used for various downstream tasks.

[0144] The present application unifies modeling of static and dynamic attributes of data in a shared representation space, eliminates semantic gaps, enables automatic feature extraction, greatly reduces manual design costs, effectively discovers and utilizes high-order interaction between static and dynamic attributes, which is beneficial to tasks, and significantly improves performance of the model in complex decision tasks.

[0145] Based on the same technical concept, an embodiment of the present application provides a structural schematic diagram of an information extraction device, as shown in Figure 4 The information extraction device 400 includes:

[0146] A composition module 401 is configured to acquire raw data of a target object, wherein the raw data includes at least one static attribute and at least one dynamic attribute of the target object.

[0147] A global layered attribute graph is constructed based on the raw data, wherein the global layered attribute graph includes a root node and a plurality of non-root nodes, the root node represents the target object, each non-root node represents the static attribute or the dynamic attribute, and each non-root node is directly or indirectly connected to the root node.

[0148] An information acquisition module 402 is configured to map the global layered attribute graph to a unified embedding space, aggregate attributes represented by the plurality of non-root nodes to the root node in the embedding space, and obtain basic representation information.

[0149] A selective aggregation mechanism is adopted to perform secondary information aggregation on the global layered attribute graph, to obtain enhanced representation information.

[0150] Based on the basic representation information and the enhanced representation information, target representation information of the target object is obtained.

[0151] Optionally, the non-root nodes in the global layered attribute graph include static nodes representing the static attributes and dynamic nodes representing the dynamic attributes.

[0152] The connection edge between the static nodes represents a hierarchical relationship between static attributes, and the connection edge between the dynamic nodes represents a time sequence relationship between dynamic attributes.

[0153] Optionally, each node in the global hierarchical attribute graph comprises a text description of an entity represented by the node, the entity being any one of the target object, the static attribute, and the dynamic attribute.

[0154] The composition module 401 is specifically configured to:

[0155] For each node in the global hierarchical attribute graph, the text description contained in the node is encoded into a text feature to obtain a corresponding text feature vector.

[0156] Based on the obtained plurality of text feature vectors, an initial node feature matrix corresponding to the global hierarchical attribute graph is constructed.

[0157] The initial node feature matrix is mapped to a unified embedding space.

[0158] Optionally, the information acquisition module 402 is specifically configured to:

[0159] Determine a plurality of leaf nodes in the plurality of non-root nodes.

[0160] In the embedding space, information is aggregated from each of the leaf nodes to the root node to obtain basic representation information.

[0161] Optionally, the embedding space comprises a static aggregation subspace and a dynamic aggregation subspace; the static aggregation subspace and the dynamic aggregation subspace are in an orthogonal relationship; the plurality of leaf nodes comprises static leaf nodes representing the static attributes and dynamic leaf nodes representing the dynamic attributes.

[0162] The information acquisition module 402 is specifically configured to:

[0163] In the static aggregation subspace, information is statically aggregated along each of the static leaf nodes to the root node, and in the dynamic aggregation subspace, information is dynamically aggregated along each of the dynamic leaf nodes to the root node to obtain basic representation information.

[0164] Optionally, the static aggregation subspace and the dynamic aggregation subspace are in an orthogonal relationship obtained by constraining an orthogonal loss function, and the orthogonal loss function is used to minimize the norm of the inner product between the static aggregation subspace and the dynamic aggregation subspace.

[0165] Optionally, the information acquisition module 402 is specifically configured to:

[0166] determine an attention weight of each non-root node representation attribute compared to the target object;

[0167] map the global hierarchical attribute graph to a unified embedding space, and aggregate the attributes represented by the plurality of non-root nodes to the root node in the embedding space according to the attention weights corresponding to the plurality of non-root nodes respectively, to obtain enhanced representation information.

[0168] Optionally, the information acquisition module 402 is specifically configured to:

[0169] determine an attention weight of each non-root node representation attribute compared to the target object based on an attention score calculation function containing a truncation hyperparameter, wherein the truncation hyperparameter is used to truncate the information flow of non-root nodes irrelevant to a downstream classification task.

[0170] Optionally, the information acquisition module 402 is further configured to:

[0171] perform a downstream classification task on the target object based on the target representation information, to obtain a classification result.

[0172] In the embodiments of the present application, a unified static and dynamic attribute information extraction method is provided, a global hierarchical attribute graph is constructed according to heterogeneous original data containing static attributes and dynamic attributes, to obtain initial node representations of nodes; the initial node representations are mapped to a unified embedding space, to obtain a basic representation of an entity at the root node, which can comprehensively reflect all attributes of the entity; then, the complex interaction relationship between the static and dynamic attributes for a downstream task is further captured, to obtain an enhanced entity representation at the root node; finally, the basic representation information and the enhanced representation information are combined to form final target representation information, which is used for various downstream tasks. The present application unifies the modeling of static and dynamic attributes of data in a shared representation space, eliminates semantic gaps, enables the entire feature extraction process to be automated, greatly reduces the cost of manual design, effectively discovers and utilizes the implicit high-order interaction relationship between static and dynamic attributes that is beneficial to the task through a novel attention weighting mechanism, and significantly improves the performance of the model in complex decision tasks.

[0173] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as a processing circuit or a memory) or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that contains the functions of the module or unit.

[0174] Based on the same technical concept, the embodiment of the present application provides a computer device, which can be Figure 1 The terminal device and / or the server shown in FIG. 1 comprises at least one processor 501 and a memory 502 connected with the at least one processor, and the specific connection medium between the processor 501 and the memory 502 is not limited in the embodiment of the present application. Figure 5 The processor 501 and the memory 502 are connected through a bus in the embodiment of the present application. The bus can be divided into an address bus, a data bus, a control bus and the like. Figure 5

[0175] In the embodiment of the present application, the memory 502 stores instructions executable by the at least one processor 501, and the at least one processor 501 can execute the steps of the information extraction method by executing the instructions stored in the memory 502.

[0176] The processor 501 is the control center of the computer device, can connect various parts of the computer device through various interfaces and lines, and can realize virtual film production by running or executing instructions stored in the memory 502 and calling data stored in the memory 502. Optionally, the processor 501 can include one or more processing units, and the processor 501 can integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface and an application program, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 501. In some embodiments, the processor 501 and the memory 502 can be realized on the same chip, and in some embodiments, they can also be realized on independent chips respectively.

[0177] The processor 501 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can realize or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0178] ​The memory 502, as a non-volatile computer readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 502 can include at least one type of storage medium, for example, can include flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. The memory 502 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer device, but is not limited to this. The memory 502 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used to store program instructions and / or data.

[0179] Based on the same inventive concept, the embodiments of the present application provide a computer readable storage medium storing a computer program executable by a computer device, which, when running on the computer device, causes the computer device to perform the steps of the information extraction method.

[0180] Based on the same inventive concept, the embodiments of the present application provide a computer program product, which comprises a computer program stored on a computer readable storage medium, the computer program comprising program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the information extraction method.

[0181] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program code.

[0182] The present application is described in reference to the flowchart and / or block diagrams of the methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more blocks or combinations of blocks in the flowchart and / or block diagram.

[0183] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more blocks or combinations of blocks in the flowchart and / or block diagram.

[0184] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that are executed on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more blocks or combinations of blocks in the flowchart and / or block diagram.

[0185] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments by those of skill in the art once they have the benefit of the present disclosure without departing from the spirit and scope of the application. Accordingly, the attached claims are intended to embrace all such variations and modifications as fall within the scope of the present application.

[0186] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. An information extraction method, characterized in that, include: Obtain the original data of the target object, wherein the original data includes at least one static attribute and at least one dynamic attribute of the target object; A global hierarchical attribute graph is constructed based on the original data. The global hierarchical attribute graph includes a root node and multiple non-root nodes. The root node represents the target object, and each non-root node represents the static attribute or the dynamic attribute. Each non-root node is directly or indirectly connected to the root node. The global hierarchical attribute graph is mapped to a unified embedding space, which includes a static aggregation subspace and a dynamic aggregation subspace of orthogonal relations. Determine multiple leaf nodes among the multiple non-root nodes, the multiple leaf nodes including: static leaf nodes representing the static attribute and dynamic leaf nodes representing the dynamic attribute; In the static aggregation subspace, information is statically aggregated from each static leaf node to the root node while retaining the components of the dynamic aggregation subspace; and in the dynamic aggregation subspace, information is dynamically aggregated from each dynamic leaf node to the root node while retaining the components of the static aggregation subspace, thereby obtaining basic representation information. A selective aggregation mechanism is used to perform secondary information aggregation on the global hierarchical attribute graph to obtain enhanced representation information. Based on the basic representation information and the enhanced representation information, the target representation information of the target object is obtained.

2. The method as described in claim 1, characterized in that, The non-root nodes in the global hierarchical attribute graph include: static nodes representing the static attributes and dynamic nodes representing the dynamic attributes; The connecting edges between static nodes represent the hierarchical relationship between static attributes, and the connecting edges between dynamic nodes represent the temporal sequence between dynamic attributes.

3. The method as described in claim 1, characterized in that, Each node in the global hierarchical attribute graph includes: a text description of the entity represented by the node, wherein the entity is any one of the following: the target object, the static attribute, and the dynamic attribute; The step of mapping the global hierarchical attribute graph to a unified embedding space includes: For each node in the global hierarchical attribute graph, the text description contained in the node is encoded into text features to obtain the corresponding text feature vector; Based on the obtained multiple text feature vectors, construct the initial node feature matrix corresponding to the global hierarchical attribute graph; The initial node feature matrix is ​​mapped to a unified embedding space.

4. The method as described in claim 1, characterized in that, The orthogonality between the static aggregation subspace and the dynamic aggregation subspace is obtained through orthogonal loss function constraints, which is used to minimize the norm of the inner product between the static aggregation subspace and the dynamic aggregation subspace.

5. The method according to any one of claims 1 to 4, characterized in that, The selective aggregation mechanism is used to perform secondary information aggregation on the global hierarchical attribute graph to obtain enhanced representation information, including: Determine the attention weight of the attribute represented by each non-root node relative to the target object; The global hierarchical attribute graph is mapped to a unified embedding space, and the attributes represented by the multiple non-root nodes are aggregated to the root node according to the attention weights corresponding to the multiple non-root nodes in the embedding space to obtain enhanced representation information.

6. The method as described in claim 5, characterized in that, Determining the attention weight of each attribute represented by a non-root node relative to the target object includes: Based on the attention score calculation function that includes a truncation hyperparameter, the attention weight of the attribute represented by each non-root node relative to the target object is determined, wherein the truncation hyperparameter is used to truncate the information flow of non-root nodes that are irrelevant to the downstream classification task.

7. The method according to any one of claims 1 to 4, characterized in that, After obtaining the target representation information of the target object based on the basic representation information and the enhanced representation information, the method further includes: Based on the target representation information, a downstream classification task is performed on the target object to obtain the classification result.

8. An information extraction device, characterized in that, include: A mapping module is used to acquire the original data of a target object, the original data including: at least one static attribute and at least one dynamic attribute of the target object; A global hierarchical attribute graph is constructed based on the original data. The global hierarchical attribute graph includes a root node and multiple non-root nodes. The root node represents the target object, and each non-root node represents the static attribute or the dynamic attribute. Each non-root node is directly or indirectly connected to the root node. Information acquisition module, used for The global hierarchical attribute graph is mapped to a unified embedding space, which includes a static aggregation subspace and a dynamic aggregation subspace of orthogonal relations. Determine multiple leaf nodes among the multiple non-root nodes, the multiple leaf nodes including: static leaf nodes representing the static attribute and dynamic leaf nodes representing the dynamic attribute; In the static aggregation subspace, information is statically aggregated from each of the static leaf nodes to the root node while retaining the components of the dynamic aggregation subspace; and in the dynamic aggregation subspace, information is dynamically aggregated from each of the dynamic leaf nodes to the root node while retaining the components of the static aggregation subspace, thereby obtaining basic representation information. A selective aggregation mechanism is used to perform secondary information aggregation on the global hierarchical attribute graph to obtain enhanced representation information. Based on the basic representation information and the enhanced representation information, the target representation information of the target object is obtained.

Images