Information extraction method and device

Abstract

The embodiment of the invention provides an information extraction method and device, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining original data, including static attributes and dynamic attributes, of a target object, and constructing a global hierarchical attribute graph based on the original data by taking the target object as a root node and taking the dynamic attributes and the static attributes as non-root nodes, mapping the global hierarchical attribute graph to a unified embedding space, and performing information aggregation in the space to obtain basic representation information; a selective aggregation mechanism is adopted to perform secondary information aggregation to obtain enhanced representation information, and then target representation information of the target object is obtained based on the basic representation information and the enhanced representation information, so that a high-order interaction relationship which is implied between static and dynamic attributes and is beneficial to tasks is effectively utilized, a semantic gap is eliminated, and the target representation efficiency is improved. The whole feature extraction process can be automatically carried out, and the performance of the model on complex decision-making tasks is remarkably improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an information extraction method and apparatus. Background Technology

[0002] In the era of big data, the data generated and collected by information systems is becoming increasingly complex and diverse. This data is often stored in a semi-structured form and exhibits an inherent hierarchical structure. How to efficiently extract valuable representational information from such mixed data containing multiple attributes to support downstream tasks such as user classification, risk assessment, and information recommendation is a significant challenge.

[0003] Related technologies use a separate modeling approach to process this type of data; specifically, static attributes and dynamic records are processed independently using distinct models to obtain different representational information; then, multiple representational information are fused through simple weighting or rules to obtain the final result.

[0004] However, in the process of processing the data described above, the representational information of static attributes and dynamic records are located in different distribution spaces, which will create a semantic gap. Thus, directly fusing different representational information cannot effectively capture the implicit relationship between static attributes and dynamic records, resulting in low accuracy of information extraction, which in turn affects the accuracy of downstream tasks. Summary of the Invention

[0005] This application provides an information extraction method and apparatus to improve the accuracy of information extraction from complex heterogeneous data.

[0006] On the one hand, embodiments of this application provide an information extraction method, including: Obtain the original data of the target object, wherein the original data includes at least one static attribute and at least one dynamic attribute of the target object; A global hierarchical attribute graph is constructed based on the original data. The global hierarchical attribute graph includes a root node and multiple non-root nodes. The root node represents the target object, and each non-root node represents the static attribute or the dynamic attribute. Each non-root node is directly or indirectly connected to the root node. The global hierarchical attribute graph is mapped to a unified embedding space, and the attributes represented by the multiple non-root nodes are aggregated to the root node in the embedding space to obtain basic representation information. A selective aggregation mechanism is used to perform secondary information aggregation on the global hierarchical attribute graph to obtain enhanced representation information. Based on the basic representation information and the enhanced representation information, the target representation information of the target object is obtained.

[0007] On one hand, embodiments of this application provide an information extraction device, including: A mapping module is used to acquire the original data of a target object, the original data including: at least one static attribute and at least one dynamic attribute of the target object; A global hierarchical attribute graph is constructed based on the original data. The global hierarchical attribute graph includes a root node and multiple non-root nodes. The root node represents the target object, and each non-root node represents the static attribute or the dynamic attribute. Each non-root node is directly or indirectly connected to the root node. The information acquisition module is used to map the global hierarchical attribute graph to a unified embedding space, and to aggregate the attributes represented by the multiple non-root nodes to the root node in the embedding space to obtain basic representation information. A selective aggregation mechanism is used to perform secondary information aggregation on the global hierarchical attribute graph to obtain enhanced representation information. Based on the basic representation information and the enhanced representation information, the target representation information of the target object is obtained.

[0008] Optionally, the non-root nodes in the global hierarchical attribute graph include: static nodes representing the static attributes and dynamic nodes representing the dynamic attributes; The connecting edges between static nodes represent the hierarchical relationship between static attributes, and the connecting edges between dynamic nodes represent the temporal sequence between dynamic attributes.

[0009] Optionally, each node in the global hierarchical attribute graph includes: a text description of the entity represented by the node, wherein the entity is any one of the following: the target object, the static attribute, and the dynamic attribute; The composition module is specifically used for: For each node in the global hierarchical attribute graph, the text description contained in the node is encoded into text features to obtain the corresponding text feature vector; Based on the obtained multiple text feature vectors, construct the initial node feature matrix corresponding to the global hierarchical attribute graph; The initial node feature matrix is ​​mapped to a unified embedding space.

[0010] Optionally, the information acquisition module is specifically used for: Identify multiple leaf nodes among the multiple non-root nodes; In the embedding space, information is aggregated from each leaf node to the root node to obtain basic representation information.

[0011] Optionally, the embedding space includes: a static aggregation subspace and a dynamic aggregation subspace; the static aggregation subspace and the dynamic aggregation subspace are orthogonal; the plurality of leaf nodes include: static leaf nodes representing the static attributes and dynamic leaf nodes representing the dynamic attributes; The information acquisition module is specifically used for: In the static aggregation subspace, information is statically aggregated from each of the static leaf nodes to the root node, and in the dynamic aggregation subspace, information is dynamically aggregated from each of the dynamic leaf nodes to the root node to obtain basic representation information.

[0012] Optionally, the orthogonality between the static aggregation subspace and the dynamic aggregation subspace is obtained through orthogonal loss function constraints, wherein the orthogonal loss function is used to minimize the norm of the inner product between the static aggregation subspace and the dynamic aggregation subspace.

[0013] Optionally, the information acquisition module is specifically used for: Determine the attention weight of the attribute represented by each non-root node relative to the target object; The global hierarchical attribute graph is mapped to a unified embedding space, and the attributes represented by the multiple non-root nodes are aggregated to the root node according to the attention weights corresponding to the multiple non-root nodes in the embedding space to obtain enhanced representation information.

[0014] Optionally, the information acquisition module is specifically used for: Based on the attention score calculation function that includes a truncation hyperparameter, the attention weight of each attribute represented by a non-root node relative to the target object is determined, wherein the truncation hyperparameter is used to truncate the information flow of non-root nodes that are irrelevant to the downstream classification task.

[0015] Optionally, the information acquisition module is further configured to: Based on the target representation information, a downstream classification task is performed on the target object to obtain the classification result.

[0016] On one hand, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described information extraction method.

[0017] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the information extraction method described above.

[0018] On one hand, embodiments of this application provide a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the above-described information extraction method.

[0019] In this embodiment, a global hierarchical attribute graph is constructed based on heterogeneous raw data containing static and dynamic attributes. This global hierarchical attribute graph is then mapped to a unified embedding space, resulting in a basic representation information at the root node that comprehensively reflects all attributes of the entity. Next, a selective aggregation mechanism is used to perform secondary information aggregation on the global hierarchical attribute graph to obtain enhanced representation information. Finally, the basic representation information and the enhanced representation information are combined to form the final target representation information for various downstream tasks. This application eliminates the semantic gap by uniformly modeling the static and dynamic attributes of data in a shared representation space, enabling the entire information extraction process to be automated and significantly reducing manual design costs. It effectively discovers and utilizes the implicit, task-beneficial high-order interaction relationships between static and dynamic attributes, significantly improving the accuracy of information extraction and the model's performance on complex decision-making tasks. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A schematic diagram of a system architecture provided in an embodiment of this application; Figure 2 A flowchart illustrating an information extraction method provided in an embodiment of this application; Figure 3 A schematic diagram of a global hierarchical attribute graph provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an information extraction device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0023] For ease of understanding, the terms used in the embodiments of this invention are explained below.

[0024] Selective Semantic Aggregation (SSAgg) is a technique that selectively focuses on and aggregates key semantic information in an input sequence using methods such as attention mechanisms. It is widely used in natural language processing tasks to improve model performance.

[0025] The following is a brief introduction to the system architecture diagram applicable to the technical solutions of the embodiments of this application. It should be noted that the system architecture diagram described below is only used to illustrate the embodiments of this application and is not intended to limit the scope of the application.

[0026] refer to Figure 1 This is a system architecture diagram applicable to the embodiments of this application. The system architecture includes at least terminal device 101 and server 102. The number of terminal devices 101 can be one or more, and the number of servers 102 can also be one or more. This application does not specifically limit the number of terminal devices 101 and servers 102.

[0027] Terminal device 101 may be a smartphone, tablet computer, laptop computer, desktop computer, smart home appliance, smart voice interaction device, smart vehicle device, etc., but is not limited to these.

[0028] Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, but it is not limited to these.

[0029] The information extraction method in this application embodiment can be executed by terminal device 101, server 102, or by interaction between terminal device 101 and server 102.

[0030] When the information extraction method in this embodiment is executed by either terminal device 101 or server 102 alone, the following steps are performed: obtaining the original data of the target object; constructing a global hierarchical attribute graph based on the original data; mapping the global hierarchical attribute graph to a unified embedding space; and aggregating the attributes represented by multiple non-root nodes to the root node in the embedding space to obtain basic representation information; using a selective aggregation mechanism to perform secondary information aggregation on the global hierarchical attribute graph to obtain enhanced representation information; and obtaining the target representation information of the target object based on the basic representation information and the enhanced representation information.

[0031] Of course, the actions described in this application can also be performed jointly by the terminal device 101 and the server 102, and this application does not make any specific limitations on this.

[0032] Both server 102 and terminal device 101 may include one or more processors, memory, and interactive I / O interfaces. Furthermore, server 102 may be configured with a database for storing target page parameters, etc. The memory of server 102 and terminal device 101 may also store program instructions required for execution in the information extraction method provided in this application embodiment. These program instructions, when executed by the processor, can be used to implement the information extraction process provided in this application embodiment.

[0033] It should be noted that when the information extraction method provided in this application embodiment is executed by either server 102 or terminal device 101 alone, the system architecture of this application may also include only a single device, either server 102 or terminal device 101. Alternatively, server 102 and terminal device 101 may be considered as the same device. Of course, in practical applications, when the information extraction method provided in this application embodiment is executed by both server 102 and terminal device 101, server 102 and terminal device 101 may also be the same device. That is, server 102 and terminal device 101 may be different functional modules of the same device, or virtual devices virtualized from the same physical device.

[0034] In this embodiment, the terminal device 101 and the server 102 can communicate directly or indirectly through one or more networks. The network can be a wired network or a wireless network; for example, the wireless network can be a mobile cellular network or a Wireless-Fidelity (WIFI) network, or other possible networks. This embodiment does not limit the types of networks used.

[0035] based on Figure 1 The system architecture diagram shown in this application illustrates the flow of an information extraction method, as provided in this embodiment. Figure 2As shown, the process of this method is executed by a computer device, which can be... Figure 1 The terminal device 101 and / or server 102 shown include the following steps: Step S201: Obtain the original data of the target object. The original data includes at least one static attribute and at least one dynamic attribute of the target object.

[0036] Specifically, the raw data is typically heterogeneous data in semi-structured formats such as JSON. Static attributes retain their values ​​throughout runtime and are usually used to describe fixed characteristics of the target object. Examples include "Occupation: Corporate Executive," "Credit Rating: Good," and "Residential Address: XX City, XX District."

[0037] Dynamic properties change their values ​​during program execution and are often used to represent a certain state of an object, such as a user's loan records.

[0038] Step S202: Construct a global hierarchical attribute graph based on the original data. The global hierarchical attribute graph includes a root node and multiple non-root nodes. The root node represents the target object, and each non-root node represents a static or dynamic attribute. Each non-root node is directly or indirectly connected to the root node. Specifically, the target object is taken as the root node, serving as the aggregation point for all information; the static and dynamic attributes of the target object are taken as non-root nodes, directly or indirectly connected to the root node, forming a directed acyclic graph with the target object as the root. This directed acyclic graph represents the global hierarchical attribute graph of the target object.

[0039] For example, let's take the scenario of "a corporate executive with a good credit record making multiple small-amount game top-ups frequently late at night on weekdays" as an example.

[0040] In the global hierarchical attribute graph, the root node represents senior executives of the company; non-root nodes are used to represent dynamic attributes such as "Occupation: Senior Executive", "Credit Rating: Good", "Income Level: High"; and dynamic attributes such as "Transaction 1 (Type: Game Recharge, Amount: 10 Yuan, Time: 23:05)".

[0041] In some embodiments, the non-root nodes in the global hierarchical attribute graph include: static nodes representing static attributes and dynamic nodes representing dynamic attributes; the connecting edges between static nodes represent the hierarchical relationship between static attributes, and the connecting edges between dynamic nodes represent the temporal sequence between dynamic attributes.

[0042] Specifically, directed links are constructed based on the relationships between attributes, resulting in two types: static links and dynamic links. Static links are established between static nodes to reflect the inherent hierarchy or parent-child relationships within the data, creating directed edges from child nodes to parent nodes. For example, directed edges are created from the three static nodes "Occupation," "Credit Rating," and "Income Level" to the root node "User." Static nodes without hierarchical or parent-child relationships do not have links; that is, no directed edges are established between the three static nodes "Occupation," "Credit Rating," and "Income Level."

[0043] Dynamic links are established only between dynamic nodes with the same attribute to capture temporal relationships. A dynamic link points from an earlier record node to the next adjacent record node. Dynamic links are usually represented by a directed edge. For example, a directed edge is established from dynamic node "Record 1" to dynamic node "Record 2", and a directed edge is established from dynamic node "Record 2" to dynamic node "Record 3".

[0044] Step S203: Map the global hierarchical attribute graph to a unified embedding space, and aggregate the attributes represented by multiple non-root nodes to the root node in the embedding space to obtain basic representation information. The embedding space is a low-dimensional, dense real vector space used to capture the semantic features of the original data. Through embedding techniques, discrete symbols (such as words, sentences, images, etc.) are mapped to mathematical representations in a continuous vector space, making semantically similar entities closer together in this space.

[0045] Basic representation information refers to the most basic numerical form that can be computed by the model to represent the original input (such as text, images, categories, etc.) in data processing and machine learning models. It is a prerequisite for subsequent feature extraction and semantic understanding.

[0046] In some embodiments, for each node in the global hierarchical attribute graph, the text description contained in the node is encoded into text features to obtain a corresponding text feature vector; based on the obtained multiple text feature vectors, an initial node feature matrix corresponding to the global hierarchical attribute graph is constructed; and the initial node feature matrix is ​​mapped to a unified embedding space.

[0047] Specifically, each node in the global hierarchical attribute graph includes: a textual description of the entity represented by the node, where the entity can be any of the following: target object, static attribute, and dynamic attribute; Text features are initialized for all nodes in the global hierarchical attribute graph. For leaf nodes, the original text is "attribute name: attribute value"; for intermediate nodes, it is "attribute name: [attribute identifier]"; and for the root node, it is a special "[root identifier]". Then, a pre-trained text encoder (e.g., the Sentence-BERT model) is used to uniformly encode these text inputs into initial text feature vectors, ultimately forming the initial node feature matrix. , where k is the feature dimension.

[0048] The obtained initial node feature matrix X is transformed by a learnable linear transformation matrix. Mapping to a higher-dimensional embedding space yields the embedding matrix. H = X·W .

[0049] For example, first, add an original text description to each node in the global hierarchical attribute graph. For the root node, the text description is: "[User Root Node]". For static nodes, such as the node representing occupation, the text description is: "Occupation: Corporate Executive"; the node representing credit rating is: "Credit Rating: Good". For dynamic nodes, use the same format, for example, the node representing the first transaction is: "Transaction Type: Game Recharge, Amount: 10 Yuan".

[0050] Next, a pre-trained text encoder (e.g., the Sentence-BERT model) is used to encode the above text content into a vector in a high-dimensional space, which is the text feature vector of the text.

[0051] Finally, a feature matrix is ​​constructed using the text feature vectors obtained above. The feature vectors encoded from the text information of all nodes in the graph are integrated (e.g., stacked row by row) to form the initial node feature matrix. , where N is the total number of nodes and k is each feature dimension.

[0052] In some embodiments, multiple leaf nodes among multiple non-root nodes are determined; in the embedding space, information is aggregated from each leaf node to the root node to obtain basic representation information.

[0053] Among them, the multiple leaf nodes include: static leaf nodes representing static attributes and dynamic leaf nodes representing dynamic attributes.

[0054] In some embodiments, the embedding space includes a static aggregation subspace and a dynamic aggregation subspace; the static aggregation subspace and the dynamic aggregation subspace are orthogonal. Static information aggregation is performed along each static leaf node towards the root node in the static aggregation subspace, and dynamic information aggregation is performed along each dynamic leaf node towards the root node in the dynamic aggregation subspace to obtain basic representation information.

[0055] Specifically, two learnable linear transformation matrices are defined. and , are used to define static aggregate subspaces and dynamic aggregate subspaces, respectively.

[0056] The static aggregation subspace can be understood as a part specifically used for interpreting and learning static attributes. When information flows through static links (e.g., from the "degree" node to the "education experience" node), its feature transformation will mainly be carried out through this subspace.

[0057] The dynamic aggregation subspace can be viewed as a part specifically designed to capture and understand temporal relationships. When information flows through dynamic links (e.g., from one transaction record to the next), its feature transformation is mainly accomplished through this dynamic subspace.

[0058] All static links together constitute the static initial node feature matrix. (N is the total number of nodes), where if the node i It is a node j The sub-attribute, then Otherwise, it is 0; all dynamic links constitute the dynamic initial node feature matrix. (N is the total number of nodes), where if the node i It is a node j The sub-attribute, then Otherwise, it is 0.

[0059] For example, such as Figure 3 As shown, the goal is to obtain raw data (JSON) in a semi-structured format from a user, which has static and dynamic attributes. The static attributes include: occupation, permanent address (city, district), and credit rating. The dynamic attributes include: transaction records (record 1, record 2, record 3, etc.).

[0060] Construct a global hierarchical attribute graph for this user, with the following nodes: "User (root node)" :"Profession", "Credit rating" "Permanent residential address" :"City", :"district", "Transaction Records" (Management of Dynamic Sequences) "Record 1" (dynamic node) "Record 2" (Dynamic Node) “Record 3” (dynamic node), where, As the parent node of a dynamic sequence, it summarizes the characteristics of the dynamic sequence. The earliest record in the dynamic sequence is connected to this node through static linking.

[0061] Therefore, for the static initial node feature matrix In general, items with a value of 1 (the rest are 0) include: (Link: → ), (Link: → ), (Link: → ), (Link: → ), (Link: → ), (Link: → ), (Link: → ).

[0062] For the dynamic initial node feature matrix In general, items with a value of 1 (the rest are 0) include: (Link: → ), (Link: → ).

[0063] The model adopts a bottom-up approach, aggregating information layer by layer from the leaf nodes towards the root node. For any non-leaf node in the global hierarchical attribute graph... Its updated representation It is based on one or more of its child nodes Updated representation The calculation is derived from the specific calculation method. and The types of links between them fall into the following two categories: Static aggregation is performed in the static aggregation subspace: if child nodes Linked via static link Its information aggregation formula is shown in the following formula (1): (1) Dynamic reasoning in a dynamic aggregation subspace: if child nodes Connected via dynamic link (That is, they are adjacent records with the same dynamic attribute), then a gated recurrent unit (GRU) network is used for time series modeling, and its information aggregation formula is shown in the following formula (2): (2) In formulas (1) and (2), Represents the current node The initial node representation before the update; Represents the child nodes of the current node (In static aggregation) or the node at the previous time step (In dynamic reasoning) the updated (i.e. processed) node representations; and Let be the linear transformation matrices for the static and dynamic aggregate subspaces, respectively. and They are and The false reversal; and (For static aggregation only) are learnable attention weights used for aggregation. Its own information and the information it comes from its child nodes Information; n (used only for static aggregation) represents a node. The total number of static child nodes it possesses; GRU (for dynamic reasoning only) stands for Gated Cyclic Unit. (For dynamic reasoning only) indicates that GRU is from the previous time step (node) The hidden state passed over.

[0064] When the information propagation finally converges at the root node r, its updated representation is obtained. This representation is the basic representation information of the target object, denoted as... = .

[0065] Specifically, the latest record in the dynamic records is linked to a node that summarizes the attributes of the dynamic nodes through static links. The root node uses the static aggregation formula (1) mentioned above to calculate the final output representation. For example, the latest record node among multiple loan records is linked to the loan record node, and then this node is linked to the root node through static links, thereby ensuring that all nodes under the root node are connected to it through static links.

[0066] In the embodiments of this application, static aggregation is mainly in Performed within the subspace (primary item), while preserving the data in the subspace. Components (secondary terms) within a subspace are used to maintain the uniformity of representation. Dynamic aggregation mainly occurs in... Performed within the subspace, while preserving the data in... Components within a subspace. In this way, regardless of whether the information path is static or dynamic, the representations of all nodes always reside in the same unified semantic space.

[0067] In some embodiments, the orthogonality between the static aggregation subspace and the dynamic aggregation subspace is obtained through orthogonal loss function constraints, which are used to minimize the norm of the inner product between the static aggregation subspace and the dynamic aggregation subspace.

[0068] To ensure that the two information processing paths are independent of each other at the parameter level, an orthogonality loss term is added to the final loss function, as shown in the following formula (3): (3) in, This represents the square of the F norm of a matrix.

[0069] This loss term is minimized during model training, thereby forcing the basis vectors of the two subspaces to be orthogonal. If the orthogonality between the two subspaces is not strong enough, then... The value of is relatively large, thus generating a "penalty" signal. In order to minimize the loss, the model will autonomously adjust the parameters of the two subspaces until they are mathematically approximately orthogonal, thereby forcing the basis vectors of the two subspaces to be independent of each other.

[0070] Wherein, the model's total loss function l The definition is shown in the following formula (4): (4) in, It is the model's total loss function; It is the loss function of the downstream task. For example, the mean absolute error used for the regression task after the cross-entropy loss of the classification task is calculated by comparing the predicted output of the model with the "true label" of the downstream task. It is an orthogonality loss, which calculates the static subspace matrix. With dynamic subspace matrix The square of the F-norm of the inner product between them is used to measure the degree of orthogonality between them; γ is a tradeoff hyperparameter used to balance the two losses, which... It is minimized during model training, therefore The item will also be minimized.

[0071] Step S204: A selective aggregation mechanism is used to perform secondary information aggregation on the global hierarchical attribute graph to obtain enhanced representation information; Selective semantic aggregation is a technique that selectively focuses on and aggregates key semantic information in an input sequence using methods such as attention mechanisms. This technique assigns learnable weights to different parts of the input sequence, highlighting important semantic units and suppressing irrelevant or redundant information, so as to achieve efficient modeling of the context.

[0072] In some embodiments, the attention weight of each attribute represented by a non-root node relative to the target object is determined based on an attention score calculation function that includes a truncation hyperparameter, wherein the truncation hyperparameter is used to truncate the information flow of non-root nodes that are irrelevant to the downstream classification task.

[0073] Specifically, first calculate the distance from the neighboring nodes to the center node. attention weights Specifically, as shown in the following formula (5): (5) in, It is determined by a learnable scoring function. The calculated attention score, yes The set of neighboring nodes; the key difference from the standard attention mechanism is the addition of a positive term in the denominator. ,in It is a learnable or preset hyperparameter. The item serves as a "cut-off" or "gating" mechanism.

[0074] By learning the scoring function ψ and hyperparameters during training The model can autonomously and selectively "shut down" certain information propagation paths. For example, if the model believes that information propagation from node... arrive If the information is noise or irrelevant to the task, it can learn a very small attention score. This makes the corresponding weights Approaching zero.

[0075] by Figure 3 Taking the middle node as an example, when the model calculates the root node... When representing information for the "user" node: the central node yes Neighbor nodes for ("Profession"), ("Credit rating") ("Permanent Residential Address") ("Transaction records"); the model will calculate → , → Attention weights for links.

[0076] In the embodiments of this application, by adjusting λ and ψ, the mechanism can functionally simulate arbitrary node selection and edge pruning, which is equivalent to information propagation on an optimal, task-related superstructure, and efficiently captures high-order interaction information across static and dynamic attributes.

[0077] In some embodiments, the attention weight of the attribute represented by each non-root node relative to the target object is determined; the global hierarchical attribute graph is mapped to a unified embedding space, and the attributes represented by the multiple non-root nodes are aggregated to the root node according to the attention weights corresponding to the multiple non-root nodes in the embedding space to obtain enhanced representation information.

[0078] For the global hierarchical attribute graph (by...) Any non-leaf "center node" in the definition. The model will use the obtained attention weights. For all its "neighboring nodes" (Right now The basic representations of the nodes are weighted and summed to calculate the total. New Representations in the Secondary Information Aggregation Stage Specifically, as shown in the following formula (6): (6) in, It is a newly defined learnable linear transformation matrix for the SSAgg secondary aggregation stage.

[0079] After another bottom-up information propagation through the SSAgg mechanism, when the information finally converges to the root node r, a new representation is obtained. This representation is the enhanced representation information of the target object, denoted as... = .

[0080] In this embodiment, the SSAgg mechanism is used to simulate arbitrary node selection and edge pruning, which is equivalent to information propagation on an optimal, task-related superstructure, and efficiently captures complex high-order interaction information between static and dynamic attributes.

[0081] Step S205: Based on the basic representation information and the enhanced representation information, obtain the target representation information of the target object.

[0082] Specifically, the obtained basic representation information and enhance representation information By combining the data, we can obtain the target representation information of the target object. Specifically, as shown in the following formula (7): (7) In some embodiments, a downstream classification task is performed on the target object based on the target representation information to obtain the classification result.

[0083] The target representation information is input into a simple fully connected layer classifier or regressor to perform downstream classification tasks (such as user churn prediction, credit risk assessment, etc.).

[0084] In this embodiment, a global hierarchical attribute graph is constructed based on heterogeneous raw data containing static and dynamic attributes to obtain the initial node representation. The initial node representation is mapped to a unified embedding space, and a basic representation that can comprehensively reflect all attributes of the entity is obtained at the root node. Then, the complex interaction relationship between static and dynamic attributes on downstream tasks is further captured, and an enhanced entity representation is obtained at the root node. Finally, the basic representation information and the enhanced representation information are combined to form the final target representation information for various downstream tasks.

[0085] This application eliminates the semantic gap by uniformly modeling the static and dynamic attributes of data in a shared representation space, enabling the entire feature extraction process to be automated and greatly reducing the cost of manual design. Through a novel attention weighting mechanism, it effectively discovers and utilizes the implicit, task-beneficial high-order interaction relationships between static and dynamic attributes, significantly improving the performance of the model on complex decision-making tasks.

[0086] Based on the same technical concept, this application provides a schematic diagram of the structure of an information extraction device, such as... Figure 4 As shown, the information extraction device 400 includes: The composition module 401 is used to acquire the original data of the target object, the original data including: at least one static attribute and at least one dynamic attribute of the target object; A global hierarchical attribute graph is constructed based on the original data. The global hierarchical attribute graph includes a root node and multiple non-root nodes. The root node represents the target object, and each non-root node represents the static attribute or the dynamic attribute. Each non-root node is directly or indirectly connected to the root node. The information acquisition module 402 is used to map the global hierarchical attribute graph to a unified embedding space, and to aggregate the attributes represented by the multiple non-root nodes to the root node in the embedding space to obtain basic representation information. A selective aggregation mechanism is used to perform secondary information aggregation on the global hierarchical attribute graph to obtain enhanced representation information. Based on the basic representation information and the enhanced representation information, the target representation information of the target object is obtained.

[0087] Optionally, the non-root nodes in the global hierarchical attribute graph include: static nodes representing the static attributes and dynamic nodes representing the dynamic attributes; The connecting edges between static nodes represent the hierarchical relationship between static attributes, and the connecting edges between dynamic nodes represent the temporal sequence between dynamic attributes.

[0088] Optionally, each node in the global hierarchical attribute graph includes: a text description of the entity represented by the node, wherein the entity is any one of the following: the target object, the static attribute, and the dynamic attribute; The composition module 401 is specifically used for: For each node in the global hierarchical attribute graph, the text description contained in the node is encoded into text features to obtain the corresponding text feature vector; Based on the obtained multiple text feature vectors, construct the initial node feature matrix corresponding to the global hierarchical attribute graph; The initial node feature matrix is ​​mapped to a unified embedding space.

[0089] Optionally, the information acquisition module 402 is specifically used for: Identify multiple leaf nodes among the multiple non-root nodes; In the embedding space, information is aggregated from each leaf node to the root node to obtain basic representation information.

[0090] Optionally, the embedding space includes: a static aggregation subspace and a dynamic aggregation subspace; the static aggregation subspace and the dynamic aggregation subspace are orthogonal; the plurality of leaf nodes include: static leaf nodes representing the static attributes and dynamic leaf nodes representing the dynamic attributes; The information acquisition module 402 is specifically used for: In the static aggregation subspace, information is statically aggregated from each of the static leaf nodes to the root node, and in the dynamic aggregation subspace, information is dynamically aggregated from each of the dynamic leaf nodes to the root node to obtain basic representation information.

[0091] Optionally, the orthogonality between the static aggregation subspace and the dynamic aggregation subspace is obtained through orthogonal loss function constraints, wherein the orthogonal loss function is used to minimize the norm of the inner product between the static aggregation subspace and the dynamic aggregation subspace.

[0092] Optionally, the information acquisition module 402 is specifically used for: Determine the attention weight of the attribute represented by each non-root node relative to the target object; The global hierarchical attribute graph is mapped to a unified embedding space, and the attributes represented by the multiple non-root nodes are aggregated to the root node according to the attention weights corresponding to the multiple non-root nodes in the embedding space to obtain enhanced representation information.

[0093] Optionally, the information acquisition module 402 is specifically used for: Based on the attention score calculation function that includes a truncation hyperparameter, the attention weight of each attribute represented by a non-root node relative to the target object is determined, wherein the truncation hyperparameter is used to truncate the information flow of non-root nodes that are irrelevant to the downstream classification task.

[0094] Optionally, the information acquisition module 402 is further configured to: Based on the target representation information, a downstream classification task is performed on the target object to obtain the classification result.

[0095] This application provides a method for extracting information from unified static and dynamic attributes. A global hierarchical attribute graph is constructed based on heterogeneous raw data containing both static and dynamic attributes to obtain initial node representations. These initial node representations are then mapped to a unified embedding space, resulting in a basic representation at the root node that comprehensively reflects all entity attributes. Next, the complex interactions between static and dynamic attributes for downstream tasks are captured, leading to an enhanced entity representation at the root node. Finally, the basic and enhanced representation information are combined to form the final target representation information for various downstream tasks. This application eliminates the semantic gap by uniformly modeling the static and dynamic attributes of data within a shared representation space, automating the entire feature extraction process and significantly reducing manual design costs. Through a novel attention weighting mechanism, it effectively discovers and utilizes implicit, task-beneficial high-order interactions between static and dynamic attributes, significantly improving model performance on complex decision-making tasks.

[0096] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0097] Based on the same technical concept, embodiments of this application provide a computer device, which can be... Figure 1 The terminal devices and / or servers shown, such as Figure 5 As shown, it includes at least one processor 501 and a memory 502 connected to at least one processor. In this embodiment, the specific connection medium between the processor 501 and the memory 502 is not limited. Figure 5 Taking the connection between processor 501 and memory 502 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0098] In this embodiment of the application, the memory 502 stores instructions that can be executed by at least one processor 501. By executing the instructions stored in the memory 502, at least one processor 501 can perform the steps of the above-described information extraction method.

[0099] The processor 501 is the control center of the computer device. It can connect to various parts of the computer device using various interfaces and lines, and realizes virtual chip production by running or executing instructions stored in the memory 502 and calling data stored in the memory 502. Optionally, the processor 501 may include one or more processing units. The processor 501 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 501. In some embodiments, the processor 501 and the memory 502 may be implemented on the same chip; in some embodiments, they may also be implemented on separate chips.

[0100] Processor 501 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0101] Memory 502, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 502 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory 502 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer device, but is not limited thereto. In the embodiments of this application, memory 502 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0102] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the above-described information extraction method.

[0103] Based on the same inventive concept, this application provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the above-described information extraction method.

[0104] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0105] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer apparatus or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0106] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer device or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0107] These computer program instructions may also be loaded onto a computer device or other programmable data processing equipment to cause a series of operational steps to be performed on the computer device or other programmable equipment to produce a process implemented by the computer device, thereby providing instructions that execute on the computer device or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0108] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0109] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An information extraction method, characterized in that, include: Obtain the original data of the target object, wherein the original data includes at least one static attribute and at least one dynamic attribute of the target object; A global hierarchical attribute graph is constructed based on the original data. The global hierarchical attribute graph includes a root node and multiple non-root nodes. The root node represents the target object, and each non-root node represents the static attribute or the dynamic attribute. Each non-root node is directly or indirectly connected to the root node. The global hierarchical attribute graph is mapped to a unified embedding space, and the attributes represented by the multiple non-root nodes are aggregated to the root node in the embedding space to obtain basic representation information. A selective aggregation mechanism is used to perform secondary information aggregation on the global hierarchical attribute graph to obtain enhanced representation information. Based on the basic representation information and the enhanced representation information, the target representation information of the target object is obtained.

2. The method as described in claim 1, characterized in that, The non-root nodes in the global hierarchical attribute graph include: static nodes representing the static attributes and dynamic nodes representing the dynamic attributes; The connecting edges between static nodes represent the hierarchical relationship between static attributes, and the connecting edges between dynamic nodes represent the temporal sequence between dynamic attributes.

3. The method as described in claim 1, characterized in that, Each node in the global hierarchical attribute graph includes: a text description of the entity represented by the node, wherein the entity is any one of the following: the target object, the static attribute, and the dynamic attribute; The step of mapping the global hierarchical attribute graph to a unified embedding space includes: For each node in the global hierarchical attribute graph, the text description contained in the node is encoded into text features to obtain the corresponding text feature vector; Based on the obtained multiple text feature vectors, construct the initial node feature matrix corresponding to the global hierarchical attribute graph; The initial node feature matrix is ​​mapped to a unified embedding space.

4. The method as described in claim 1, characterized in that, The step of aggregating the attributes represented by the plurality of non-root nodes in the embedding space to the root node to obtain basic representation information includes: Identify multiple leaf nodes among the multiple non-root nodes; In the embedding space, information is aggregated from each leaf node to the root node to obtain basic representation information.

5. The method as described in claim 4, characterized in that, The embedding space includes: a static aggregation subspace and a dynamic aggregation subspace; the static aggregation subspace and the dynamic aggregation subspace are orthogonal; the plurality of leaf nodes include: static leaf nodes representing the static attributes and dynamic leaf nodes representing the dynamic attributes; In the embedded space, information is aggregated from each leaf node to the root node to obtain basic representation information, including: In the static aggregation subspace, information is statically aggregated from each of the static leaf nodes to the root node, and in the dynamic aggregation subspace, information is dynamically aggregated from each of the dynamic leaf nodes to the root node to obtain basic representation information.

6. The method as described in claim 5, characterized in that, The orthogonality between the static aggregation subspace and the dynamic aggregation subspace is obtained through orthogonal loss function constraints, which is used to minimize the norm of the inner product between the static aggregation subspace and the dynamic aggregation subspace.

7. The method as described in any one of claims 1 to 6, characterized in that, The selective aggregation mechanism is used to perform secondary information aggregation on the global hierarchical attribute graph to obtain enhanced representation information, including: Determine the attention weight of the attribute represented by each non-root node relative to the target object; The global hierarchical attribute graph is mapped to a unified embedding space, and the attributes represented by the multiple non-root nodes are aggregated to the root node according to the attention weights corresponding to the multiple non-root nodes in the embedding space to obtain enhanced representation information.

8. The method as described in claim 7, characterized in that, Determining the attention weight of each attribute represented by a non-root node relative to the target object includes: Based on the attention score calculation function that includes a truncation hyperparameter, the attention weight of each attribute represented by a non-root node relative to the target object is determined, wherein the truncation hyperparameter is used to truncate the information flow of non-root nodes that are irrelevant to the downstream classification task.

9. The method according to any one of claims 1 to 6, characterized in that, After obtaining the target representation information of the target object based on the basic representation information and the enhanced representation information, the method further includes: Based on the target representation information, a downstream classification task is performed on the target object to obtain the classification result.

10. An information extraction device, characterized in that, include: A mapping module is used to acquire the original data of a target object, the original data including: at least one static attribute and at least one dynamic attribute of the target object; A global hierarchical attribute graph is constructed based on the original data. The global hierarchical attribute graph includes a root node and multiple non-root nodes. The root node represents the target object, and each non-root node represents the static attribute or the dynamic attribute. Each non-root node is directly or indirectly connected to the root node. The information acquisition module is used to map the global hierarchical attribute graph to a unified embedding space, and to aggregate the attributes represented by the multiple non-root nodes to the root node in the embedding space to obtain basic representation information. A selective aggregation mechanism is used to perform secondary information aggregation on the global hierarchical attribute graph to obtain enhanced representation information. Based on the basic representation information and the enhanced representation information, the target representation information of the target object is obtained.

Images