Knowledge graph construction methods, devices and terminal equipment

Abstract

This application relates to the field of data processing technology and provides a method, apparatus, and terminal device for constructing a knowledge graph. The method includes: acquiring knowledge data and encoding feature vectors of the knowledge data using an embedding layer, where the feature vectors are identifiers after encoding the knowledge data; extracting first-sequence features from the feature vectors using a first-sequence feature extraction model and extracting second-sequence features from the feature vectors using a second-sequence feature extraction model; concatenating the first-sequence features and the second-sequence features to obtain third-sequence features; and selecting third-sequence features that conform to the expression logic using a CRF layer to obtain a triple sequence. The triple sequence consists of entity-relation-entity, and multiple triple sequences constitute a complete knowledge graph. This application can reduce the influence of interfering entity information, enhance the feature extraction capability of the knowledge graph, thereby completing entity identification and relation extraction for audit rules, and ultimately constructing a knowledge graph that conforms to financial audit rules.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to a knowledge graph construction method, apparatus and terminal equipment. Background Technology

[0002] As an important component of the economic supervision system, financial auditing involves business rules, regulations, and financial statements. The abstract semantic data in the audit rules can be transformed into visualized knowledge graph information by extracting the entity relationships. This not only facilitates auditors' learning, analysis, and judgment of problem types, frequencies, and manifestations, but also enables the reasonable integration and cross-domain analysis of audit information, providing valuable references for digital auditing.

[0003] Traditional knowledge graph mining and construction typically employs either feature-based or kernel-function-based model building. Feature-based models often require designing a large number of syntactic, semantic, and syntactic features before feeding them into a classifier like a Support Vector Machine (SVM) for classification, consuming significant time and effort in constructing suitable features. While kernel-function-based models avoid constructing numerous features, designing and selecting appropriate kernel functions remains extremely difficult, resulting in low accuracy and poor analytical capabilities in the constructed knowledge graph.

[0004] Therefore, there is an urgent need for a knowledge graph construction method to reduce the influence of interfering entity information, enhance the feature extraction capability of the knowledge graph, thereby completing entity identification and relation extraction for audit rules, and then constructing a knowledge graph that conforms to financial audit rules. Summary of the Invention

[0005] To overcome the problems existing in related technologies, embodiments of this application provide a knowledge graph construction method, apparatus and terminal device to reduce the influence of interfering entity information, enhance the feature extraction capability of knowledge graph, thereby completing entity identification and relationship extraction of audit rules, and then constructing a knowledge graph that conforms to financial audit rules.

[0006] This application is achieved through the following technical solution:

[0007] In a first aspect, embodiments of this application provide a knowledge graph construction method, comprising: acquiring knowledge data and encoding feature vectors of the knowledge data using an embedding layer, wherein the feature vectors are identifiers after encoding the knowledge data; extracting first sequence features from the feature vectors using a first sequence feature extraction model and extracting second sequence features from the feature vectors using a second sequence feature extraction model; concatenating the first sequence features and the second sequence features to obtain third sequence features; and selecting third sequence features that conform to the expression logic using a CRF layer to obtain a triple sequence, wherein the triple sequence consists of entity-relation-entity, and multiple triple sequences constitute a complete knowledge graph.

[0008] In one possible implementation of the first aspect, the first sequence feature extraction model includes a first BiGRU layer and a second BiGRU layer, which are connected in a stacked manner. Each of the first and second BiGRU layers consists of two layers of GRU units. The first layer of GRU units receives a forward-facing GRU sequence as input, while the second layer receives a backward-facing GRU sequence. The feature vector is first extracted through the first BiGRU layer, and then through the second BiGRU layer, outputting the first sequence features.

[0009] In one possible implementation of the first aspect, the second sequence feature extraction model includes a BiLSTM layer, which is used to extract semantic features from character-level feature vectors; the BiLSTM layer receives feature vectors and outputs character-level feature vectors.

[0010] In one possible implementation of the first aspect, the second sequence feature extraction model further includes a Self-Attention layer. This Self-Attention layer assigns attention weights to character-level feature vectors, where each attention weight represents the importance of each character-level feature vector to the knowledge graph construction. The Self-Attention layer receives feature extraction from the BiLSTM layer's output of the feature vectors, and outputs the second sequence features.

[0011] In one possible implementation of the first aspect, concatenating the first sequence features and the second sequence features to obtain the third sequence features includes: using the Concat function to concatenate the first sequence features and the second sequence features to obtain the third sequence features, wherein the expression for the third sequence features is:

[0012] H(x) = concat[F(x) + G(x)]

[0013] In the formula, H(x) represents the third sequence feature, F(x) represents the first sequence feature, and G(x) represents the second sequence feature.

[0014] In one possible implementation of the first aspect, before using the CRF layer to select the third sequence features that conform to the expression logic, the method includes: using a fully connected layer to encode the third sequence features and integrating the multiple dimensions of the third sequence features into one dimension.

[0015] In one possible implementation of the first aspect, the knowledge data includes image data and text data; after acquiring the knowledge data, before encoding the feature vector of the knowledge data using an embedding layer, the graph construction method also includes: using an OCR recognition model to recognize the text in the image data; using a Jieba word segmentation model to divide the text data into words according to Chinese grammar; and constructing a triplet feature template, the form of which is entity, relation, and entity form.

[0016] Secondly, embodiments of this application provide a knowledge graph construction apparatus, comprising: a data acquisition module for acquiring knowledge data and encoding feature vectors of the knowledge data using an embedding layer; a feature extraction module for extracting first sequence features from the feature vectors using a first sequence feature extraction model and extracting second sequence features from the feature vectors using a second sequence feature extraction model; a feature concatenation module for concatenating the first sequence features and the second sequence features to obtain third sequence features; and a decoding output module for selecting third sequence features that conform to the expression logic using a CRF layer to obtain a triple sequence, wherein the triple sequence is composed of entity-relation-entity, and multiple triple sequences constitute a complete knowledge graph.

[0017] Thirdly, embodiments of this application provide a terminal device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the knowledge graph construction method as described in any of the first aspects.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the knowledge graph construction method as described in any of the first aspects.

[0019] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the knowledge graph construction method described in any one of the first aspects.

[0020] The beneficial effects of the embodiments of this application compared with the prior art are:

[0021] In this embodiment, a first sequence feature is extracted from the feature vector by a first sequence feature extraction model, and a second sequence feature is extracted from the feature vector by a second sequence feature extraction model. The first sequence feature and the second sequence feature are then concatenated to obtain a third sequence feature. This allows the first sequence feature extraction model and the second sequence feature extraction model to extract entities from the same feature vector, thereby identifying more entity information and achieving higher accuracy in relation identification.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating a knowledge graph construction method provided in an embodiment of this application;

[0025] Figure 2 This is a schematic diagram of the structure of a first sequence feature extraction model provided in an embodiment of this application;

[0026] Figure 3 This is a schematic diagram of the structure of a second sequence feature extraction model provided in an embodiment of this application;

[0027] Figure 4 This is a schematic diagram of the splicing of the first sequence feature and the second sequence feature provided in an embodiment of this application;

[0028] Figure 5 This is a partial financial audit rule knowledge graph provided in one embodiment of this application;

[0029] Figure 6 This is a schematic diagram of the knowledge graph construction apparatus provided in the embodiments of this application;

[0030] Figure 7 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0031] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0032] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0033] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0034] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0035] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0036] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0037] Existing knowledge graph construction methods typically employ entity co-occurrence, feature vector-based model building, or kernel function-based model building, resulting in a dual decline in accuracy and efficiency. The specific drawbacks of these three methods are as follows:

[0038] (1) Entity co-occurrence refers to the assumption that when two entities appear at the same time, they are related. Among them, the co-occurrence frequency scoring is used to eliminate the situation of misidentification caused by occasional co-occurrence. This scheme can achieve a high recall rate, but the accuracy is not high and it cannot show the latent relationship.

[0039] (2) The key to the model based on feature vectors is to construct feature vectors. The more feature values ​​there are, the higher the accuracy of the extracted relationship, but the greater the computational complexity and the weaker the feature generalization ability. It is necessary to redesign the corresponding features according to different fields.

[0040] (3) Although models built based on kernel functions do not require building large feature engineering, it is very difficult to design and select appropriate kernel functions.

[0041] To address the aforementioned issues, this application provides a knowledge graph construction method. Based on traditional recurrent neural networks, a residual recurrent neural network entity learning model incorporating an attention mechanism is studied. A recurrent neural network with residual properties is designed for knowledge graph construction, which can reduce the impact of negative samples on the network and increase its entity extraction capability. Specifically, an attention mechanism is introduced into the residual part to capture dependencies over long time steps, reducing interference from non-entity information and enhancing the network's feature extraction capability. This enables entity recognition and relation extraction for audit rules, thereby constructing the knowledge graph.

[0042] This application uses financial auditing as an example to describe in detail the technical solution for which this application seeks protection.

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described below are for illustrative purposes only and are not intended to limit the scope of this application.

[0044] Figure 1 This is a flowchart illustrating a knowledge graph construction method provided in an embodiment of this application, with reference to... Figure 1 This method can be implemented by executing steps 101 to 104, as detailed below:

[0045] In step 101, knowledge data is acquired, and the feature vector of the knowledge data is encoded using an embedding layer.

[0046] In some embodiments, relevant knowledge data can be obtained from multiple knowledge bases of financial auditing.

[0047] For example, a knowledge base could be relevant financial audit documents online, relevant laws and regulations on financial auditing, or internal audit rules and regulations of an organization.

[0048] There are various ways to obtain knowledge data from knowledge bases, such as crawling the web using Python technology or scanning internal documents. This application does not impose further limitations.

[0049] Optionally, the acquired knowledge data may include image data and text data.

[0050] In some embodiments, after acquiring knowledge data, an embedding layer can be used to encode the feature vectors of the knowledge data.

[0051] For example, the embedding layer can be implemented using an embedding layer to encode feature vectors of knowledge data.

[0052] Optionally, the embedding layer may include a token embedding vector, a segment embedding vector, and a position embedding vector. The feature vectors of knowledge data can be encoded by merging the token embedding vector, segment embedding vector, and position embedding vector, a method that preserves the order of Chinese words.

[0053] In step 102, the first sequence feature extraction model is used to extract the first sequence feature from the feature vector, and the second sequence feature extraction model is used to extract the second sequence feature from the feature vector.

[0054] In some embodiments, two feature extraction models are employed: a first sequence feature extraction model and a second sequence feature extraction model. The first sequence feature extraction model is used to extract the first sequence features, i.e., deep sequence features, from the feature vector of the knowledge data, while the second sequence feature extraction model is used to extract the second sequence features, i.e., shallow sequence features, from the feature vector of the knowledge data.

[0055] In a descriptive sense, the so-called deep sequence features are characterized by a greater emphasis on global information, such as semantic information, and are mainly coarse-grained feature information.

[0056] In a descriptive sense, shallow sequence features are characterized by a greater focus on detailed information, such as character-level information, and are mainly fine-grained features.

[0057] In some embodiments, the first sequence feature extraction model may include a first BiGRU layer and a second BiGRU layer. The first BiGRU layer and the second BiGRU layer are connected in a stacked manner. Figure 2 A schematic diagram of the structure of a first sequence feature extraction model provided in an embodiment of this application is shown.

[0058] Reference Figure 2 The feature vector is first extracted through the first BiGRU layer, and then through the second BiGRU layer to extract the first feature sequence.

[0059] Optionally, the first BiGRU layer and the second BiGRU each consist of two layers of GRU units, with the input sequence of the first layer GRU unit being a forward GRU and the input sequence of the second layer GRU unit being a reverse GRU.

[0060] For example, the feature vector is used as the input to the first sequence feature extraction model, and the feature vector x at time t is... t By updating gate Z t and reset door r t Then, based on the state information h from the previous moment... t-1 Go to obtain the current hidden state h t .

[0061] Update Gate Z t The calculation formula is: z t =σ(W z ·[h t-1 ,x t In the formula, at time t, the input sequence X... t With weight W z Multiplication performs a linear transformation, while simultaneously retaining the information h retained within time t-1. t-1 and weight W z The products are multiplied and then linearly transformed. Their sum is then activated using a sigmoid layer. During this process, the update gate determines how much information from past and current time steps should be passed on.

[0062] The formula for resetting the door is: t′=tan(W·[r t * t-1 ,x t In the formula, r t and h t-1 The product of x and x represents the amount of information retained and forgotten before the information is forgotten. After a linear transformation of x, it is activated by a hyperbolic tangent tanh layer. This formula is mainly used to determine the amount of information ignored in the past time.

[0063] Finally, the final memory information at the current moment is obtained. An update gate is used to control the inflow of new information, resulting in the final output. The expression for this calculation is: t = (1 - z) t )* t-1 +z t * t ′.

[0064] After the above calculation, more semantic information is obtained. The calculation expression for the BiGRU output in the last state t is as follows:

[0065]

[0066]

[0067]

[0068] In some embodiments, the second sequence feature extraction model may include a BiLSTM layer, which is used to extract semantic features from character-level feature vectors. Figure 3 A schematic diagram of the structure of a second sequence feature extraction model provided in an embodiment of this application is shown.

[0069] Optionally, the BiLSTM layer receives feature vectors and outputs character-level feature vectors.

[0070] In some embodiments, the second sequence feature extraction model further includes a Self-Attention layer, which is used to assign attention weights to character-level feature vectors. The attention weights characterize the importance of each character-level feature vector to the construction of the knowledge graph.

[0071] In the process of knowledge graph construction, not only is word segmentation and entity extraction based on context important, but sentiment in the text also plays a crucial role. To make the extracted entities more closely resemble the user's true intent, this application introduces an attention mechanism, inputting the character-level feature vectors processed by BiLSTM into the Self-Attention layer.

[0072] In some embodiments, let the character-level feature vector be X = {x1, x2, ..., x...} n}, x i This represents the i-th character, utilizing k in the Self-Attention layer. i with f i Represents the word x i The corresponding key vector. The Self-Attention layer assigns the query vector q to the key vector k corresponding to each word. i By performing similarity analysis, we can obtain the similarity value s for each character.i , then s i This represents the importance of each character. Next, we will obtain s. i After normalization, we can obtain the weight value w for each word. i The SoftMax function is used for normalization. Finally, each weight value w is... i and the corresponding key vector f i Weighted summation yields the text vector Question Attention, which includes the attention mechanism. Its expression is:

[0073]

[0074] The Question Attention mechanism comprises three matrices: Q(Query), K(Key), and V(Value). A linear transformation is applied to the Query, Key, and Value, and the calculation expression is as follows:

[0075] Q′=Q*W j Q

[0076] K′=K*W j K

[0077] V′=V*W j V

[0078] Finally, perform the scaling dot product attention calculation:

[0079]

[0080] In the above formula, the dot product of matrices Q and K is calculated first, and then divided by... Then, the weights of matrix V are obtained using the SoftMax function. To ensure gradient stability, This is for smoothing. This process is repeated multiple times, with the parameters of the linear transformation changing each time, and finally the multiple layers are concatenated. This process gives higher weight to important words, allowing semantic features to focus on key information, strengthening the representation of important semantic information and avoiding interference from irrelevant information.

[0081] In some embodiments, the Self-Attention layer receives the feature extraction of the feature vector from the output of the BiLSTM layer, and the Self-Attention layer outputs the second sequence features.

[0082] In this step, the first sequence feature extraction model and the second sequence feature extraction model share the feature vector of the embedding layer, ensuring that the first sequence feature extraction model and the second sequence feature extraction model can learn from the same data, preparing for subsequent steps and preventing the problem of feature splicing of different data in the output layer due to out-of-order input during neural network training.

[0083] In step 103, the first sequence features and the second sequence features are spliced ​​together to obtain the third sequence features.

[0084] Figure 4 This is a schematic diagram illustrating the splicing of a first sequence feature and a second sequence feature provided in an embodiment of this application. (Refer to...) Figure 4 .

[0085] In some embodiments, the Concat function can be used to concatenate the first sequence features and the second sequence features to obtain the third sequence features. The expression for the third sequence features is as follows:

[0086] H(x) = concat[F(x) + G(x)]

[0087] In the formula, H(x) represents the third sequence feature, F(x) represents the first sequence feature, and G(x) represents the second sequence feature.

[0088] This step, by concatenating the first sequence features and the second sequence features, can effectively preserve the sequence features obtained from the training of the two neural networks, avoiding the problem of insufficient feature capture capability of recurrent neural networks for long text data.

[0089] In step 104, the CRF layer is used to select the third sequence features that conform to the expression logic to obtain the triplet sequence.

[0090] The third sequence feature output in step 103 is still labeled haphazardly, so a CRF layer is needed to select the best label sequence, that is, to sort all the entities.

[0091] In some embodiments, for a given sequence of observations x = {x1, x2, ..., x...} n The sequence y = {y1, y2, ..., y} can be predicted. n The expressions for scoring sequences x and y are as follows:

[0092]

[0093] In the formula, A is the transition matrix, A i,j For the transition from label i to label j, P is an n×k matrix. i,j It is the score of the j-th tag of the i-th word.

[0094] The probability of obtaining sequence y using the Softmax function is:

[0095]

[0096] In the formula, Y is the true labeled value. x It is the set of all possible tags.

[0097] Therefore, the likelihood probability of maximizing the correct label sequence during training is:

[0098]

[0099] Finally, the Viterbi algorithm is used to obtain the optimal predicted label sequence:

[0100] y * =argmax(S(x,y))

[0101] In some embodiments, a CRF layer is used to select third sequence features that conform to the expression logic to obtain a triple sequence. The triple sequence includes at least one triple, and each triple can represent a single knowledge graph.

[0102] For example, in the process of decomposing audit suspicious text into <entity, relation, entity> triples, different types of reimbursement topics, fund usage and other events are involved. Each triple represents a single audit rule, forming a structured representation of the entire audit rule base.

[0103] In some embodiments, based on Figure 1 In the embodiments shown, before selecting the third sequence features that conform to the expression logic using the CRF layer, the above knowledge graph construction method may further include:

[0104] A fully connected layer is used to encode the third sequence feature, integrating multiple dimensions of the third sequence feature into one dimension. This reduces the impact of the position of the third sequence feature on the classification result and improves the robustness of the entire neural network.

[0105] In some embodiments, after acquiring the knowledge data and before encoding the feature vectors of the knowledge data using an embedding layer, the knowledge graph construction method may further include preprocessing the knowledge data. Preprocessing may involve the following operations:

[0106] Optionally, OCR can be used to recognize the text contained in the model image data;

[0107] The Jieba word segmentation model was used to segment text data into words according to Chinese grammar.

[0108] Construct a triplet feature template, which takes the form of <entity, relation, entity>.

[0109] To verify the beneficial effects and credibility of the technical solution of this application, a set of comparative cases are also provided. Compared with traditional knowledge graphs based on recurrent neural networks and relationship detection algorithms, the knowledge graph construction method claimed in this application identifies more information and has higher accuracy in relationship identification. Table 1 shows the amount of information identified in the knowledge graphs constructed by different algorithms, and Table 2 shows the accuracy of relationship identification for different numbers of financial statements.

[0110] Table 1 Information Recognition Quantity

[0111]

[0112] Table 2. Accuracy of Financial Audit Rules Knowledge Graph Recognition

[0113]

[0114] Figure 5 This illustration shows a partial knowledge graph of financial audit rules provided in one embodiment of this application.

[0115] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0116] Corresponding to the knowledge graph construction method described in the above embodiments, Figure 6 A structural block diagram of the knowledge graph construction apparatus 200 provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0117] See Figure 6 The knowledge graph construction device in this application embodiment may include a data acquisition module 201, a feature extraction module 202, a feature splicing module 203, and a decoding output module 204.

[0118] The data acquisition module 201 is used to acquire knowledge data and encode the feature vectors of the knowledge data using an embedding layer.

[0119] The feature extraction module 202 is used to extract the first sequence features from the feature vector using the first sequence feature extraction model.

[0120] Optionally, the first sequence feature extraction model includes a first BiGRU layer and a second BiGRU layer. The first BiGRU layer and the BiGRU layer are connected in a stacked manner. Each of the first and second BiGRU layers consists of two layers of GRU units. The input sequence of the first layer GRU unit is a forward-facing GRU, and the input sequence of the second layer GRU unit is a backward-facing GRU. The feature vector is first extracted through the first BiGRU layer, and then through the second BiGRU layer to output the first sequence features.

[0121] The feature extraction module 202 is also used to extract second sequence features from the feature vector using a second sequence feature extraction model.

[0122] Optionally, the second sequence feature extraction model includes a BiLSTM layer, which is used to extract semantic features from character-level feature vectors; the BiLSTM layer receives feature vectors and outputs character-level feature vectors.

[0123] Optionally, the second sequence feature extraction model also includes a Self-Attention layer, which is used to assign attention weights to character-level feature vectors. These attention weights characterize the importance of each character-level feature vector to the knowledge graph construction. The Self-Attention layer receives the feature extraction from the BiLSTM layer's output of the feature vectors, and outputs the second sequence features.

[0124] The feature splicing module 203 is used to splice the first sequence features and the second sequence features to obtain the third sequence features.

[0125] Optionally, the Concat function is used to concatenate the first and second sequence features to obtain the third sequence feature. The expression for the third sequence feature is as follows:

[0126] H(x) = concat[F(x) + G(x)]

[0127] In the formula, H(x) represents the third sequence feature, F(x) represents the first sequence feature, and G(x) represents the second sequence feature.

[0128] The decoding output module is used to select the third sequence features that conform to the expression logic using the CRF layer to obtain the triplet sequence.

[0129] Among them, the triple sequence consists of entity-relation-entity, and multiple triple sequences constitute a complete knowledge graph.

[0130] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0131] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0132] This application also provides a terminal device, see [link to relevant documentation] Figure 7 The terminal device 300 may include at least one processor 310 and a memory 320. The memory 320 stores a computer program 321 that can run on the at least one processor 310. When the processor 310 executes the computer program 321, it implements the steps in any of the above-described method embodiments, for example... Figure 1 Steps 101 to 103 in the illustrated embodiment. Alternatively, when processor 310 executes computer program 321, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 6 The functions of modules 201 to 204 are shown.

[0133] For example, computer program 321 may be divided into one or more modules / units, one or more of which are stored in memory 320 and executed by processor 310 to complete this application. One or more modules / units may be a series of computer program segments capable of performing a specific function, which describe the execution process of the computer program in terminal device 300.

[0134] Those skilled in the art will understand that Figure 6 This is merely an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, buses, etc.

[0135] The processor 310 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0136] The memory 320 can be an internal storage unit of the terminal device 300 or an external storage device of the terminal device 300, such as a plug-in hard disk, a smart media card (SMC), a secure digital card (SD), or a flash card. The memory 320 is used to store the computer program 321 and other programs and data required by the terminal device 300. The memory 320 can also be used to temporarily store data that has been output or will be output.

[0137] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0138] The knowledge graph construction method provided in this application can be applied to terminal devices such as computers, wearable devices, in-vehicle devices, tablet computers, laptop computers, netbooks, personal digital assistants (PDAs), augmented reality (AR) / virtual reality (VR) devices, and mobile phones. This application does not impose any restrictions on the specific type of terminal device.

[0139] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the various embodiments of the knowledge graph construction method described above.

[0140] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the various embodiments of the knowledge graph construction method.

[0141] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0142] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0143] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0144] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0145] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0146] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for constructing a knowledge graph, characterized in that, include: Acquire knowledge data and encode the feature vector of the knowledge data using an embedding layer. The feature vector is an identifier after encoding the knowledge data, wherein the knowledge data includes image data and text data. A first sequence feature extraction model is used to extract the first sequence feature from the feature vector. The first sequence feature extraction model includes a first BiGRU layer and a second BiGRU layer, and the first BiGRU layer and the second BiGRU layer are connected by stacking. The first BiGRU layer and the second BiGRU each consist of two layers of GRU units. The input sequence of the first layer GRU unit is a forward GRU, and the input sequence of the second layer GRU unit is a reverse GRU. The feature vector is first extracted through the first BiGRU layer, and then extracted through the second BiGRU layer to output the first sequence features. A second sequence feature extraction model is used to extract second sequence features from the feature vector. The second sequence feature extraction model includes a BiLSTM layer and a Self-Attention layer. The BiLSTM layer is used to extract semantic features from the character-level feature vector. The Self-Attention layer is used to assign attention weights to the character-level feature vectors. The attention weights represent the importance of each character-level feature vector to the knowledge graph construction. The Self-Attention layer receives the feature extraction results output by the BiLSTM layer and outputs the second sequence features. By concatenating the first sequence feature and the second sequence feature, a third sequence feature is obtained; The CRF layer is used to select the third sequence features that conform to the expression logic to obtain the triple sequence. The triple sequence is composed of entity-relation-entity, and multiple triple sequences constitute a complete knowledge graph. After acquiring the knowledge data and before encoding the feature vector of the knowledge data using an embedding layer, the method further includes: The text in the image data is identified using an OCR recognition model; The Jieba word segmentation model was used to segment the text data into words according to Chinese grammar. Construct a triplet feature template, wherein the triplet feature template is in the form of entity-relation-entity.

2. The knowledge graph construction method as described in claim 1, characterized in that, The BiLSTM layer receives the feature vector and outputs the character-level feature vector.

3. The knowledge graph construction method as described in claim 1, characterized in that, The step of concatenating the first sequence features and the second sequence features to obtain the third sequence features includes: The first sequence feature and the second sequence feature are concatenated using the Concat function to obtain the third sequence feature, the expression of which is: In the formula, Indicates the characteristics of the third sequence. Indicates the features of the first sequence. This indicates the characteristics of the second sequence.

4. The knowledge graph construction method as described in claim 1, characterized in that, Before selecting the third sequence feature that conforms to the expression logic using the CRF layer, the method includes: The third sequence feature is encoded using a fully connected layer, integrating multiple dimensions of the third sequence feature into one dimension.

5. A knowledge graph construction device, characterized in that, For implementing the knowledge graph construction method as described in any one of claims 1-4, the knowledge graph construction apparatus comprises: The data acquisition module is used to acquire knowledge data and encode the feature vector of the knowledge data using an embedding layer; The feature extraction module is used to extract a first sequence feature from the feature vector using a first sequence feature extraction model, and is also used to extract a second sequence feature from the feature vector using a second sequence feature extraction model; A feature splicing module is used to splice the first sequence feature and the second sequence feature to obtain a third sequence feature; as well as, The decoding output module is used to select the third sequence features that conform to the expression logic using the CRF layer to obtain a triple sequence. The triple sequence consists of entity-relation-entity, and multiple triple sequences constitute a complete knowledge graph.

6. A terminal device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 4.

Images