A method, device and storage medium for extracting keywords from automotive text

Abstract

The present invention relates to the technical field of natural language processing, and discloses a method, device and storage medium for extracting keywords from text in the automotive field. The method generates a triple knowledge graph based on text in the automotive field, and in the triple knowledge graph, calculates the weight of each node and the weight of each edge; calculates the influence of the target node based on the weight of the adjacent nodes of the target node and the weight of the edge connected to the target node; selects nodes whose influence meets the set requirements to obtain keywords of the text, and the influence can evaluate the coverage of the keywords represented by the node to the entire text and the fit to the technology, avoids extracting key words that are irrelevant to the technical content, and takes into account both semantic coverage and technical coverage.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method, device and storage medium for extracting text keywords in the automotive field. Background Art

[0002] Keyword extraction has always been an important topic in modern natural language processing. Its application ranges from ontology generation, fact verification in summary texts, and recommendation systems. Different from traditional keyword extraction, technical keywords in the automotive field focus more on technical names and key technical descriptions. Moreover, technical vocabulary is updated quickly, and the description texts are mostly news and reports, with many non-professional vocabulary. Traditional keyword extraction schemes are prone to obtain non-technical vocabulary.

[0003] For example, the KeyBERT algorithm is currently used to extract keywords from text. First, BERT is used to extract document embedding to obtain a document-level vector representation. Subsequently, word vectors are extracted for N-gram words / phrases, and then cosine similarity is used to find the words / phrases that are most similar to the document. Finally, the most similar words can be identified as the words that best describe the entire document. However, the above schemes have the following problems in keyword extraction in the field of automotive technology: 1. Lack of technical judgment of keywords: Since this method mainly considers extraction schemes based on semantic coverage, in actual applications, most of the content involved in the technical description text is technical details, and the professional vocabulary described is not technical text; 2. There are problems with the alignment between keywords and the entire document, and there is a certain deviation in the similarity comparison between words and semantics and paragraph-level semantics.

[0004] In view of this, the present invention is proposed. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method, device and storage medium for extracting text keywords in the automotive field. Based on the triple knowledge graph, a classification model is used to screen out the edges described by the subordinate relationship class, and further suitable keywords are obtained through comprehensive calculation, taking into account both semantic coverage and technical coverage.

[0006] An embodiment of the present invention provides a method for extracting keywords from text in the automotive field, the method comprising:

[0007] Generate a triple knowledge graph based on the text in the automotive field; nodes represent entities, edges represent attributes; a triple consists of an edge and two nodes at both ends;

[0008] In the triple knowledge graph, the weight of each node and the weight of each edge are calculated; wherein the weight of the edge is determined according to the relevance of the edge and the two nodes at both ends to the technology;

[0009] Calculate the influence of the target node according to the weights of the adjacent nodes of the target node and the weights of the edges connected to the target node; wherein the target node is any node in the triple knowledge graph;

[0010] Nodes whose influence meets the set requirements are selected to obtain keywords of the text.

[0011] An embodiment of the present invention provides an electronic device, the electronic device comprising:

[0012] Processor and memory;

[0013] The processor is used to execute the steps of the method for extracting keywords from text in the automotive field described in any embodiment by calling the program or instruction stored in the memory.

[0014] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program or instruction, wherein the program or instruction enables a computer to execute the steps of the method for extracting keywords from text in the automotive field as described in any embodiment.

[0015] The embodiments of the present invention have the following technical effects:

[0016] 1. Extract keywords based on the triple knowledge graph, retain the semantic information at the chapter level, and avoid information loss caused by similarity coverage; moreover, the keywords extracted by this scheme can retain a certain hierarchical relationship.

[0017] 2. The weight of the edge is determined based on the relevance of the edge and the two nodes at both ends to the technology, and then the influence of the node is calculated based on the weight of the edge and the weight of the node. This influence can evaluate the coverage of the keywords represented by the node to the entire text and the fit to the technology, avoiding the extraction of keywords that are not related to the technical content.

[0018] 3. The core idea of ​​the present invention is not to align the entire document with keywords, nor is it intended to fully obtain all the semantics of the text, but to construct candidate words through triplets, and then find the key nodes that best express the text semantics and fit the technical description, taking into account both semantic coverage and technical coverage. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1is a flow chart of a method for extracting keywords from text in the automotive field provided by an embodiment of the present invention;

[0021] Figure 2 is a schematic diagram of a knowledge graph provided by an embodiment of the present invention;

[0022] Figure 3 is a schematic diagram of another knowledge graph provided by an embodiment of the present invention;

[0023] Figure 4 is a schematic diagram of the training process of the classification model provided by an embodiment of the present invention;

[0024] Figure 5 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0026] The method for extracting keywords from automotive text provided by the embodiment of the present invention is mainly suitable for extracting technical keywords including technical details from automotive text, and is closer to the semantics of the entire text. The method for extracting keywords from automotive text provided by the embodiment of the present invention can be executed by electronic devices such as computers.

[0027] Embodiment 1

[0028] Figure 1 1 is a flowchart of a method for extracting keywords from text in the automotive field provided by an embodiment of the present invention. Figure 1 , the method for extracting keywords from text in the automotive field specifically includes:

[0029] S110. Generate a triple knowledge graph based on the text in the automotive field.

[0030] This embodiment constructs triple information of the text through a relation extraction model, and ensures coverage with the source text through triple information, so as to retain the semantic information at the chapter level while extracting keywords, avoiding information loss caused by similarity coverage.

[0031] Specifically, an artificial intelligence big data model is used to generate a triple knowledge graph. The present invention does not limit the type of big data model, and it can be a language model composed of an encoder and a decoder, etc.

[0032] For example, the original text is as follows:

[0033]

[0034] Based on the above text, the triples that can be constructed are as follows. The triples include an edge and two nodes at both ends. The triples knowledge graph can be found in Figure 2 , nodes represent entities and edges represent attributes.

[0035]

[0036] For example, the original text is as follows:

[0037]

[0038] Based on the above text, the triples that can be constructed are as follows. The triple knowledge graph can be found in Figure 3 , nodes represent entities and edges represent attributes.

[0039]

[0040] S120. In the triple knowledge graph, calculate the weight of each node and the weight of each edge.

[0041] Calculating the weight of each node includes: calculating the degree and core value of each node in the triple knowledge graph.

[0042] The degree value (k) of a node indicates the number of neighboring nodes of the node. The larger the degree value, the stronger the correlation between the node and other nodes, and the more it can reflect the semantics of the entire text. The core value (ks) of a node indicates the position of the node in the graph. The core value calculation method can refer to the K-shell method. The specific process is as follows: First, continuously delete nodes with a degree of 1 and their edges in the graph G until such nodes no longer appear in the graph G. At this time, all deleted nodes are classified as 1-shell layers and assigned a ks value of 1. Secondly, continuously delete nodes with a degree of 2 and their edges in the graph G, classify the deleted nodes as 2-shell layers, and assign them a ks value of 2. Repeat the above process until all nodes in the graph G are layered and assigned ks values. Among them, the layer to which the node with the largest ks value belongs is the core layer of the graph, and the nodes in the core layer have the greatest influence.

[0043] The weight of the edge is determined based on the relevance of the edge and the two nodes at both ends to the technology. Since the edge represents an attribute and the node represents an entity, the edge and the nodes at both ends constitute a triple representation of "entity-attribute-entity". This triple can already express certain semantics. If the semantics it expresses is highly relevant to the technology, the weight of the edge is large. If the semantics it expresses is less relevant to the technology, the weight of the edge is small. How to determine the relevance to the technology will be described in Example 2.

[0044] S130: Calculate the influence of the target node according to the weights of the neighboring nodes of the target node and the weights of the edges connected to the target node.

[0045] The target node is any node in the triple knowledge graph, that is, the influence of each node in the graph must be calculated according to the steps provided in S130.

[0046] This embodiment reflects that the influence of the target node is obtained by combining the weights of the adjacent nodes and the weights of the connected edges, and the specific influence calculation formula can be adjusted according to actual needs.

[0047] Optionally, the influence of the target node is calculated using the following formula: :

[0048]

[0049] in, i , j is the node number, 1 minus the node i With Node j The value obtained by the weight of the edge between k is the degree value, ks is the core value. It should be noted that the node j Should be a node i The adjacent nodes of .

[0050] Assume that the neighboring nodes of node A include node B and node C. The degree value of node A is 2, the core value is 2, the degree value of node B is 2, the core value is 1, the degree value of node C is 3, the core value is 2, the weight of the edge between node A and node B is 0.8, and the weight of the edge between node A and node C is 0.6, then the influence of node A is:

[0051]

[0052] The influence of a node can be used to evaluate the coverage of the keywords represented by the node to the entire text and its fit with the technology.

[0053] S140: Select nodes whose influence meets the set requirements to obtain keywords of the text.

[0054] After calculating the influence of all nodes, you can select the keywords corresponding to the nodes whose influence is greater than the set value, or sort all nodes in order of influence from large to small, and select the keywords corresponding to the nodes with the highest ranking. The set value and the number of keywords to be selected can be determined according to actual needs.

[0055] The technical effects that can be achieved by the embodiments of the present invention are:

[0056] 1. Extract keywords based on the triple knowledge graph, retain the semantic information at the chapter level, and avoid information loss caused by similarity coverage; moreover, the keywords extracted by this scheme can retain a certain hierarchical relationship.

[0057] 2. The weight of the edge is determined based on the relevance of the edge and the two nodes at both ends to the technology, and then the influence of the node is calculated based on the weight of the edge and the weight of the node. This influence can evaluate the coverage of the keywords represented by the node to the entire text and the fit to the technology, avoiding the extraction of keywords that are not related to the technical content.

[0058] 3. The core idea of ​​the present invention is not to align the entire document with keywords, nor is it intended to fully obtain all the semantics of the text, but to construct candidate words through triplets, and then find the key nodes that best express the text semantics and fit the technical description, taking into account both semantic coverage and technical coverage.

[0059] Embodiment 2

[0060] This embodiment further optimizes the weight calculation method of each edge based on the above embodiment. Specifically, for the convenience of description and distinction, the target edge is taken as any edge in the triple knowledge graph, and the target edge and the two nodes at both ends are called a triple.

[0061] The triplet is input into the classification model to obtain the category to which it belongs. The category includes a subordination class description and an entity attribute class description. The probability of the subordination class description is used as the weight of the target edge to evaluate the technical relevance between the nodes.

[0062] The following texts are typical descriptions:

[0063]

[0064] Among them, the affiliation class description is more used for feedback technology entities, so the probability of this class is used as the weight of the target edge.

[0065] The classification model can be any model that can perform binary classification on text. Preferably, this embodiment uses the contrastive learning method to construct the classification model, and the Transformers encoder and the pooling layer constitute the main body of the model. In this way, no additional costs (data, reasoning performance, etc.) are added. Only by using contrastive learning, a relatively low-cost technical solution in the field of natural language processing is formed to improve the effect of existing models. The loss function is constructed by jointly using cross entropy loss and contrastive learning loss, so that the model can converge to the optimal state as soon as possible, and the accuracy and efficiency of classification can be improved.

[0066] Specifically, see Figure 4 The classification model includes a word embedding layer, a Transformers encoder, and a pooling layer. The classification model is constructed using the contrastive learning method. Contrastive learning is a self-supervised learning method in deep learning that learns effective data representation by comparing the similarities and differences between different samples.

[0067] The input of the classification model described in this method is a triple structure, entity-attribute-entity, and the output is the probability of the two categories of "subordinate relationship description" and "entity attribute description". Since the number of output categories is 2, the output of the model can be represented by the probability of the value range [0, 1]. Among them, when the model output probability is <0.5, the "entity attribute description" class is taken, otherwise it is "subordinate relationship description".

[0068] "Description of affiliation" and "Description of entity attributes" are two types of labels used for text classification model training and prediction. These two labels are introduced to better understand and process entity relationships in technical texts, thereby improving the accuracy and relevance of keyword extraction. In technical texts, understanding the relationships and attributes between technical concepts is crucial for accurate keyword extraction. In this way, the model can not only identify keywords, but also evaluate the importance and relevance of these keywords in technical texts, thereby improving the quality and practicality of keyword extraction.

[0069] "Subordinate relationship description" tag: used to describe the hierarchy or causal relationship between technical entities, such as how one technical concept is inspired or solved by another concept. This relationship is usually dynamic and can reflect the interaction and influence between technical concepts. For example, a new technology may solve an existing problem (cause-solution), or a new mechanism may inspire a new application (inspiration-application). By identifying these subordinate relationships, the model can more accurately extract keywords that are closely related to the technical content, rather than just based on word frequency or semantic coverage.

[0070] "Entity attribute description" tag: This tag is used to describe the static attributes of technical entities, such as material properties, product performance parameters, etc. These attributes are usually descriptive and can directly provide specific information about the technical entity. For example, the yield strength of a steel, the efficiency of an algorithm, etc. By identifying these entity attributes, the model can extract key words that describe technical details to help users quickly understand the technical content of the text. Specifically, this type of tag includes but is not limited to the following tags: "concept definition description", "process step description", "comparison description", "solution description", etc.

[0071] In the specific training process, the first step is to select triplets of subordinate relationship classes and triplets of entity attribute classes from the triple knowledge graph samples and input them into the classification model to be trained.

[0072] For the label Input sample ,This method first searches for triples that also contain entities in all triples and constructs a batch, ,in Tagged with Taking the original sample of (high-strength steel, welding performance, good) as an example, the entities "high-strength steel" and "good" can be expanded to any one of the three tuples (high-strength steel, delivery status, normalized rolling), (high-strength steel, delivery status, mechanically controlled rolling), and (high-strength steel, delivery status, quenching and tempering (quenching and tempering)). The above two samples are merged into a batch and processed into the model preset format.

[0073] Specifically, for each input triple (entity 1, attribute, entity 2), it is concatenated into a single string, where the entity and attribute are separated by a special delimiter ([SEP]). For example, if the triple is ("New York", "located in", "United States"), the processed input can be "[CLS]New York [SEP] located in [SEP] United States [SEP]"

[0074] Input the triples into the word embedding layer to convert each word into a vector in a high-dimensional space. This process can be done with the help of pre-trained Transformer models such as BERT, RoBERTa, etc. Then enter the Transformers encoder, each encoder layer uses a self-attention mechanism to process the input sequence and capture various dependencies within the sequence. Finally, enter the pooling layer. The output of the Transformer encoder can be obtained by pooling the pooling layer (such as taking the first token of the sequence, which is usually the output of the special [CLS] token) to obtain a fixed-size vector representation, which can be regarded as the global information of the entire input sequence.

[0075] The second step is to construct a loss function, which includes cross entropy loss and contrastive learning loss.

[0076] A) Obtaining cross entropy loss: The triple representation vector of the above two texts is input into the linear layer mapping, and its prediction loss is obtained through the cross entropy loss function.

[0077] B) Obtaining contrastive learning loss: Based on the above sample augmented batch, the cosine similarity between the triple representations is calculated respectively, and the cosine similarity between samples of different categories is used as the loss.

[0078] Contrastive Learning Loss The calculation is as follows:

[0079]

[0080] in, h = pooling(Transformers( x )), h' = pooling(Transformers( x' )), h and h ' is the output of the pooling layer.

[0081] Loss Function Loss as follows:

[0082]

[0083] in is the cross entropy loss function.

[0084] The third step is to update the parameters of the classification model to minimize the loss function.

[0085] When the model is actually used, the probability of the model output being the "subordinate relationship class description" class is taken = dependency relationship), and calculate based on this value = affiliation), where P is the probability and F is the classification model.

[0086] Embodiment 3

[0087] Figure 5 Schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 5 As shown, the electronic device 400 includes one or more processors 401 and a memory 402 .

[0088] The processor 401 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0089] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may run the program instructions to implement the method for extracting text keywords in the automotive field of any embodiment of the present invention described above and / or other desired functions. Various contents such as initial external parameters, thresholds, etc. may also be stored in the computer-readable storage medium.

[0090] In one example, the electronic device 400 may further include: an input device 403 and an output device 404, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). The input device 403 may include, for example, a keyboard, a mouse, etc. The output device 404 may output various information to the outside, including early warning prompt information, braking force, etc. The output device 404 may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0091] Of course, to simplify, Figure 5 Only some of the components related to the present invention in the electronic device 400 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 400 may also include any other appropriate components.

[0092] In addition to the above methods and devices, an embodiment of the present invention may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method for extracting keywords from automotive text provided by any embodiment of the present invention.

[0093] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the embodiments of the present invention, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0094] In addition, an embodiment of the present invention may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method for extracting keywords from automotive-related texts provided by any embodiment of the present invention.

[0095] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0096] It should be noted that the terms used in the present invention are only for describing specific embodiments, rather than limiting the scope of the present application. As shown in the present specification, unless the context clearly indicates an exception, the words "one", "a", "a kind of" and / or "the" do not specifically refer to the singular, but may also include the plural. The terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the presence of other identical elements in the process, method or device including the elements.

[0097] It should also be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", etc. should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting keywords from text in the automotive field, characterized in that: include: Generate triple knowledge graph based on text in the automotive field; Nodes represent entities, edges represent attributes; a triple consists of an edge and two nodes at both ends; In the triple knowledge graph, the weight of each node and the weight of each edge are calculated; wherein the weight of the edge is determined according to the relevance of the edge and the two nodes at both ends to the technology; Calculate the influence of the target node according to the weights of the adjacent nodes of the target node and the weights of the edges connected to the target node; wherein the target node is any node in the triple knowledge graph; Select nodes whose influence meets the set requirements to obtain keywords of the text; Wherein, in the triple knowledge graph, the weight of each edge is calculated, including: inputting the triple into the classification model to obtain the category to which it belongs; the category includes a subordination class description and an entity attribute class description; using the probability of the subordination class description as the weight of the target edge; wherein the target edge is any edge in the triple knowledge graph; wherein the subordination class description is used to describe the hierarchy or causal relationship between technical entities, and the entity attribute class description is used to describe the static attributes of the technical entity; The calculating the influence of the target node according to the weights of the neighboring nodes of the target node and the weights of the edges connected to the target node includes: The influence of the target node is calculated using the following formula: : in, i , j is the node number, 1 minus the node i With Node j The value obtained by the weight of the edge between k is the degree value, ks is the core value; The degree value of the node represents the number of neighbor nodes of the node; the core value of the node represents the position of the node in the graph, and the K-shell method is used to calculate the core value.

2. The method according to claim 1, characterized in that In the triple knowledge graph, the weight of each node is calculated, including: In the triple knowledge graph, the degree and core value of each node are calculated.

3. The method according to claim 1, characterized in that The classification model includes a word embedding layer, a Transformers encoder, and a pooling layer.

4. The method according to claim 3, characterized in that The classification model is constructed using a contrastive learning method; Select triples of subordinate relationship classes and triples of entity attribute classes from the triple knowledge graph samples, and input them into the classification model to be trained; Constructing a loss function, wherein the loss function includes a cross entropy loss and a contrastive learning loss; The parameters of the classification model are updated to minimize the loss function.

5. An electronic device, characterized in that: The electronic device comprises: Processor and memory; The processor is used to execute the steps of the method for extracting keywords from text in the automotive field as claimed in any one of claims 1 to 4 by calling the program or instruction stored in the memory.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and the program or instruction enables a computer to execute the steps of the method for extracting keywords from text in the automotive field as claimed in any one of claims 1 to 4.

Images