Knowledge extraction method and device, equipment and storage medium

Abstract

The invention provides a network operation and maintenance field knowledge extraction method based on a generative artificial intelligence model, and the method comprises the steps: taking a rare Chinese character as an auxiliary symbol, combining the rare Chinese character with a relation word list, and connecting the rare Chinese character with an original text to form a new input sequence; a pre-trained generative artificial intelligence model is selected, P-Tuning fine tuning is carried out to adapt to a knowledge extraction task in the field of network operation and maintenance, and in model output, specific isolation words are utilized to segment and recognize different components of a knowledge triple. By means of the technical scheme, the technical problems that in the prior art, a model is difficult to process a complex entity structure and cannot flexibly output knowledge triples can be effectively solved.

Description

Technical Field

[0001] This application relates to the fields of network operation and maintenance and artificial intelligence technology, specifically to a knowledge extraction method, apparatus, device, and storage medium. Background Technology

[0002] In the field of Natural Language Processing (NLP), knowledge graph construction is one of the main applications of information extraction. Knowledge triple extraction is a core component of knowledge graph construction, transforming unstructured text into structured knowledge representations by automatically analyzing and recognizing entities and their relationships within the text. Currently, Named Entity Recognition (NER) and Relation Extraction (RE) are two common knowledge triple extraction methods, typically based on deep learning models such as BERT+Bi-LSTM+CRF for entity recognition and relation classification.

[0003] Existing technical solutions primarily rely on sequence labeling techniques and relation classification algorithms. These methods use a combination of Bidirectional Long Short-Term Memory Network (Bi-LSTM) and Conditional Random Field (CRF) models for named entity recognition, followed by relation extraction models for relation classification to construct a knowledge graph. While this approach demonstrates some effectiveness and accuracy in processing structured and semi-structured data, it suffers from the following shortcomings in the specific domain of network operations and maintenance, particularly for highly unstructured text data: 1) Difficulty in handling complex entity structures: Texts in the field of network operation and maintenance often contain complex entity nesting and intersection phenomena, such as device configuration parameters, network topology information, etc. The existence of these complex entity structures makes it difficult for traditional NER methods based on sequence labeling to accurately identify and distinguish entity boundaries, resulting in a decrease in the accuracy of triple extraction.

[0004] 2) Lack of flexibility in model output: When extracting knowledge triples, existing models rely too heavily on existing continuous words in the text. When the triples to be extracted do not appear completely in the original text, the model cannot automatically fill in the missing information, which limits the scope of knowledge extraction. In particular, when faced with the specific terms and concepts of network devices, the model's generalization and generation capabilities are insufficient, which limits its application in knowledge graph construction in the operation and maintenance field.

[0005] These shortcomings demonstrate that existing technologies have significant limitations when facing specific challenges in network operations and maintenance, particularly in dealing with complex entity structures and flexible output requirements. Therefore, a new solution is urgently needed to overcome these deficiencies and more effectively extract knowledge triples from the network operations and maintenance domain, thereby improving the accuracy and coverage of knowledge graph construction and meeting the growing demand for automated network operations and maintenance. Summary of the Invention

[0006] This application provides a knowledge extraction method to overcome the significant shortcomings of existing sequence labeling-based knowledge extraction methods in handling complex entity structures and flexibly outputting knowledge triples. On the one hand, traditional models such as LSTM+CRF struggle to effectively identify nested or intersecting entities, limiting their ability to handle complex structures. On the other hand, when text information is incomplete, the model cannot generate missing knowledge triples, affecting the completeness and accuracy of the extraction. Therefore, there is an urgent need for a knowledge extraction method that can flexibly generate knowledge triples, adapt to complex entity structures, and possess strong semantic understanding capabilities to improve the comprehensiveness and intelligence of knowledge graph construction.

[0007] Firstly, a knowledge extraction method is provided. This includes: determining a list of relational terms contained in the acquired original text; concatenating the original text according to the list of relational terms and a pre-defined first-class Chinese character set to obtain an input sequence; inputting the input sequence into a pre-trained text processing model, enabling the model to generate a corresponding output sequence based on the input sequence, wherein the output sequence contains at least one knowledge triplet corresponding to the original text, and at least one first-class Chinese character from the first-class character set is used as a delimiter to segment each knowledge triplet in the output sequence; and parsing the output sequence to obtain at least one knowledge triplet corresponding to the original text.

[0008] Based on the methods described above, this method constructs a list of relational terms and combines it with a pre-defined set of first-class Chinese characters to concatenate the original text, effectively improving the structured representation of the input sequence and enhancing the model's ability to recognize and process complex entity structures. The concatenated input sequence is then fed into a pre-trained text processing model. Leveraging the advantages of generative artificial intelligence models, it can flexibly generate output sequences containing knowledge triples. Accurate segmentation of knowledge triples is achieved using first-class Chinese characters as delimiters, significantly improving the accuracy and completeness of knowledge extraction. This method is particularly suitable for complex scenarios in network operations and maintenance that are difficult to handle with traditional NER solutions, especially when the output answer does not appear in the original text, demonstrating strong generalization ability and practical application value.

[0009] In conjunction with the first aspect, in some possible implementations of the first aspect, the first set of Chinese characters is selected from rare characters outside the "General Standard Chinese Characters List", and the frequency of their occurrence in regular corpora is lower than a preset threshold.

[0010] In conjunction with the first aspect, in some possible implementations of the first aspect, the original text is concatenated according to the list of relation words and at least one preset set of first-class Chinese characters to obtain the input sequence. Specifically, this includes: concatenating each relation word in the list of relation words with the original text to obtain the initial input sequence; and segmenting each relation word in the initial input sequence with the original text according to at least one first-class Chinese character in the set of first-class Chinese characters to obtain the input sequence.

[0011] In conjunction with the first aspect, in some possible implementations of the first aspect, at least one first-class Chinese character is used as a delimiter to segment the knowledge triples contained in the output sequence. Specifically, this includes: using the first Chinese character from the first-class Chinese character set as the starting identifier of the knowledge triple; using the second Chinese character from the first-class Chinese character set as the delimiter between the subject and predicate in the knowledge triple; using the third Chinese character from the first-class Chinese character set as the delimiter between the predicate and object in the knowledge triple; and using the fourth Chinese character from the first-class Chinese character set as the ending identifier of the knowledge triple.

[0012] In conjunction with the first aspect, in some possible implementations of the first aspect, the text processing model is built on the Transformer architecture encoder-decoder model.

[0013] In conjunction with the first aspect, in some possible implementations of the first aspect, a pre-trained text processing model is specifically included, comprising: obtaining a training dataset, which includes multiple training samples, each training sample including original text and corresponding knowledge triplet labels; determining a list of relation words based on the original text of the training samples; concatenating the original text of each training sample according to the list of relation words and a preset first-class set of Chinese characters to obtain a training input sequence; generating a training output sequence according to the knowledge triplet labels and the preset first-class set of Chinese characters, wherein the first-class Chinese characters are used as delimiters to segment the knowledge triplets; and fine-tuning the parameters of the pre-trained model based on the Transformer architecture using the training input sequence and the training output sequence to obtain the text processing model.

[0014] Secondly, a knowledge extraction device is provided. It includes: a text acquisition module for acquiring original text; a relation word determination module for determining one or more relation words contained in the original text to form a relation word list; a sequence construction module for concatenating the original text according to the relation word list and a preset first-class Chinese character set to obtain an input sequence, wherein the first-class Chinese character set consists of uncommon characters; a model processing module for inputting the input sequence into a pre-trained text processing model to generate a corresponding output sequence, wherein the output sequence contains at least one knowledge triplet corresponding to the original text, separated by Chinese characters from the first-class Chinese character set; and a result parsing module for parsing the output sequence and extracting at least one knowledge triplet.

[0015] Based on the methods described above, this knowledge extraction device introduces a relation word determination module and a sequence construction module, and combines a pre-set first-class set of Chinese characters (rare characters) to concatenate the original text, effectively enhancing the semantic expressiveness and structural complexity of the input sequence, thereby improving the model's accuracy in processing complex entity structures. The model processing module uses a pre-trained text processing model, which can generate output sequences of knowledge triples separated by rare characters, solving the shortcomings of traditional NER schemes in processing non-explicit entities and complex structures. The result parsing module further extracts structured knowledge triples, achieving efficient extraction of implicit knowledge in the field of network operation and maintenance. This method not only improves the model's ability to recognize rare entities and complex semantic structures, but also significantly improves the limitations of traditional sequence labeling-based methods in knowledge triple extraction tasks, especially suitable for scenarios where the output answer does not appear directly in the original text, and has good practical application value.

[0016] In conjunction with the second aspect, in some possible implementations of the first aspect, the sequence construction module is specifically used for: The initial input sequence is obtained by concatenating each relation word in the relation word list with the original text; the initial input sequence is obtained by segmenting each relation word in the initial input sequence with the original text based on at least one first-class Chinese character in the first-class Chinese character set.

[0017] Thirdly, an electronic device includes: one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more application programs are configured to perform the methods as described in any of the foregoing aspects.

[0018] Fourthly, a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the methods of any of the above aspects. Attached Figure Description

[0019] Figure 1 This is a schematic diagram illustrating a knowledge extraction method applicable to an embodiment of this application. Figure 2 This is a schematic diagram of a knowledge extraction device applicable to the embodiments of this application. Detailed Implementation To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The specific operating methods in the method embodiments can also be applied to the device embodiments or system embodiments. In the description of this application, unless otherwise stated, "multiple" means two or more.

[0020] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of different embodiments are consistent and can be referenced by each other. The technical features of different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0021] It is understood that the various numerical designations used in this application are merely for descriptive convenience and are not intended to limit the scope of this application. The order of the process numbers does not imply the order of execution; the execution order of each process should be determined by its function and internal logic.

[0022] The terms "first," "second," "third," "fourth," and other various terminology (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] To facilitate understanding of the embodiments of this application, the terminology involved in the embodiments of this application will be briefly explained below.

[0024] Knowledge triples: A knowledge representation consisting of a subject, predicate, and object, used to describe semantic relationships between entities, and is the most basic data unit in a knowledge graph.

[0025] Transformer architecture: A deep learning model architecture based on self-attention mechanism, widely used in natural language processing tasks. It consists of two parts: encoder and decoder, and can effectively capture long-distance dependencies in text.

[0026] Rare characters: These are Chinese characters that appear very infrequently in regular corpora and are usually not in the "General Standard Chinese Characters List". In this invention, they are used as separators to achieve structured segmentation of knowledge triples.

[0027] Pre-trained models: Deep learning models pre-trained on large-scale corpora have good language understanding and generation capabilities, and can be fine-tuned to adapt to specific tasks, such as knowledge extraction.

[0028] AIGC model: Artificial Intelligence Generated Content model, is a type of generative artificial intelligence model that can generate high-quality text, images and other content. In this invention, it is used to generate and extract knowledge triples.

[0029] Separator: Special characters used to physically separate different components (such as subject, predicate, and object) of knowledge triples in the generated output sequence. This invention uses rare Chinese characters as separators to avoid conflicts with natural language.

[0030] Based on this, this application provides a method for knowledge extraction in the network operation and maintenance field based on a generative artificial intelligence model, so as to overcome the obvious shortcomings of existing knowledge extraction methods based on sequence labeling in handling complex entity structures and flexibly generating knowledge triples.

[0031] Figure 1 This is a schematic diagram of a knowledge extraction method included in an embodiment of this application. For example... Figure 1 As shown, the method includes steps S101, S102, S103, and S104. S101, Based on the obtained original text, determine the list of relational terms contained in the original text. Specifically, keywords representing relationships between entities, such as "send" and "represent," are extracted from the original text and used as a list of relation words for subsequent concatenation processing and model input construction. S102, based on the list of relation words and the preset first type of Chinese character set, the original text is concatenated to obtain the input sequence. Specifically, each relation word in the relation word list is concatenated with the original text to form an initial input sequence. Then, specific Chinese characters from the first set of Chinese characters are inserted into this initial input sequence as delimiters to achieve an explicit expression of the knowledge triple structure, thereby generating the final input sequence. S103, input the input sequence into the pre-trained text processing model so that the text processing model generates the corresponding output sequence according to the input sequence. The output sequence contains at least one knowledge triple corresponding to the original text. In the output sequence, at least one first-class Chinese character from the first-class Chinese character set is used as a delimiter to segment each knowledge triple contained in the output sequence. Specifically, the concatenated input sequence is input into a text processing model built on the Transformer architecture encoder-decoder model. The model generates an output sequence based on the input sequence, where each knowledge triple is separated by a specific character from the first type of Chinese character set as the start, separator, and end identifier, thereby achieving the structured output of the knowledge triple. S104, parse the output sequence to obtain at least one knowledge triple corresponding to the original text. Specifically, based on the delimiters used in the output sequence, the output content is split into multiple independent knowledge triples, and the head entity, relation, and tail entity are extracted to complete the task of knowledge triple identification and extraction. In some implementations, such as Figure 1 In the method shown, the first set of Chinese characters is selected from rare characters outside the "General Standard Chinese Characters List," and their frequency of occurrence in regular corpora is below a preset threshold. By using rare characters as delimiters, semantic conflicts with natural language text are effectively avoided, providing the model with a clean structured coding space, thereby improving the accuracy and interpretability of knowledge triple extraction. In some implementations, such as Figure 1 The method shown concatenates each relation word in the relation word list with the original text to obtain the initial input sequence. Then, based on at least one first-class Chinese character from the first-class character set, each relation word in the initial input sequence is segmented with the original text to obtain the final input sequence. By introducing a relation word list and combining it with first-class Chinese characters for segmentation, the model can more clearly identify the boundaries of knowledge triples, improving the information extraction effect. In some implementations, such as Figure 1In the method shown, the first Chinese character from the first set of characters is used as the starting identifier of the knowledge triple; the second Chinese character from the first set of characters is used as the separator between the subject and predicate in the knowledge triple; the third Chinese character from the first set of characters is used as the separator between the predicate and object in the knowledge triple; and the fourth Chinese character from the first set of characters is used as the ending identifier of the knowledge triple. By setting separators with different roles, the explicit expression of the knowledge triple structure is achieved, which helps the model to accurately generate and parse triple content. In some implementations, such as Figure 1 In the method shown, the text processing model is built based on the Transformer architecture encoder-decoder model. The model using the Transformer architecture has powerful context modeling capabilities and long-distance dependency processing capabilities, which can more effectively handle complex text structures and improve the accuracy and robustness of knowledge triple extraction. In some implementations, such as Figure 1 The method described involves obtaining a training dataset, which includes multiple training samples. Each training sample includes original text and corresponding knowledge triplet labels. A list of relation words is determined based on the original text of the training samples. The original text of each training sample is concatenated using the relation word list and a pre-defined first-class Chinese character set to obtain the training input sequence. A training output sequence is generated based on the knowledge triplet labels and the pre-defined first-class Chinese character set, where first-class Chinese characters are used as delimiters to segment the knowledge triples. The parameters of a pre-trained model based on the Transformer architecture are fine-tuned using the training input and training output sequences to obtain the text processing model. By introducing a structured encoding mechanism and targeted training, the performance of the model in specific domain knowledge extraction tasks is significantly improved. In such Figure 1 The method shown involves concatenating the list of relational terms with the original text and inputting it into a text processing model based on the Transformer architecture. It uses a preset first-class Chinese character as the delimiter for knowledge triples, thereby achieving efficient extraction of implicit knowledge triples from the original text. This solves the shortcomings of traditional NER schemes in processing non-explicit answers and improves the accuracy and practicality of knowledge extraction.

[0032] Figure 2 This is a schematic diagram of the structure of a knowledge extraction device provided in an embodiment of this application. Figure 2 As shown, the knowledge extraction device includes a text acquisition module 201, a relation word determination module 202, a sequence construction module 203, a model processing module 204, and a result parsing module 205. Among them, the text acquisition module 201 is used to acquire the original text. Specifically, the text acquisition module 201 can read text content including device configuration information, topology structure, fault records, etc. from data sources in the field of network operation and maintenance. Among them, the relation word determination module 202 is used to determine one or more relation words included in the original text to form a relation word list. Specifically, the relation word determination module 202 can identify verbs, prepositions or domain-specific terms in the text through natural language processing technology and extract them as relation words into the relation word list. Among them, the sequence construction module 203 is used to splice the original text according to the relation word list and a preset first type of Chinese character set to obtain an input sequence. The first type of Chinese character set consists of rare Chinese characters. Specifically, the sequence construction module 203 can splice each relation word in the relation word list with the original text to obtain an initial input sequence; then, according to at least one first type of Chinese character in the first type of Chinese character set, split each relation word and the original text in the initial input sequence to obtain the input sequence. The first type of Chinese character set includes four characters, namely "慶", "諝", "語", and "訖", which are used to represent the triple start symbol, the head entity-relation separator, the relation-tail entity separator, and the triple end symbol respectively. Among them, the model processing module 204 is used to input the input sequence into a pre-trained text processing model to generate a corresponding output sequence. The output sequence contains at least one knowledge triple corresponding to the original text separated by Chinese characters in the first type of Chinese character set. Specifically, the model processing module 204 can adopt the Randeng-T5-Char-57M-Chinese pre-trained model, which is based on the T5 architecture and has good performance in Chinese text generation tasks after corpus adaptive pre-training. Among them, the result parsing module 205 is used to parse the output sequence and extract at least one knowledge triple. Specifically, the result parsing module 205 can separate and extract the triples in the format of "head entity-relation-tail entity" by scanning the separator words in the output sequence. In some implementation manners, the sequence construction module 203 splices each relation word in the relation word list with the original text to obtain an initial input sequence; according to at least one first type of Chinese character in the first type of Chinese character set, split each relation word and the original text in the initial input sequence to obtain the input sequence. This implementation manner improves the recognizability and accuracy of knowledge triples by introducing separator words for explicit structured encoding of triples. In some implementation manners, such as Figure 2The structure shown may also include a preset first type of Chinese character set, which consists of rare Chinese characters, and the first type of Chinese character set includes four characters: "慶", "諝", "語", and "訖". By using low-frequency rare Chinese characters as the structured identifiers of the triples, this implementation effectively avoids semantic conflicts with natural texts and improves the model's ability to recognize the boundaries of triples. In some implementations, the text processing model adopted by the model processing module 204 is a pre-trained model based on the T5 architecture, and this model has undergone corpus adaptive pre-training to improve the understanding ability of the semantics in the field of Chinese network operation and maintenance. By selecting a suitable pre-trained model, this implementation enhances the knowledge extraction performance of the model in a specific field. In the method as Figure 2 shown, by combining the relational words with the original text and using rare Chinese characters as the structured identifiers of the triples, the accurate extraction of knowledge triples is achieved, solving the problem that traditional NER solutions cannot handle the situation where the answer does not appear in the original text, and significantly improving the accuracy and robustness of knowledge extraction.

[0033] Next, the method of the present application will be described in conjunction with specific embodiments. Embodiment 1 In some implementations, constructing a dataset in the field of network operation and maintenance includes collecting data from the expert experience and in-network knowledge in the field of communication networks. The data includes, but is not limited to, the configuration information of network devices, network topology structures, records of network faults and potential hazards, and device operating states, etc. This dataset is annotated by communication experts to form high-quality in-network knowledge data, and for the first time, a data volume of tens of thousands is opened. The dataset is a knowledge Q&A task dataset in the field of communication network operation and maintenance, and its data format is "text - knowledge triple", where the text is synthesized from the Q&A pairs of knowledge Q&A in the field of network operation and maintenance, and the knowledge triple is a triple containing entities and relationships extracted from the corresponding text. Specifically, each text entry corresponds to one or more knowledge triples for subsequent natural language processing tasks, such as named entity recognition (NER) and knowledge extraction.

[0034] Embodiment 2 In some embodiments, the method of using rare Chinese characters to assist in text generation includes the following steps: First, four rare Chinese characters ["慶", "諝", "語", "訖"] that are not used in conventional corpora are selected as auxiliary characters. These characters are all from the set of obscure characters outside the General Standard Chinese Character Table, and their Unicode codes are: 慶 (U+6176), 諝 (U+8ADD), 語 (U+8A9E), 訖 (U+8A16), and their occurrence frequencies in conventional corpora are lower than 10^-7, thus avoiding semantic conflicts with natural texts. Second, in the input processing stage, the original text is combined with the list of auxiliary information relation words to form a new input sequence. Specifically, the input format is "慶B1諝B2諝B3...訖", where B1, B2, B3, etc. are the relation words and corresponding quantities existing in the text. The output format is "慶A1諝B1語C1訖慶A2諝B2語C2訖慶A3諝B3語C3訖", where A is the head entity, B is the relation, and C is the tail entity. "慶" is used as the start symbol of the triple, "諝" is used as the separator between the head entity and the relation, "語" is used as the separator between the relation and the tail entity, and "訖" is used as the end symbol of the triple. For example, for the text "SH is used to send TDMA frame numbers and BSIC information to the mobile station", the knowledge triples are "(SH, send, TDMA frame number)(SH, send, BSIC information)". After processing, the input is "慶发送諝发送訖SH is used to send TDMA frame numbers and BSIC information to the mobile station", and the output result is "慶SH諝发送語TDMA frame number訖慶SH諝发送語BSIC information訖". Through this structured encoding mechanism, the model can effectively identify and extract knowledge triples, and at the same time solve the problems of fuzzy semantic boundaries and confused relations in traditional NER schemes. In addition, this method also has the technical advantages of structure explicitization, multi-hop reasoning support, and error propagation suppression, and can significantly improve the accuracy and robustness of the knowledge extraction task in the field of network operation and maintenance.

[0035] Example 3 In some embodiments, the selection of the AIGC model is based on the model's performance, applicability, and support ability for the knowledge extraction task in the field of Chinese network operation and maintenance. The present invention selects the Randeng-T5-Char-57M-Chinese model developed by the Institute of Digital Economy of the Guangdong-Hong Kong-Macao Greater Bay Area (IDEA Institute) as the base model. This model is a Chinese pre-trained model based on the Google T5 architecture, has good text generation and conversion capabilities, and is suitable for the knowledge triple extraction task in the present invention.

[0036] The model selection criteria mainly include the following points: First, considering that data in the network operation and maintenance field has highly specialized characteristics and specific semantic features, the model needs to have strong Chinese understanding capabilities and the ability to recognize professional terms; second, traditional NER methods such as BERT combined with CRF have limitations when dealing with cases where the output answer does not appear in the original text, while generative AIGC models can effectively solve this problem through structured encoding mechanisms; third, the model needs to have sufficient flexibility and scalability to support the rare Chinese character-assisted structured encoding mechanism proposed in this invention, thereby achieving accurate extraction of triples.

[0037] The Randeng-T5-Char-57M-Chinese model features a unified T5 text transformation model architecture, incorporating a Transformer encoder and decoder, making it flexible for various Chinese NLP tasks such as text generation, question answering, and summarization. For pre-training, the model utilizes a large-scale Chinese corpus, covering encyclopedias, news, forums, and other content, totaling approximately 57M Chinese characters, providing broad coverage and rich semantic knowledge. Furthermore, the model employs Corpus Adaptive Pre-training (CAPT) technology, further optimized on the Wudao Corpus (180G version), resulting in superior performance on Chinese tasks compared to other mainstream Chinese pre-trained models, such as T5-small.

[0038] To better adapt to Chinese tasks, this model uses only BertTokenizer and supports both Chinese and English vocabulary lists, improving its ability to understand Chinese words. These features enable the Randeng-T5-Char-57M-Chinese model to efficiently complete the knowledge triple extraction task in this invention, especially performing well when dealing with complex and discontinuous entities and relationships in the field of network operations and maintenance.

[0039] Example 4 In some implementations, the construction of an AIGC model applied to NER includes the following steps: First, the input text is processed by combining the original text with a list of auxiliary information relation words to form a new input sequence. The relation words in the auxiliary information relation word list are used to indicate the structure and content of the triples, helping the model to more accurately identify entities and their relationships. The input processing strategy uses special characters as delimiters to isolate different parts of the triples, thereby enhancing the model's ability to extract knowledge triples.

[0040] The data processing flow includes data loading, data partitioning, data shaping, and data input augmentation. The data loading stage uses the Pandas library to read a "text-knowledge triple" format dataset from the network operations and maintenance domain. The data partitioning stage divides the dataset into training, validation, and test sets. The data shaping stage uses a function to convert the original knowledge triples into a uniform table format and cleans and standardizes irregular data. The data input augmentation stage extracts relation words for each training sample from the auxiliary information relation word list and inserts them into the original text according to a specific format to form a new input sequence.

[0041] The model training and inference process is based on fine-tuning of the pre-trained Randeng-T5-Char-57M-Chinese model. During training, labeled training data is processed and then input into the pre-trained model. The model parameters are adjusted using the P-Tuning method to adapt to the needs of knowledge extraction tasks in the network operations and maintenance domain. After training, the model performance is evaluated on the validation set, and the best-performing model is saved for subsequent inference.

[0042] In the model inference phase, the text data from the test set is processed through the same data processing flow and then input into the pre-trained tuned model. The model outputs a set of knowledge triples containing a head entity, relation, and tail entity. Predefined separators are used in the output to separate the different parts of the triples, facilitating subsequent knowledge extraction and analysis. Finally, the model's predictions are compared with the corresponding answers in the test set to evaluate the model's performance. This method effectively solves the problem that traditional NER schemes cannot handle output answers that do not appear in the original text, improving the accuracy and completeness of knowledge triple extraction.

[0043] The sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0044] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of different embodiments are consistent and can be referenced by each other. The technical features of different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0045] Those skilled in the art will recognize that, based on the units and algorithm steps described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0046] The apparatus provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings. The description of the apparatus embodiments corresponds to the description of the method embodiments. Therefore, for content not described in detail, please refer to the method embodiments above. For the sake of brevity, some content will not be repeated.

[0047] This application also provides a computer-readable storage medium storing computer instructions for implementing the methods performed by the devices or apparatuses in the above method embodiments.

[0048] For example, when the computer program is executed by a computer, it enables the computer to implement the methods performed by the devices or apparatus in the various embodiments of the above methods.

[0049] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0050] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0051] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of apparatus or units may be electrical, mechanical, or other forms.

[0052] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0053] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0054] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0055] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A knowledge extraction method, characterized in that, include: Based on the obtained original text, determine the list of relational words contained in the original text; The original text is concatenated based on the list of relation words and the preset first type of Chinese character set to obtain the input sequence; The input sequence is input into a pre-trained text processing model so that the text processing model generates a corresponding output sequence based on the input sequence. The output sequence contains at least one knowledge triple corresponding to the original text. In the output sequence, at least one first-class Chinese character from the first-class Chinese character set is used as a delimiter to segment each knowledge triple contained in the output sequence. The output sequence is parsed to obtain at least one knowledge triple corresponding to the original text.

2. The method according to claim 1, characterized in that, The first type of Chinese character set is selected from rare characters outside the "General Standard Chinese Character List", and the frequency of their occurrence in regular corpora is lower than a preset threshold.

3. The method according to claim 1, characterized in that, The step of concatenating the original text according to the list of relation words and at least one preset set of first-class Chinese characters to obtain the input sequence specifically includes: Each relation word in the relation word list is concatenated with the original text to obtain the initial input sequence; Based on at least one first-class Chinese character from the first-class Chinese character set, the relation words in the initial input sequence are segmented with the original text to obtain the input sequence.

4. The method according to claim 1, characterized in that, The step of using at least one of the first type of Chinese characters as a delimiter to segment the knowledge triples contained in the output sequence specifically includes: Use the first Chinese character in the first type of Chinese character set as the starting identifier of the knowledge triple; The second Chinese character in the first type of Chinese character set is used as the separator between the subject and predicate in the knowledge triple; The third Chinese character in the first type of Chinese character set is used as the separator between the predicate and the object in the knowledge triple; The fourth Chinese character in the first type of Chinese character set is used as the termination identifier of the knowledge triple.

5. The method according to claim 1, characterized in that, The text processing model is built on the Transformer architecture encoder-decoder model.

6. The method according to claim 5, characterized in that, Pre-training the text processing model specifically includes: Obtain a training dataset, which includes multiple training samples, each of which includes the original text and the corresponding knowledge triplet label; Based on the original text of the training samples, determine the list of relational words; Based on the list of relation words and the preset first set of Chinese characters, the original text of each training sample is concatenated to obtain the training input sequence; Based on the knowledge triplet labels and the preset first set of Chinese characters, a training output sequence is generated, wherein the first set of Chinese characters is used as a delimiter to segment the knowledge triplets; The pre-trained model based on the Transformer architecture is fine-tuned using the training input sequence and training output sequence to obtain the text processing model.

7. A knowledge extraction device, characterized in that, include: The text acquisition module is used to acquire raw text; The relation word determination module is used to determine one or more relation words contained in the original text to form a relation word list; The sequence construction module is used to concatenate the original text according to the list of relation words and the preset first type of Chinese character set to obtain the input sequence, wherein the first type of Chinese character set consists of rare characters; The model processing module is used to input the input sequence into a pre-trained text processing model to generate a corresponding output sequence. The output sequence contains at least one knowledge triple corresponding to the original text, separated by Chinese characters from the first type of Chinese character set. The result parsing module is used to parse the output sequence and extract at least one knowledge triple.

8. The apparatus according to claim 7, characterized in that, The sequence construction module is specifically used for: Each relation word in the relation word list is concatenated with the original text to obtain the initial input sequence; Based on at least one first-class Chinese character from the first-class Chinese character set, the relation words in the initial input sequence are segmented with the original text to obtain the input sequence.

9. An electronic device, characterized in that, include: One or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to perform the method as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 6.

Images