Method and device for constructing multi-level power field knowledge base

Abstract

The application discloses a method and device for constructing a multi-level power field knowledge base, and relates to the field of artificial intelligence, and aims to improve the field knowledge base constructed based on the existing method, which is not good for improving the reliability of large language model application. The method comprises the following steps: separating text and images from original data to obtain field knowledge original text and field knowledge original images; performing adaptive semantic text block segmentation on the original text to construct a semantic text block knowledge base; performing double-channel collaborative knowledge extraction on the text blocks, and fusing the extraction results to construct a knowledge graph knowledge base; performing adaptive text description generation on the field knowledge original images to construct an image and text description knowledge base; performing vector conversion and mapping alignment on the combination of the field knowledge text blocks, the knowledge graph triples, the field knowledge original images and the corresponding text descriptions to construct a semantic vector knowledge base; and constructing a multi-level field knowledge base based on the above multiple knowledge bases.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a multi-level power field knowledge base construction method and device. BACKGROUND

[0002] In today's digital age, knowledge services have become a key factor in driving efficient development and innovation in various industries. Knowledge services refer to the provision of accurate, efficient, and personalized knowledge support and solutions to users through the combination of information technology and professional knowledge. It covers multiple aspects such as knowledge retrieval, question answering, and writing, aiming to help users quickly access the information they need, solve complex problems, and improve the efficiency of knowledge creation and dissemination. With the rapid development of artificial intelligence technology, large language models have made significant achievements in natural language processing, exhibiting strong language generation and understanding capabilities, including semantic understanding, task generalization, knowledge versatility, modal innovation generation, and human-computer interaction. These comprehensive capabilities give large models broad application potential in various industries, especially in knowledge services. Due to the versatility of large language models, there are limitations in specific domain tasks, especially in the power industry, which is highly specialized. There are problems such as insufficient mastery of deep professional knowledge and inaccurate understanding of domain logic, resulting in reduced application reliability. To solve this problem, attaching a power domain knowledge base to a large language model has become a mainstream approach.

[0003] Currently, one of the most representative methods for constructing a power domain knowledge base is the multi-granularity retrieval-based knowledge base construction method, which specifically involves splitting domain knowledge raw text according to different granularities, and then converting the domain knowledge units obtained by splitting into vectors to form a vector knowledge base.

[0004] However, since existing methods mainly focus on multi-granularity splitting, they lack deep understanding of domain knowledge, resulting in low application reliability of large language models with attached power domain knowledge bases. SUMMARY

[0005] Therefore, the present application provides a multi-level power domain knowledge base construction method and device, aiming to improve the power domain knowledge base constructed based on existing methods, which is not effective in improving the application reliability of large language models.

[0006] According to one aspect of the present application, a multi-level power domain knowledge base construction method is provided, comprising:

[0007] Obtain domain knowledge raw data in the power domain, perform text-image separation processing on the domain knowledge raw data, and obtain domain knowledge raw text and domain knowledge raw images;

[0008] perform adaptive semantic text block segmentation processing on the domain knowledge raw text to obtain a plurality of domain knowledge text blocks, and construct a semantic text block knowledge base based on the plurality of domain knowledge text blocks;

[0009] respectively perform double-channel collaborative knowledge extraction processing on each of the domain knowledge text blocks to obtain a plurality of open knowledge triples and a plurality of controlled knowledge triples, and perform fusion processing on the plurality of open knowledge triples and the plurality of controlled knowledge triples to obtain a plurality of knowledge graph triples, and construct a knowledge graph knowledge base based on the plurality of knowledge graph triples;

[0010] respectively perform adaptive text description generation processing on each of the domain knowledge raw images to obtain a plurality of combinations of domain knowledge raw images and corresponding text descriptions, and construct an image and text description knowledge base based on the plurality of combinations of domain knowledge raw images and corresponding text descriptions;

[0011] respectively perform vector conversion processing and mapping alignment processing on each of the domain knowledge text blocks, each of the knowledge graph triples, and each of the combinations of domain knowledge raw images and corresponding text descriptions to obtain a plurality of mapped and aligned vectors, and construct a semantic vector knowledge base based on the plurality of mapped and aligned vectors;

[0012] construct a multi-level power domain knowledge base based on the semantic text block knowledge base, the knowledge graph knowledge base, the image and text description knowledge base, and the semantic vector knowledge base.

[0013] According to another aspect of the present application, a multi-level power domain knowledge base construction device is provided, comprising:

[0014] a text image separation module configured to obtain domain knowledge raw data of a power domain, and perform text image separation processing on the domain knowledge raw data to obtain domain knowledge raw text and domain knowledge raw images;

[0015] a semantic text block knowledge base construction module configured to perform adaptive semantic text block segmentation processing on the domain knowledge raw text to obtain a plurality of domain knowledge text blocks, and construct a semantic text block knowledge base based on the plurality of domain knowledge text blocks;

[0016] a knowledge graph knowledge base construction module configured to respectively perform double-channel collaborative knowledge extraction processing on each of the domain knowledge text blocks to obtain a plurality of open knowledge triples and a plurality of controlled knowledge triples, and perform fusion processing on the plurality of open knowledge triples and the plurality of controlled knowledge triples to obtain a plurality of knowledge graph triples, and construct a knowledge graph knowledge base based on the plurality of knowledge graph triples;

[0017] The image and text description knowledge base construction module is configured to perform adaptive text description generation processing on each of the domain knowledge original images respectively, to obtain a plurality of sets of combinations of the domain knowledge original images and corresponding text descriptions, and to construct an image and text description knowledge base based on the plurality of sets of combinations of the domain knowledge original images and corresponding text descriptions.

[0018] The semantic vector knowledge base construction module is configured to perform vector conversion processing and mapping alignment processing on each of the domain knowledge text blocks, each of the knowledge graph triples, and each set of combinations of the domain knowledge original images and corresponding text descriptions respectively, to obtain a plurality of mapped and aligned vectors, and to construct a semantic vector knowledge base based on the plurality of mapped and aligned vectors.

[0019] The multi-level power domain knowledge base construction module is configured to construct a multi-level power domain knowledge base based on the semantic text block knowledge base, the knowledge graph knowledge base, the image and text description knowledge base, and the semantic vector knowledge base.

[0020] According to another aspect of the present application, a storage medium is provided, and the storage medium stores at least one executable instruction, which causes a processor to perform operations corresponding to the construction method of the multi-level power domain knowledge base.

[0021] According to still another aspect of the present application, a terminal is provided, which comprises a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete communication with each other through the communication bus.

[0022] The memory is configured to store at least one executable instruction, which causes the processor to perform operations corresponding to the construction method of the multi-level power domain knowledge base.

[0023] Through the above technical solutions, the technical solutions provided by the embodiments of the present application have at least the following advantages:

[0024] The application provides a multi-level power field knowledge base construction method and device. Compared with the prior art, the field knowledge original text is divided into a plurality of independent semantic units through adaptive semantic segmentation processing, which provides a basis for accurate understanding and analysis; the field knowledge structure is displayed in the form of a graph through knowledge extraction and triple fusion processing, improving the convenience of query and reasoning; the field knowledge original image and its text description are combined through adaptive text description generation processing, expanding the application range; the multi-modal field knowledge is quantified in the form of a vector through vector conversion and alignment processing, improving the matching efficiency; further, the above-constructed semantic text block knowledge base, knowledge graph knowledge base, image and text description knowledge base, and semantic vector knowledge base are combined to construct a multi-level field knowledge base, through the mutual cooperation between the plurality of knowledge bases, the multi-modal field knowledge query is realized, the accuracy of the query result is improved, and therefore the application reliability of the large language model externally connected with the multi-level field knowledge base is effectively improved.

[0025] The above description is only a summary of the technical solutions of the application. In order to enable the technical means of the application to be more clearly understood and implemented according to the content of the specification, and in order to enable the above and other purposes, characteristics and advantages of the application to be more apparent and easy to understand, the following specific embodiments of the application are described. BRIEF DESCRIPTION OF DRAWINGS

[0026] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not meant to limit the present application. Furthermore, the same reference numerals are used throughout the several drawings to designate the same or similar parts. In the drawings:

[0027] Figure 1 A multi-level power field knowledge base construction method flowchart provided by an embodiment of the application is shown;

[0028] Figure 2 A field knowledge text block segmentation flowchart provided by an embodiment of the application is shown;

[0029] Figure 3 A knowledge graph triple construction flowchart provided by an embodiment of the application is shown;

[0030] Figure 4 A construction flowchart of the combination of field knowledge original images and corresponding text descriptions provided by an embodiment of the application is shown;

[0031] Figure 5 A vector construction flowchart provided by an embodiment of the application is shown;

[0032] Figure 6A block diagram of a device for constructing a multi-level power field knowledge base is shown.

[0033] Figure 7 A structural schematic diagram of a terminal is shown. DETAILED DESCRIPTION

[0034] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0035] The present embodiment provides a method for constructing a multi-level power field knowledge base, as shown in the figure, the method comprises: Figure 1

[0036] 101, acquire domain knowledge raw data in the power field, perform text-image separation processing on the domain knowledge raw data to obtain domain knowledge raw text and domain knowledge raw images.

[0037] Among them, in addition to the power field, any field can be targeted, for example, medical, legal, financial and other fields with strong professionalism; the domain knowledge raw data can be multi-source heterogeneous data; the text-image separation processing is used to separate the image and text content contained in the domain knowledge raw data. In the present embodiment, the current execution end can be a plug-in database construction module of a large language model.

[0038] It should be noted that separating the text and the image provides support for subsequent construction of multi-level knowledge modeling based on text and image as data basis.

[0039] 102, perform adaptive semantic text block segmentation processing on the domain knowledge raw text to obtain a plurality of domain knowledge text blocks, and construct a semantic text block knowledge base based on the plurality of domain knowledge text blocks.

[0040] Among them, the adaptive semantic text block segmentation processing is a processing on the text data separated in the embodiment step 101, specifically, the segmentation is performed based on the segmentation total score of each set of adjacent sentence pairs in the domain knowledge raw text as the segmentation point, and the adaptivity is reflected in that the weight coefficient used when calculating the segmentation total score is dynamically adjusted according to the term density change rate trend of the adjacent sentence pairs.

[0041] ​It should be noted that the adaptive semantic text block segmentation processing can effectively overcome the disadvantages brought by fixed length segmentation or single similarity threshold segmentation, intelligently adjust the sensitivity of the block by dynamically adjusting the calculation weight coefficient of the segmentation total score, more accurately capture the context semantic boundary, avoid the situation of semantic break or information loss caused by improper sentence breaking, and thus improve the rationality of the text block segmentation.

[0042] 103. respectively, each domain knowledge text block is subjected to double-channel collaborative knowledge extraction processing, to obtain a plurality of open knowledge triples and a plurality of controlled knowledge triples, and the plurality of open knowledge triples and the plurality of controlled knowledge triples are subjected to fusion processing to obtain a plurality of knowledge graph triples, and a knowledge graph knowledge base is constructed based on the plurality of knowledge graph triples.

[0043] Among them, the double-channel collaborative knowledge extraction processing includes open knowledge extraction processing and controlled knowledge extraction processing; the open knowledge extraction processing is used to discover new knowledge and expand the domain boundary, which can improve the knowledge coverage and flexibility; the controlled knowledge extraction processing is used to standardize the accuracy and standardization of knowledge expression, which can improve the structural consistency and application reliability; the open knowledge triple is obtained by open knowledge extraction processing on each domain knowledge text block; the controlled knowledge triple is obtained by controlled knowledge extraction processing on each domain knowledge text block; the fusion processing is used to fuse the open knowledge triple and the controlled knowledge triple on the semantic level, which can realize semantic complementation, guarantee the breadth of knowledge extraction, and strengthen the knowledge quality, which can specifically include knowledge triple fusion processing, conflict resolution processing, entity alignment processing and entity disambiguation processing, the knowledge triple fusion processing is used to map and fuse the open entity and the open relationship of the open knowledge triple to the controlled entity and the controlled relationship of the controlled knowledge triple matched with the open knowledge triple, the conflict resolution processing is used to eliminate the knowledge triples with conflicts, the entity alignment processing is used to align the same name different entity, and the entity disambiguation processing is used to distinguish the different name same entity.

[0044] 104. respectively, each domain knowledge original image is subjected to adaptive text description generation processing to obtain a plurality of groups of domain knowledge original images and corresponding text description combinations, and an image and text description knowledge base is constructed based on the plurality of groups of domain knowledge original images and corresponding text description combinations.

[0045] The adaptive text description generation process is used to generate a text description corresponding to the domain knowledge original image. Specifically, prompt words required by a large language model are constructed according to knowledge graph triples matched with the domain knowledge original image, and then the large model is used to generate a text description corresponding to the domain knowledge original image based on the prompt words. The adaptiveness is reflected in that the matched knowledge graph triples, that is, the most relevant domain knowledge, can be dynamically retrieved and injected from the knowledge graph knowledge base according to the context information of the domain knowledge original image, and the retrieved knowledge graph triples are used to construct the prompt words, so as to ensure that the generated description always focuses on the domain core and is professional and accurate in content. This is a dynamic interaction process with the knowledge graph knowledge base constructed in step 103 of the embodiment, and is not an isolated image-to-text conversion process.

[0046] It should be noted that the image information and the domain knowledge can be deeply fused through the adaptive text description generation process.

[0047] 105, respectively, each domain knowledge text block, each knowledge graph triple, and each group of domain knowledge original images and corresponding text descriptions are subjected to vector conversion processing and mapping alignment processing to obtain a plurality of mapped and aligned vectors, so as to construct a semantic vector knowledge base based on the plurality of mapped and aligned vectors.

[0048] The vector conversion processing is used to independently convert the domain knowledge text block, the knowledge graph triple, and the combination of the domain knowledge original image and the corresponding text description into a vector in a respective vector space. The mapping alignment processing is used to map the vector of the knowledge graph triple and the vector of the combination of the domain knowledge original image and the corresponding text description to the vector space of the vector of the domain knowledge text block, so that the domain knowledge of different sources and modalities has comparability and correlation in the same high-dimensional semantic vector space, thereby supporting cross-modal and cross-level precise retrieval.

[0049] 106, based on the semantic text block knowledge base, the knowledge graph knowledge base, the image and text description knowledge base, and the semantic vector knowledge base, a multi-level power domain knowledge base is constructed.

[0050] In the embodiment of the application, the above-mentioned semantic text block knowledge base, knowledge graph knowledge base, image and text description knowledge base, and semantic vector knowledge base are combined to construct a multi-level domain knowledge base. Through the mutual cooperation between the multiple knowledge bases, when a user queries in natural language, not only semantically similar text blocks can be retrieved, but also related knowledge graph triples and matching images can be accurately retrieved, thereby truly realizing seamless fusion and efficient utilization of multi-level and multi-modal domain knowledge.

[0051] Compared with the prior art, the embodiment of the application divides the domain knowledge original text into multiple independent semantic units through adaptive semantic segmentation processing, providing a basis for accurate understanding and analysis; through knowledge extraction and triple fusion processing, the domain knowledge structure is displayed in the form of a graph, improving the convenience of querying and reasoning; through adaptive text description generation processing, the domain knowledge original image is combined with its text description, expanding the application range; through vector conversion and alignment processing, the multi-modal domain knowledge is quantified in the form of a vector, improving the matching efficiency; further, the above-constructed semantic text block knowledge base, knowledge graph knowledge base, image and text description knowledge base, and semantic vector knowledge base are combined to construct a multi-level domain knowledge base, through the mutual cooperation between multiple knowledge bases, the multi-modal domain knowledge query is realized, the accuracy of the query result is improved, and thus the application reliability of the large language model with the multi-level domain knowledge base is effectively improved.

[0052] In one embodiment of the application, in order to further limit and illustrate, as shown in Figure 2 The embodiment step 102 performs adaptive semantic text block segmentation processing on the domain knowledge original text to obtain multiple domain knowledge text blocks, including:

[0053] 201. Divide multiple groups of adjacent sentence pairs, and take each group of adjacent sentence pairs as a target adjacent sentence pair one by one.

[0054] Correspondingly, the embodiment step 201 specifically includes: performing sentence splitting processing on the domain knowledge original text to obtain multiple sentences; performing adjacent sentence pair division processing on the multiple sentences to obtain multiple groups of adjacent sentence pairs, and taking each group of adjacent sentence pairs as a target adjacent sentence pair one by one.

[0055] Among them, the target adjacent sentence pair contains a first target sentence and a second target sentence; the sentence splitting processing is used for sentence-level splitting of the domain knowledge original text, that is, splitting at the end of each sentence (for example, at the position of a period, an exclamation mark, etc.), to obtain multiple independent sentences, for example, splitting "The arc-drop monitoring of the conductor is one of the important indicators for the state evaluation of the power transmission line." and "The indicator can be uploaded to the monitoring center in real time through the OPGW optical fiber communication network." into two sentences; the adjacent sentence pair division processing is used for dividing each two adjacent sentences into an adjacent sentence pair, such as sentence 1 and sentence 2, sentence 2 and sentence 3, sentence 3 and sentence 4, etc., to obtain multiple groups of adjacent sentence pairs.

[0056] It should be noted that before the sentence splitting process is performed, the domain knowledge original text can also be subjected to structure recognition processing based on layout analysis technology to determine different areas such as the main text, figure description, table content, footnotes, page numbers, etc. Further, the text content can be extracted based on OCR technology, and the non-text content such as the header, footer, and page number can be filtered to ensure the semantic coherence and rationality of the reading order of the extracted content. For example, after the text in the "XX Operation and Maintenance Regulations" is subjected to OCR processing, the non-text information such as "XX Power Grid Company" and "page X" is removed, and the technical description part is retained.

[0057] 202. Calculate the semantic similarity score of the target adjacent sentence pair.

[0058] Correspondingly, the embodiment step 202 specifically includes: performing sentence semantic vector conversion processing on the first target sentence and the second target sentence respectively to obtain the first target sentence semantic vector and the second target sentence semantic vector, and calculating the semantic similarity between the first target sentence semantic vector and the second target sentence semantic vector to obtain the semantic similarity score of the target adjacent sentence pair.

[0059] Among them, the sentence semantic vector conversion processing is used to encode the semantics of the sentence to generate a high-dimensional semantic vector to capture the semantic relationship between sentences, for example, to generate high-dimensional semantic vector representations of "the conductor vertical arc degree anomaly may cause the line trip." and "the optical fiber communication network can realize the remote transmission of state data." respectively; the semantic similarity score is used to represent the semantic similarity between the two sentences in each adjacent sentence pair, which reflects the degree of semantic association between adjacent sentences, and the cosine similarity can be used as a measurement index.

[0060] It can be understood that a higher similarity (such as exceeding a similarity threshold) indicates that the two in the adjacent sentence pair are strongly related in semantics, i.e., they are in the same semantic segment; otherwise, it indicates weak semantic association, and there may be a semantic boundary, for example, sentence A is "the conductor vertical arc degree anomaly may cause the line trip.", sentence B is "the optical fiber communication network can realize the remote transmission of state data.", the semantic similarity between the two is 0.35, which is lower than the preset threshold 0.6, so it is considered that the semantic association is weak, which indicates that there may be a semantic boundary.

[0061] 203. Calculate the term density change rate score of the target adjacent sentence pair.

[0062] Correspondingly, the embodiment step 203 specifically includes: obtaining a domain term dictionary in the power field, respectively counting the number of domain terms in the first target sentence and the second target sentence based on the domain term dictionary to obtain the first domain term number and the second domain term number, and respectively calculating the proportion of the domain term number in the number of sentence terms to obtain the first term density and the second term density, and calculating the term density change rate between the first term density and the second term density to obtain the term density change rate score of the target adjacent sentence pair.

[0063] Wherein, the domain term dictionary is used to record common terms in the domain, for example, the power grid domain term dictionary includes terms such as conductor, OPGW, sag, trip, optical fiber communication, monitoring system, etc.; the term density change rate score is used to represent the term density change rate between the first term density and the second term density, which is used to reflect the mutation of the use of terms between adjacent sentences. The greater the term density change, the more likely the context has a shift in theme or technical focus. For example, the first target sentence contains 2 terms, and the number of sentence terms is 15, so the first term density is 2 / 15=0.13. Similarly, the second term density is calculated, which is assumed to be 0.02. The term density change rate is It should be noted that when the term density change rate between the first term density and the second term density is large, such as exceeding the term density change rate threshold, it means that there is a high possibility of a semantic breakpoint between the two sentences.

[0064] 204, determine the logical association degree score of the target adjacent sentence pair.

[0065] Correspondingly, the embodiment step 204 specifically includes: obtaining a logical term dictionary, and determining the logical association degree score of the target adjacent sentence pair based on the logical term dictionary. If the first word of the second target sentence is a logical term, the logical association degree score of the target adjacent sentence pair is 0, and if the first word of the second target sentence is a non-logical term, the logical association degree score of the target adjacent sentence pair is 1.

[0066] Wherein, the logical term dictionary is used to record common logical terms, for example, definition, explanation, for example, conclusion, therefore, however, in addition, first, second, etc.; the logical association degree score is used to indicate whether the first word of the second target sentence is a logical term, which is used to enhance the recognition ability of the continuity of the text context. Specifically, if it is a logical term, the logical association degree score of the target adjacent sentence pair is 0, and if it is a non-logical term, the logical association degree score of the target adjacent sentence pair is 1.

[0067] It should be noted that when the first word is a logical word, it means that the two sentences are logically strongly related and have a higher possibility of being in the same text block, otherwise, they are logically weakly related and have a lower possibility of being in the same text block, for example, when a sentence is "In summary, the main reason for the line trip is the abnormal sag." Since it starts with the logical word "In summary", it can be determined as a context continuity point and marked as a merging candidate.

[0068] 205、determine the weight coefficient matched with the target adjacent sentence pair.

[0069] Correspondingly, the step 205 specifically comprises: determining a term density change rate trend between the first group of adjacent sentence pairs and the target adjacent sentence pair according to the term density change rate scores of the multiple groups of adjacent sentence pairs between the first group of adjacent sentence pairs and the target adjacent sentence pair, and determining the weight coefficient matched with the target adjacent sentence pair according to the term density change rate trend.

[0070] Wherein, the weight coefficient is dynamically adjusted according to the term density change rate trend, specifically, the term density change rate scores of the multiple groups of adjacent sentence pairs between the first group of adjacent sentence pairs and the target adjacent sentence pair are obtained to determine the term density change rate trend between the first group of adjacent sentence pairs and the target adjacent sentence pair, if the term density change rate trend is continuously rising, it means that the text theme or focus is shifting, at this time, the weight coefficient corresponding to the difference between 1 and the semantic similarity score can be increased, so as to dynamically improve the sensitivity to slight decline in semantic similarity, and ensure that the semantic boundary generated by content mutation can be accurately captured.

[0071] In addition, the weight coefficient corresponding to the term density change rate score can also be dynamically adjusted according to the average term density of the text, for example, when the average term density is low, it means that it is in a non-core discussion area, at this time, the weight coefficient corresponding to the term density change rate score can be reduced to avoid the case of misjudging as a breakpoint due to the accidental appearance of a small amount of terms; on the contrary, in the core discussion area with high term density, the weight coefficient corresponding to the term density change rate score is kept or appropriately increased. At the same time, when multiple logical words appear continuously, it means that the text is in a paragraph with a dense logical structure, at this time, the weight coefficient corresponding to the logical association degree score can be reduced to ensure that the complete expression is retained in one text block. It can be understood that the sum of the weight coefficients is always 1.

[0072] 206、based on the weight coefficient, calculate the weighted sum between the term density change rate score, the logical association degree score and the difference between 1 and the semantic similarity score to obtain the segmentation total score of the target adjacent sentence pair.

[0073] In the embodiment of the application, the segmentation total score calculation formula is represented by the following formula,

[0074]

[0075] wherein, denotes the segmentation total score of the adjacent sentence pair, denotes the weight coefficient corresponding to the semantic similarity score, denotes the semantic similarity score between the sentence and the sentence . denotes the weight coefficient corresponding to the term density change rate score, denotes the term density change rate score, denotes the weight coefficient corresponding to the logical association degree score, denotes the logical association degree score.

[0076] 207、obtain the segmentation total score of each group of adjacent sentence pairs as the text block segmentation point, and respectively compare each segmentation total score with a preset segmentation total score threshold.

[0077] 208、if the segmentation total score is greater than the preset segmentation total score threshold, perform text block segmentation processing at the end of the first sentence in the adjacent sentence pair corresponding to the segmentation total score.

[0078] 209、until the threshold comparison and the text block segmentation processing are completed for each group of adjacent sentence pairs, a plurality of domain knowledge text blocks are obtained.

[0079] In the embodiment steps 207-209, when the segmentation total score is greater than the preset segmentation total score threshold, it indicates that the probability that the two sentences in the current adjacent sentence pair belong to different text blocks is relatively high. At this time, the position between the two sentences is taken as the text block segmentation point, and the text block segmentation processing is performed until the threshold comparison and the text block segmentation processing are completed for all adjacent sentence pairs, and a plurality of domain knowledge text blocks are obtained.

[0080] As a case corresponding to the embodiment step 208, if the segmentation total score is less than or equal to the preset segmentation total score threshold, the adjacent sentence pair corresponding to the segmentation total score does not need to be segmented.

[0081] In an embodiment of the present application, in order to further limit and illustrate, as shown in Figure 3 , the embodiment step 103 respectively performs double-channel collaborative knowledge extraction processing on each domain knowledge text block to obtain a plurality of open knowledge triples and a plurality of controlled knowledge triples, and performs fusion processing on the plurality of open knowledge triples and the plurality of controlled knowledge triples to obtain a plurality of knowledge graph triples, so as to construct a knowledge graph knowledge base based on the plurality of knowledge graph triples, including:

[0082] 301、Obtain a first prompt word, and based on the first prompt word, respectively, perform open knowledge extraction processing on each domain knowledge text block to obtain a plurality of open knowledge triples.

[0083] Among them, the first prompt word includes a domain knowledge element example and an extraction requirement, which is used to guide the model output. Exemplarily,

[0084] Please extract all possible domain knowledge elements from the following text, including but not limited to: 1. Entities (such as people, organizations, terms, products, technologies, etc.) and their types; 2. Relationships (such as "invent", "depend", "affect", "belong to", etc.) and associated entity pairs; 3. Events (such as "release", "cooperate", "malfunction", etc.) and participants, time, and results; 4. Attributes (such as performance indicators, prices, rules, etc.).

[0085] Requirements:

[0086] 1. Output in a structured format (triples SPO formatted dictionary list), format as follows:

[0087] {

[0088] "subject": {"value": "Entity A", "type": "Entity Type"},

[0089] "predicate": "Relationship Name",

[0090] "object": {"value": "Entity B", "type": "Entity Type"}

[0091] }

[0092] 2. Preserve implicit but logically reasonable knowledge in the original text (such as "X is compatible with Y" can infer "X has a technical relationship with Y");

[0093] 3. The output should be a dictionary list (JSON List) and not include other explanatory text.

[0094] It should be noted that the open knowledge extraction processing used in the embodiments of the present application automatically identifies and extracts entities, relationships, events, and attributes for each domain knowledge text block, without the need for pre-defined ontologies or knowledge models. It can flexibly cope with emerging terminology and complex sentence structures, and is suitable for the cold start stage of professional fields. In addition, the format of the output open knowledge triples can be set to a structured SPO triple dictionary list, ensuring consistency in subsequent processing. Furthermore, during open knowledge extraction processing, implicit but logically reasonable knowledge in the original text can be preserved, thereby enhancing the completeness and practicality of the extraction results.

[0095] 302、cluster the plurality of open knowledge triples to obtain the domain ontology of the power field.

[0096] Correspondingly, the embodiment step 302 specifically comprises: integrating open entities and open relations in the plurality of open knowledge triples respectively to obtain an open entity set and an open relation set, and respectively calculating open entity semantic similarity between any two open entities in the open entity set, and dividing the two open entities into the same open entity cluster when the open entity semantic similarity exceeds a preset open entity semantic similarity threshold, to obtain a plurality of open entity clusters, and respectively calculating open relation semantic similarity between any two open relations in the open relation set, and dividing the two open relations into the same open relation cluster when the open relation semantic similarity exceeds a preset open relation semantic similarity threshold, to obtain a plurality of open relation clusters, and respectively performing induction processing on each open entity cluster and each open relation cluster to generate entity types and relation types, so as to obtain the domain ontology of the power field.

[0097] In the embodiment of the application, the large model can be used to perform induction processing on each open entity cluster and each open relation cluster. For example, the prompt words used for inducing the open entity cluster are as follows:

[0098] You are a domain knowledge standardization assistant. Please perform semantic type induction on the entity cluster according to the following input information.

[0099] Input information: entity cluster list: {"candidates":[ ]}

[0100] Your task is:

[0101] - analyze the semantic commonality of all entities;

[0102] - induce and output a most standardized and most general entity type;

[0103] - provide a selection reason. The output format is as follows:

[0104] {

[0105] "category":"",

[0106] "reason":""

[0107] }

[0108] Wherein, represents the list of open entity clusters to be induced.

[0109] The prompt words used for inducing the open relation cluster are as follows:

[0110] You are a domain knowledge standardization assistant. Please induce the semantic types of the relation clusters based on the following input information.

[0111] Input information: Relation cluster list: {"candidates": [ ]}

[0112] Your task is:

[0113] - Analyze the semantic commonalities of all relations;

[0114] - Induce and output the most standardized and general relation type;

[0115] - Provide selection reasons. The output format is as follows:

[0116] {

[0117] "relation_type": "",

[0118] "definition": "",

[0119] "subject_constraint": "",

[0120] "object_constraint": ""

[0121] }

[0122] Where, represents the open relation cluster list to be induced.

[0123] The final ontology structure is reviewed by experts or automatically adjusted through an iterative optimization mechanism. After data format conversion, the following simple domain ontology can be obtained, for example:

[0124] Entity types include: organization, technology, monitoring parameter, communication method, application function, event.

[0125] Relation types include:

[0126] 1. Propose: subject is organization, object is technology;

[0127] 2. Promote content: subject is technology, object is technology;

[0128] 3. Monitor content: subject is technology, object is monitoring parameter;

[0129] 4. Data transmission method: subject is technology, object is communication method;

[0130] 5. Implement application: subject is technology, object is application function;

[0131] 6. Occurs: Subject is organization, object is event;

[0132] 7. Proposes time: Subject is technology, object is event.

[0133] 303、Based on the domain ontology, a second prompt word is constructed, and based on the second prompt word, controlled knowledge extraction processing is performed on each domain knowledge text block respectively to obtain a plurality of controlled knowledge triples.

[0134] The first prompt word includes a domain ontology Schema and extraction requirements, and is used to guide the model output. For example,

[0135] Please extract knowledge from the text according to the following domain knowledge model, and only output the content that meets the model definition.

[0136] Domain model constraints:

[0137] Entity types include: organization, technology, monitoring parameter, communication method, application function, and event.

[0138] Relationship types include:

[0139] 1. Proposes: Subject is organization, object is technology;

[0140] 2. Promotes content: Subject is technology, object is technology;

[0141] 3. Monitoring content: Subject is technology, object is monitoring parameter;

[0142] 4. Data transmission method: Subject is technology, object is communication method;

[0143] 5. Realizes application: Subject is technology, object is application function;

[0144] 6. Occurs: Subject is organization, object is event;

[0145] 7. Proposes time: Subject is technology, object is event.

[0146] Requirements:

[0147] 1. Strictly match the entity types and relationship types in the model, and ignore irrelevant information in the model;

[0148] 2. Normalize the identified entities (for example, "OPGW optical fiber network" is unified as "optical fiber communication", and "dynamic capacity increase" remains as "dynamic capacity increase", etc.);

[0149] 3. The output result is presented in the form of a structured SPO triple dictionary list, each triple contains a subject, a predicate, and an object; each subject and object is an object containing two fields: value (value) and type (type), and the output format is as follows:

[0150] {

[0151] "subject": {"value": "entity A", "type": "entity type"},

[0152] "predicate": "relation name",

[0153] "object": {"value": "Entity B", "type": "Entity Type"}

[0154] }

[0155] 4. If a field is missing, please mark it as "NULL";

[0156] 5. It does not output any explanatory text, but only returns a list of knowledge triples that meet the format requirements.

[0157] Preferably, the output format of the controlled knowledge triples is set to be consistent with the output format of the open knowledge triples, that is, a structured SPO triple dictionary list, to ensure consistency in subsequent processing.

[0158] 304. Perform knowledge triple fusion processing on multiple open knowledge triples and multiple controlled knowledge triples to obtain multiple fused knowledge triples.

[0159] Accordingly, step 304 of the embodiment specifically includes: integrating the controlled entities and controlled relations in multiple controlled knowledge triples to obtain a set of controlled entities and a set of controlled relations; calculating the entity mapping semantic similarity between the open entity and each controlled entity in the controlled entity set for each open entity; mapping the open entity to the controlled entity corresponding to the maximum entity mapping semantic similarity when the maximum entity mapping semantic similarity exceeds a preset entity mapping semantic similarity threshold, and receiving the type label of the controlled entity; and calculating the relationship mapping semantic similarity between the open relation and each controlled relation in the controlled relation set for each open relation; mapping the open relation to the controlled relation corresponding to the maximum relationship mapping semantic similarity when the maximum relationship mapping semantic similarity exceeds a preset relationship mapping semantic similarity threshold, and receiving the type label of the controlled relation, thereby obtaining multiple fused knowledge triples.

[0160] In the embodiments of the present application, the open entity is mapped to the controlled entity corresponding to the maximum entity mapping semantic similarity, and the type label of the controlled entity is received, that is, the open entity is converted into the standard naming form used in controlled knowledge extraction, and the corresponding entity type label is assigned; similarly, the open relationship is mapped to the controlled relationship corresponding to the maximum relationship mapping semantic similarity, and the type label of the controlled relationship is received, that is, the open relationship is converted into the standard relationship used in controlled knowledge extraction, and the corresponding relationship type label is assigned. In addition, if the maximum entity mapping semantic similarity does not exceed the preset entity mapping semantic similarity threshold, it is retained as an extended entity for subsequent review or model iteration optimization, and the relationship mapping is the same. For example, the open knowledge extraction may identify entities such as “OPGW optical cable”, “OPGW communication line”, and “optical fiber composite overhead ground wire”. Through the above method, the entities are mapped to the entity “optical fiber communication” in the controlled knowledge extraction, and the type label “communication mode” is inherited.

[0161] 305. The plurality of fused knowledge triples are subjected to conflict resolution processing to obtain a plurality of conflict-resolved knowledge triples.

[0162] Correspondingly, the step 305 of the embodiment specifically comprises: screening conflict knowledge triple pairs from the plurality of fused knowledge triples, calculating the confidence scores of the two fused knowledge triples contained in each conflict knowledge triple pair, and screening out conflict knowledge triples with low confidence scores to obtain a plurality of conflict-resolved knowledge triples.

[0163] Among them, the conflict knowledge triple pair is used to represent two knowledge triples with conflicting content; the confidence score is the weighted sum of the confidence parameter output by the large model and the triple source parameter, and the triple source includes controlled knowledge extraction and open knowledge extraction, which can be expressed as the following formula,

[0164]

[0165] Among them, represents the confidence score of the knowledge triple , represents the confidence parameter of the knowledge triple , represents the weight parameter corresponding to the confidence parameter, represents the triple source parameter. For example, two triples A (from open knowledge extraction) and B (from controlled knowledge extraction) are identified, and the confidence scores are 0.5 and 0.8 respectively. If the confidence threshold is set to 0.6, then triple B is retained and triple A is excluded.

[0166] In addition, a reliability score threshold can be preset. When the confidence scores of both knowledge triples are less than the preset reliability score threshold, a large model can be used for secondary judgment and semantic correction to improve the accuracy of conflict resolution. For example, if the two confidence scores are calculated to be 0.4 and 0.5 respectively, and the confidence threshold is still set to 0.6, then a large model needs to be introduced to perform secondary judgment and semantic correction. The prompt is as follows: Please judge and correct the conflicting triples according to the input content.

[0167] Your task is:

[0168] 1. Analyze the consistency of content and the reasonableness of context between the two triples;

[0169] 2. Determine which triplet best fits the meaning of the original text;

[0170] 3. If neither is accurate, try generating a new, more reasonable triplet;

[0171] 4. Output the final selected or modified triplet, and indicate the basis for selection or the reason for modification.

[0172] The output format is as follows:

[0173] {

[0174] "selected_triple": {

[0175] "subject": {"value": "entity A", "type": "entity type"},

[0176] "predicate": "relation name",

[0177] "object": {"value": "Entity B", "type": "Entity Type"}

[0178] },

[0179] "reason": ""

[0180] }

[0181] Input content: {The two conflicting triples to be judged and their source text}

[0182] 306. Perform entity alignment processing on multiple conflict-resolved knowledge triples to obtain multiple sets of entity-aligned knowledge triples.

[0183] Correspondingly, the embodiment step 306 specifically includes: integrating the entities in the plurality of conflict-resolved knowledge triples to obtain an entity set, respectively calculating the entity semantic similarity between any two entities in the entity set, and when the entity semantic similarity is greater than a preset entity semantic similarity threshold, merging the two entities to obtain a plurality of entity-aligned knowledge triples.

[0184] In the embodiments of the present application, the merging processing can specifically be constructing a synonymous entity cluster, selecting the highest frequency entity name in the synonymous entity cluster as the primary naming form of the synonymous entity cluster, and taking other entity names as aliases, and establishing a mapping relationship between the aliases and the knowledge triples; if there are multiple entity names with the same frequency, a large model can be used for selection, and the prompt words are as follows:

[0185] You are a domain knowledge standardization assistant responsible for selecting the most appropriate primary naming form from multiple synonymous entity names.

[0186] The input is a set of synonymous entity names and their context information. Please judge and select the most suitable entity name as the primary naming according to the naming normativity, domain consistency, semantic clarity and naming conciseness.

[0187] The input information includes:

[0188] 1. Entity name candidate list: {"candidates": [ ]}

[0189] 2. Context fragments where the entity appears:

[0190] Your task is:

[0191] - Analyze the normativity of the candidate names in the domain terminology;

[0192] - Determine which name best fits industry standards or common usage habits;

[0193] - If multiple names are equally normative, choose the name with the clearest semantics and the simplest structure;

[0194] - Output the final selected primary naming form and explain the selection reason.

[0195] The output format is as follows:

[0196] {

[0197] "preferred_name": "",

[0198] "reason": ""

[0199] }

[0200] in, This represents a candidate list of entity names. This represents the context in which the entity appears.

[0201] 307. Perform entity disambiguation processing on the knowledge triples after multiple sets of entity alignment to obtain multiple knowledge graph triples.

[0202] Accordingly, step 307 of the embodiment specifically includes: distinguishing entities with the same name but different meanings in the knowledge triples after multiple entity alignment, so as to complete entity disambiguation processing and obtain multiple knowledge graph triples.

[0203] In one embodiment of this application, for further definition and explanation, such as Figure 4 As shown, in step 104 of the embodiment, adaptive text description generation processing is performed on the original images of each domain knowledge to obtain a combination of multiple sets of original images of domain knowledge and corresponding text descriptions, including:

[0204] 401. Take the original images of knowledge from each domain as the original images of the target domain knowledge one by one; obtain the associated information that matches the original images of the target domain knowledge.

[0205] The relevant information includes, but is not limited to, the title above the original image of the target domain knowledge, the caption below it, and the semantic block content of the chapter in which the image is located, etc., in the original domain knowledge data. For example, the caption below the target "insulator string" image is "Schematic diagram of the structure of a new type of anti-pollution flashover insulator string".

[0206] 402. Based on the association information, perform a retrieval operation in the knowledge graph knowledge base to obtain knowledge graph triples that match the original image of the target domain knowledge.

[0207] In this embodiment, the relevant information obtained in step 401 of the embodiment is used as keywords to perform a search operation in the knowledge graph knowledge base, and knowledge graph triplets that match the original image of the target domain knowledge are obtained, such as <insulator, function, electrical connection between conductor and tower>, <insulator, material, ceramic>, <flashover, cause, insulator surface contamination leads to a decrease in insulation performance under humid conditions>, <anti-flashover, measures, adopting a large creepage distance design or applying RTV coating>.

[0208] 403. Serialize the matching knowledge graph triples to obtain the knowledge graph triple description text, and construct a third prompt word that matches the original image of the target domain knowledge based on the knowledge graph triple description text.

[0209] For example, the four triplets in step 402 of the embodiment are serialized to obtain the description text: "Related field knowledge: The function of an insulator is to isolate the electrical connection between the conductor and the tower, and it is usually made of ceramic material. Pollution flashover refers to the phenomenon that the insulation performance of an insulator deteriorates due to contamination on its surface under humid conditions. Anti-pollution flashover measures include adopting a large creepage distance design or applying RTV coating." The description text is then concatenated with a general prompt word template to obtain the third prompt word.

[0210] 404. Based on the third prompt word, perform text description processing on the original image of the target domain knowledge to obtain a combination of the original image of the target domain knowledge and the corresponding text description.

[0211] 405. Obtain a combination of multiple sets of original images of domain knowledge and their corresponding text descriptions.

[0212] The third prompt word is obtained by concatenating and fusing the description requirements and the knowledge graph triple description text. For example,

[0213] You are a professional image description generation assistant. Please carefully observe the image below and generate a natural and fluent Chinese description based on its content. Requirements are as follows:

[0214] 1. The description should cover the main objects, scenes, actions, or events in the image.

[0215] 2. If characters are present, please describe their behavior, emotions, or interactions as much as possible.

[0216] 3. Pay attention to environmental details, such as time, place, weather, and background.

[0217] 4. If there is text, logo, or special symbols in the image, please try to identify and explain them.

[0218] 5. Avoid subjective assumptions, maintain objectivity and accuracy, and use vivid and descriptive language.

[0219] 6. Keep the description between 100 and 200 words to ensure the information is complete, concise, and clear.

[0220] 7. Please refer to the following domain knowledge and incorporate it naturally into your description:

[0221] Related knowledge: The function of insulators is to isolate the electrical connection between conductors and towers, and they are usually made of ceramic. Pollution flashover refers to the phenomenon where contamination on the insulator surface degrades its insulation performance under humid conditions. Anti-pollution flashover measures include using large creepage distance designs or applying RTV coatings.

[0222] Please output the results starting with "Image description:".

[0223] In one embodiment of this application, for further definition and explanation, such as Figure 5 As shown, in step 105 of the embodiment, vector transformation and mapping alignment are performed on the combination of each domain knowledge text block, each knowledge graph triple, and each set of original domain knowledge images and corresponding text descriptions to obtain multiple mapped and aligned vectors, including:

[0224] 501. Perform semantic vector transformation on the text blocks of knowledge in each domain to obtain semantic vectors of multiple domain knowledge text blocks.

[0225] In this context, the semantic vector of the domain knowledge text block is a high-dimensional dense semantic vector. For example, the sentence "Insulators are important components in power transmission lines. Their main function is to achieve electrical isolation between conductors and towers to prevent current leakage. Common insulator materials include ceramics and glass." is transformed into a high-dimensional dense semantic vector.

[0226] 502. Based on the knowledge graph embedding algorithm, each knowledge graph triple is processed by knowledge graph embedding vector transformation to obtain multiple knowledge graph embedding vectors.

[0227] Specifically, each knowledge graph triple is taken as the target knowledge graph triple; based on the knowledge graph embedding algorithm, the entities of the target knowledge graph triple are converted into entity vectors, and the relations are converted into relation vectors; based on the knowledge graph embedding algorithm, the knowledge graph embedding vector of the target knowledge graph triple is generated according to the entity vector and relation vector, resulting in multiple knowledge graph embedding vectors.

[0228] In this embodiment of the application, the knowledge graph embedding vector is a single vector of complete facts obtained through weighting or concatenation.

[0229] 503. Perform image-text joint vector transformation on the combination of the original images and corresponding text descriptions of each group of domain knowledge to obtain multiple image-text joint vectors.

[0230] Accordingly, step 503 of the embodiment specifically includes: for each combination of original domain knowledge images and corresponding text descriptions, based on the image vector generation model that has completed model training, performing image vector transformation processing on the original domain knowledge images to obtain image vectors, and based on the text description vector generation model that has completed model training, performing text description vector transformation processing on the text descriptions to obtain text description vectors, and concatenating or weighting the image vectors and text description vectors to generate a combined image-text vector corresponding to the combination of original domain knowledge images and corresponding text descriptions, thus obtaining multiple combined image-text vectors.

[0231] In one embodiment of this application, to further define and illustrate the method, before performing image vector conversion processing on the original domain knowledge images based on the image vector generation model that has completed model training, and before performing text description vector conversion processing on the text descriptions based on the text description vector generation model that has completed model training, the method further includes: acquiring a combination of multiple sets of training domain knowledge images and corresponding training text descriptions; constructing an initial image vector generation model and an initial text description vector generation model; performing image vector conversion processing on each training domain knowledge image based on the initial image vector generation model to obtain multiple training image vectors; and performing text description vector conversion processing on each training text description based on the initial text description vector generation model. This description vector transformation process yields multiple training text description vectors. The similarity between each training image vector and each training text description vector is calculated, generating a similarity matrix. In this matrix, diagonal elements represent the similarity of positive samples, and off-diagonal elements represent the similarity of negative samples. Positive samples represent combinations of training domain knowledge images and their corresponding training text descriptions, while negative samples represent combinations of training domain knowledge images and other training text descriptions besides their corresponding descriptions. Based on the similarity matrix, the joint loss function of the initial image vector generation model and the initial text description vector generation model is calculated. The joint loss function is expressed by the following formula.

[0232] ,

[0233] in, This represents the value of the joint loss function. Let i represent the training image vector of the training domain knowledge image. Representing training domain knowledge images The corresponding training text description, This represents the similarity between positive samples, and τ represents the temperature coefficient. Indicates the first Other training text descriptions;

[0234] The model parameters of the initial image vector generation model and the initial text description vector generation model are adjusted to minimize the joint loss function value, resulting in an image vector generation model and a text description vector generation model that have been trained. Image vector conversion processing is performed based on the image vector generation model that has been trained, and text description vector conversion processing is performed based on the text description vector generation model that has been trained.

[0235] It should be noted that the above model training process is based on a cross-modal contrastive learning algorithm, which makes the image vectors and text description vectors of the matching images and text generated by the image vector generation model and the text description vector generation model closer in the vector space.

[0236] 504. Based on the vector mapping model that has been trained, each knowledge graph embedding vector and each graph-text joint vector are mapped to the vector space where the semantic vector of the domain knowledge text block is located, resulting in multiple mapped vectors.

[0237] In this embodiment of the application, the knowledge graph embedding vector and the graph-text joint vector are mapped to the vector space where the semantic vector of the domain knowledge text block is located, so as to keep the three types of vectors consistent in semantics.

[0238] In one embodiment of this application, to further define and illustrate, before mapping each knowledge graph embedding vector and each graph-text joint vector to the vector space where the semantic vector of the domain knowledge text block resides, based on the vector mapping model that has completed model training, the method of the embodiment further includes: selecting multiple anchor knowledge points, and respectively obtaining the domain knowledge text block semantic vector, knowledge graph embedding vector, and graph-text joint vector corresponding to each anchor knowledge point, wherein the anchor knowledge points are used to represent the domain knowledge that exists in the semantic text block knowledge base, the knowledge graph knowledge base, and the image and text description knowledge base; constructing an initial vector mapping model, and Based on the initial vector mapping model, the knowledge graph embedding vectors and graph-text joint vectors of each anchor knowledge are mapped to the vector space containing the semantic vectors of the domain knowledge text blocks, resulting in multiple mapped knowledge graph embedding vectors and multiple mapped graph-text joint vectors. The loss values ​​between the multiple mapped knowledge graph embedding vectors and multiple mapped graph-text joint vectors and the multiple domain knowledge text block semantic vectors are calculated. The model parameters of the initial vector mapping model are adjusted to minimize the loss value, resulting in a vector mapping model based on the completed model training. Vector space mapping processing is then performed using the vector mapping model based on the completed model training.

[0239] In this embodiment, the loss function can be the mean squared error loss function, which ensures that the semantically consistent knowledge graph embedding vector and the semantic vector of the domain knowledge text block in the graph-text joint vector are semantically consistent.

[0240] 505. Determine multiple mapped vectors and multiple semantic vectors of domain knowledge text blocks as multiple mapped and aligned vectors.

[0241] In this embodiment of the application, the mapped vector and the semantic vector of the domain knowledge text block are collectively referred to as the mapped and aligned vector.

[0242] In one embodiment of this application, for further definition and explanation, before obtaining the original domain knowledge data in the power field in step 101, the method further includes: collecting domain knowledge in the power field from multiple data sources; performing format conversion processing on each domain knowledge to obtain domain knowledge in multiple standard formats; and performing cleaning and integration processing on each standard format domain knowledge to obtain the original domain knowledge data.

[0243] The cleaning process includes redundancy removal and noise removal, such as removing redundant symbols, advertising information, HTML tags, non-verbal text, and duplicate content. It also includes automatic correction of grammar and spelling. Data sources include, but are not limited to, domain-specific data (text files, PDF documents), industry websites, and open data platforms. For web page data, automated tools such as Selenium can be used for data scraping. Format conversion processing is used to convert data into a standard, unified encoding format.

[0244] This application provides a method for constructing a multi-level knowledge base in the power industry. Compared with existing technologies, it divides the original text of domain knowledge into multiple independent semantic units through adaptive semantic segmentation, providing a foundation for accurate understanding and analysis. Through knowledge extraction and triplet fusion, it displays the domain knowledge structure in graph form, improving the convenience of querying and reasoning. Through adaptive text description generation, it combines the original image of domain knowledge with its text description, expanding the application scope. Through vector transformation and alignment, it quantifies multimodal domain knowledge in vector form, improving matching efficiency. Furthermore, it combines the constructed semantic text block knowledge base, knowledge graph knowledge base, image and text description knowledge base, and semantic vector knowledge base to construct a multi-level domain knowledge base. Through the cooperation between multiple knowledge bases, it realizes the query of multimodal domain knowledge, improves the accuracy of query results, and effectively enhances the reliability of the application of a large language model with an external multi-level domain knowledge base.

[0245] Furthermore, as a response to the above Figure 1 The implementation of the method shown in this application provides a device for constructing a multi-level knowledge base in the power field, such as... Figure 6 As shown, the device includes:

[0246] Text image separation module 61, semantic text block knowledge base construction module 62, knowledge graph knowledge base construction module 63, image and text description knowledge base construction module 64, semantic vector knowledge base construction module 65, multi-level power field knowledge base construction module 66;

[0247] The text-image separation module 61 is used to acquire the original domain knowledge data in the power field, and perform text-image separation processing on the original domain knowledge data to obtain the original domain knowledge text and the original domain knowledge image.

[0248] The semantic text block knowledge base construction module 62 is used to perform adaptive semantic text block segmentation processing on the original text of the domain knowledge to obtain multiple domain knowledge text blocks, so as to construct a semantic text block knowledge base based on the multiple domain knowledge text blocks.

[0249] The knowledge graph knowledge base construction module 63 is used to perform dual-channel collaborative knowledge extraction processing on each domain knowledge text block to obtain multiple open knowledge triples and multiple controlled knowledge triples, and to perform fusion processing on the multiple open knowledge triples and multiple controlled knowledge triples to obtain multiple knowledge graph triples, so as to construct a knowledge graph knowledge base based on the multiple knowledge graph triples.

[0250] The image and text description knowledge base construction module 64 is used to perform adaptive text description generation processing on the original images of each domain knowledge respectively to obtain a combination of multiple sets of original images of domain knowledge and corresponding text descriptions, so as to construct an image and text description knowledge base based on the combination of multiple sets of original images of domain knowledge and corresponding text descriptions.

[0251] The semantic vector knowledge base construction module 65 is used to perform vector transformation and mapping alignment processing on each of the domain knowledge text blocks, each of the knowledge graph triples, and each combination of the original images of the domain knowledge and the corresponding text descriptions to obtain multiple mapped and aligned vectors, and to construct a semantic vector knowledge base based on the multiple mapped and aligned vectors.

[0252] The multi-level power domain knowledge base construction module 66 is used to construct a multi-level power domain knowledge base based on the semantic text block knowledge base, the knowledge graph knowledge base, the image and text description knowledge base, and the semantic vector knowledge base.

[0253] In specific application scenarios, the semantic text block knowledge base construction module includes:

[0254] The segmentation total score calculation unit is used to calculate the segmentation total score of each group of adjacent sentence pairs in the original text of the domain knowledge as text block segmentation points. The segmentation total score is a weighted sum of the term density change rate score, the logical relevance score, and the difference between 1 and the semantic similarity score. The weight coefficients used to calculate the weighted sum are dynamically adjusted according to the trend of the term density change rate of adjacent sentence pairs.

[0255] The segmentation total score threshold comparison unit is used to compare each of the segmentation total scores with a preset segmentation total score threshold;

[0256] A text block segmentation unit is used to perform text block segmentation processing at the end of the first statement in the adjacent statement pair corresponding to the total segmentation score if the total segmentation score is greater than the preset total segmentation score threshold.

[0257] The text block segmentation unit is also used to obtain multiple domain knowledge text blocks until all adjacent statement pairs have completed threshold comparison and text block segmentation processing.

[0258] In specific application scenarios, the segmentation total score calculation unit is used for:

[0259] The original text of the domain knowledge is split into multiple sentences.

[0260] The multiple statements are divided into adjacent statement pairs to obtain multiple groups of adjacent statement pairs. Each group of adjacent statement pairs is then used as a target adjacent statement pair. The target adjacent statement pair includes a first target statement and a second target statement.

[0261] The first target statement and the second target statement are processed by statement semantic vector transformation respectively to obtain the semantic vector of the first target statement and the semantic vector of the second target statement. The semantic similarity between the semantic vector of the first target statement and the semantic vector of the second target statement is calculated to obtain the semantic similarity score of the target adjacent statement pair.

[0262] A domain terminology dictionary for the power industry is obtained. Based on the domain terminology dictionary, the number of domain terms contained in the first target statement and the second target statement are counted to obtain the number of first domain terms and the number of second domain terms. The proportion of the number of domain terms to the number of words in the statement is calculated to obtain the first term density and the second term density. The term density change rate between the first term density and the second term density is calculated to obtain the term density change rate score of the target adjacent statement pair.

[0263] Obtain a logical word dictionary, and based on the logical word dictionary, determine the logical association score of the target adjacent sentence pair. If the first word of the second target sentence is a logical word, the logical association score of the target adjacent sentence pair is determined to be 0. If the first word of the second target sentence is a non-logical word, the logical association score of the target adjacent sentence pair is determined to be 1.

[0264] Based on the term density change rate scores of multiple adjacent statement pairs from the first adjacent statement pair to the target adjacent statement pair, the term density change rate trend from the first adjacent statement pair to the target adjacent statement pair is determined, and the weight coefficient matching the target adjacent statement pair is determined based on the term density change rate trend.

[0265] Based on the weighting coefficients, a weighted sum is calculated between the term density change rate score, the logical relevance score, and the difference between 1 and the semantic similarity score to obtain the total segmentation score of the target adjacent sentence pair.

[0266] In specific application scenarios, the knowledge graph knowledge base construction module includes:

[0267] A dual-channel collaborative knowledge extraction unit is used to perform open knowledge extraction processing on each of the domain knowledge text blocks based on a first prompt word to obtain multiple open knowledge triples, and to perform controlled knowledge extraction processing on each of the domain knowledge text blocks based on a second prompt word to obtain multiple controlled knowledge triples. The second prompt word is constructed based on the domain ontology obtained by clustering the multiple open knowledge triples.

[0268] The knowledge triple fusion unit is used to integrate controlled entities and controlled relations in multiple controlled knowledge triples to obtain a set of controlled entities and a set of controlled relations. For each open entity, it calculates the entity mapping semantic similarity between the open entity and each controlled entity in the controlled entity set. When the maximum entity mapping semantic similarity exceeds a preset entity mapping semantic similarity threshold, it maps the open entity to the controlled entity corresponding to the maximum entity mapping semantic similarity and receives the type label of the controlled entity. For each open relation, it calculates the relationship mapping semantic similarity between the open relation and each controlled relation in the controlled relation set. When the maximum relationship mapping semantic similarity exceeds a preset relationship mapping semantic similarity threshold, it maps the open relation to the controlled relation corresponding to the maximum relationship mapping semantic similarity and receives the type label of the controlled relation, thus obtaining multiple fused knowledge triples.

[0269] The conflict resolution unit is used to select conflicting knowledge triple pairs from multiple fused knowledge triple pairs. For each conflicting knowledge triple pair, the confidence scores of the two fused knowledge triple pairs contained in the conflicting knowledge triple pair are calculated, and the conflicting knowledge triple pairs with low confidence scores are filtered out to obtain multiple conflict-resolved knowledge triple pairs.

[0270] An entity alignment unit is used to integrate entities in multiple conflict-resolved knowledge triples to obtain an entity set, calculate the semantic similarity between any two entities in the entity set, and merge the two entities when the semantic similarity is greater than a preset semantic similarity threshold to obtain multiple sets of entity-aligned knowledge triples.

[0271] The entity disambiguation unit is used to distinguish entities with the same name but different meanings in multiple knowledge triples after entity alignment, so as to complete the entity disambiguation process and obtain multiple knowledge graph triples.

[0272] In specific application scenarios, the dual-channel collaborative knowledge extraction unit is used for:

[0273] Obtain the first prompt word, and based on the first prompt word, perform open knowledge extraction processing on each of the domain knowledge text blocks to obtain multiple open knowledge triples, wherein the first prompt word includes examples of domain knowledge elements and extraction requirements;

[0274] The open entities and open relations in multiple open knowledge triples are integrated to obtain open entity sets and open relation sets. The semantic similarity between any two open entities in the open entity set is calculated. When the semantic similarity exceeds a preset open entity semantic similarity threshold, the two open entities are assigned to the same open entity cluster, resulting in multiple open entity clusters. Similarly, the semantic similarity between any two open relations in the open relation set is calculated. When the semantic similarity exceeds a preset open relation semantic similarity threshold, the two open relations are assigned to the same open relation cluster, resulting in multiple open relation clusters. Each open entity cluster and each open relation cluster is then subjected to inductive processing to generate entity types and relation types, thereby obtaining a domain ontology for the power field.

[0275] Based on the domain ontology, a second prompt word is constructed, and based on the second prompt word, controlled knowledge extraction processing is performed on each of the domain knowledge text blocks to obtain multiple controlled knowledge triples.

[0276] In specific application scenarios, the image and text description knowledge base construction module is used for:

[0277] Each of the original images of knowledge in the aforementioned domains is used as the original image of knowledge in the target domain.

[0278] Obtain association information that matches the original image of the target domain knowledge;

[0279] Based on the association information, a retrieval operation is performed in the knowledge graph knowledge base to obtain a knowledge graph triplet that matches the original image of the target domain knowledge.

[0280] The matched knowledge graph triples are serialized to obtain knowledge graph triple description text, and a third prompt word matching the original image of the target domain knowledge is constructed based on the knowledge graph triple description text.

[0281] Based on the third prompt word, the original image of the target domain knowledge is processed with text description to obtain a combination of the original image of the target domain knowledge and the corresponding text description.

[0282] This yields a combination of multiple sets of original images of domain knowledge and their corresponding text descriptions.

[0283] In specific application scenarios, the semantic vector knowledge base construction module includes:

[0284] The text block semantic vector conversion unit is used to perform text block semantic vector conversion processing on each of the domain knowledge text blocks to obtain multiple domain knowledge text block semantic vectors.

[0285] The knowledge graph embedding vector conversion unit is used to perform knowledge graph embedding vector conversion processing on each of the knowledge graph triples based on the knowledge graph embedding algorithm to obtain multiple knowledge graph embedding vectors.

[0286] The image-text joint vector conversion unit is used to perform image vector conversion processing on the original images of the domain knowledge and the corresponding text descriptions for each combination of original images of domain knowledge and the corresponding text descriptions, based on the image vector generation model that has been trained, to obtain image vectors, and to perform text description vector conversion processing on the text descriptions based on the text description vector generation model that has been trained, to obtain text description vectors, and to concatenate or weight the image vectors and the text description vectors to generate image-text joint vectors corresponding to the combination of original images of the domain knowledge and the corresponding text descriptions, thus obtaining multiple image-text joint vectors;

[0287] The vector mapping unit is used to map each of the knowledge graph embedding vectors and each of the graph-text joint vectors to the vector space where the semantic vectors of the domain knowledge text blocks are located, based on the vector mapping model that has been trained, to obtain multiple mapped vectors.

[0288] The vector mapping unit is further configured to determine multiple mapped vectors and multiple semantic vectors of domain knowledge text blocks as multiple mapped and aligned vectors.

[0289] In specific application scenarios, prior to the image-text joint vector conversion unit, the semantic vector knowledge base construction module further includes a first model training unit, used for:

[0290] Obtain a combination of multiple sets of training domain knowledge images and corresponding training text descriptions;

[0291] Construct an initial image vector generation model and an initial text description vector generation model;

[0292] Based on the initial image vector generation model, image vector transformation processing is performed on each of the training domain knowledge images to obtain multiple training image vectors;

[0293] Based on the initial text description vector generation model, each of the training text descriptions is transformed into a text description vector to obtain multiple training text description vectors.

[0294] The similarity between each training image vector and each training text description vector is calculated respectively, and a similarity matrix is ​​generated. The diagonal elements in the similarity matrix represent the similarity of positive samples, and the off-diagonal elements in the similarity matrix represent the similarity of negative samples. Positive samples are used to represent the combination of training domain knowledge images and corresponding training text descriptions, and negative samples are used to represent the combination of training domain knowledge images and other training text descriptions besides the corresponding training text descriptions.

[0295] Based on the similarity matrix, the joint loss function value of the initial image vector generation model and the initial text description vector generation model is calculated. The joint loss function is expressed as follows:

[0296] ,

[0297] in, This represents the value of the joint loss function. Let i represent the training image vector of the training domain knowledge image. Representing training domain knowledge images The corresponding training text description, This represents the similarity between positive samples, and τ represents the temperature coefficient. Indicates the first Other training text descriptions;

[0298] The model parameters of the initial image vector generation model and the initial text description vector generation model are adjusted to minimize the joint loss function value, thereby obtaining an image vector generation model and a text description vector generation model that have completed model training. Image vector conversion processing is performed based on the image vector generation model that has completed model training, and text description vector conversion processing is performed based on the text description vector generation model that has completed model training.

[0299] In specific application scenarios, before the vector mapping unit, the semantic vector knowledge base construction module further includes a second model training unit, used for:

[0300] Multiple anchor knowledge points are selected, and the semantic vector of the domain knowledge text block, the knowledge graph embedding vector, and the image-text joint vector corresponding to each anchor knowledge point are obtained respectively. The anchor knowledge points are used to represent the domain knowledge that exists in the semantic text block knowledge base, the knowledge graph knowledge base, and the image and text description knowledge base.

[0301] An initial vector mapping model is constructed, and based on the initial vector mapping model, the knowledge graph embedding vector and graph-text joint vector of each anchor point knowledge are mapped to the vector space where the semantic vector of the domain knowledge text block is located, to obtain multiple mapped knowledge graph embedding vectors and multiple mapped graph-text joint vectors.

[0302] Calculate the loss values ​​between the multiple mapping knowledge graph embedding vectors and the multiple mapping graph-text joint vectors, and the multiple domain knowledge text block semantic vectors;

[0303] The model parameters of the initial vector mapping model are adjusted to minimize the loss value, resulting in a vector mapping model based on the completed model training. Vector space mapping is then performed based on the vector mapping model based on the completed model training.

[0304] This application provides a device for constructing a multi-level knowledge base in the power field. Compared with existing technologies, it divides the original text of domain knowledge into multiple independent semantic units through adaptive semantic segmentation, providing a foundation for accurate understanding and analysis; it displays the domain knowledge structure in graph form through knowledge extraction and triple fusion, improving the convenience of querying and reasoning; it combines the original image of domain knowledge with its text description through adaptive text description generation, expanding the application scope; it quantifies multimodal domain knowledge in vector form through vector transformation and alignment, improving matching efficiency; furthermore, it combines the constructed semantic text block knowledge base, knowledge graph knowledge base, image and text description knowledge base, and semantic vector knowledge base to construct a multi-level domain knowledge base. Through the cooperation between multiple knowledge bases, it realizes the query of multimodal domain knowledge, improves the accuracy of query results, and thus effectively improves the application reliability of a large language model with an external multi-level domain knowledge base.

[0305] According to one embodiment of this application, a storage medium is provided, the storage medium storing at least one executable instruction, which can execute the method for constructing a multi-level power domain knowledge base in any of the above method embodiments.

[0306] Figure 7 The diagram shows a structural schematic of a terminal according to one embodiment of the present application. The specific embodiments of the present application do not limit the specific implementation of the terminal.

[0307] like Figure 7 As shown, the terminal may include: a processor 702, a communications interface 704, a memory 706, and a communications bus 708.

[0308] The processor 702, communication interface 704, and memory 706 communicate with each other via communication bus 708.

[0309] The communication interface 704 is used to communicate with other network elements such as clients or other servers.

[0310] The processor 702 is used to execute program 710, specifically to execute the relevant steps in the above-described embodiment of the method for constructing a multi-level power domain knowledge base.

[0311] Specifically, program 710 may include program code that includes computer operation instructions.

[0312] The processor 702 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The computer device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0313] Memory 706 is used to store program 710. Memory 706 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0314] Specifically, program 710 can be used to cause processor 702 to perform the operations in steps 101-104 of the embodiment.

[0315] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for constructing a multi-level knowledge base in the power field, characterized in that, include: Obtain raw domain knowledge data in the power field, and perform text-image separation processing on the raw domain knowledge data to obtain raw domain knowledge text and raw domain knowledge image. The original text of the domain knowledge is subjected to adaptive semantic text block segmentation to obtain multiple domain knowledge text blocks, and a semantic text block knowledge base is constructed based on the multiple domain knowledge text blocks. Each domain knowledge text block is subjected to dual-channel collaborative knowledge extraction processing to obtain multiple open knowledge triples and multiple controlled knowledge triples. The multiple open knowledge triples and multiple controlled knowledge triples are then fused to obtain multiple knowledge graph triples, and a knowledge graph knowledge base is constructed based on the multiple knowledge graph triples. Adaptive text description generation processing is performed on the original images of each domain knowledge to obtain a combination of multiple sets of original domain knowledge images and corresponding text descriptions, so as to construct an image and text description knowledge base based on the combination of multiple sets of original domain knowledge images and corresponding text descriptions. Vector transformation and mapping alignment are performed on each of the domain knowledge text blocks, each of the knowledge graph triples, and the combination of the original images of the domain knowledge and the corresponding text descriptions to obtain multiple mapped and aligned vectors. A semantic vector knowledge base is then constructed based on these multiple mapped and aligned vectors. A multi-layered power domain knowledge base is constructed based on the semantic text block knowledge base, the knowledge graph knowledge base, the image and text description knowledge base, and the semantic vector knowledge base.

2. The method according to claim 1, characterized in that, The adaptive semantic text block segmentation process of the original domain knowledge text yields multiple domain knowledge text blocks, including: The total segmentation score is calculated for each pair of adjacent sentences in the original text of the domain knowledge as the text block segmentation point. The total segmentation score is a weighted sum of the term density change rate score, the logical relevance score, and the difference between 1 and the semantic similarity score. The weight coefficients used to calculate the weighted sum are dynamically adjusted according to the trend of the term density change rate of adjacent sentence pairs. Each of the segment total scores is compared with a preset segment total score threshold; If the total segmentation score is greater than the preset total segmentation score threshold, then text block segmentation is performed at the end of the first statement in the adjacent statement pair corresponding to the total segmentation score. This process continues until all adjacent statement pairs have undergone threshold comparison and text block segmentation, resulting in multiple domain knowledge text blocks.

3. The method according to claim 2, characterized in that, The calculation of the total segmentation score for each group of adjacent sentences in the original text of the domain knowledge, which serves as the text block segmentation point, includes: The original text of the domain knowledge is split into multiple sentences. The multiple statements are divided into adjacent statement pairs to obtain multiple groups of adjacent statement pairs. Each group of adjacent statement pairs is then used as a target adjacent statement pair. The target adjacent statement pair includes a first target statement and a second target statement. The first target statement and the second target statement are processed by statement semantic vector transformation respectively to obtain the semantic vector of the first target statement and the semantic vector of the second target statement. The semantic similarity between the semantic vector of the first target statement and the semantic vector of the second target statement is calculated to obtain the semantic similarity score of the target adjacent statement pair. A domain terminology dictionary for the power industry is obtained. Based on the domain terminology dictionary, the number of domain terms contained in the first target statement and the second target statement are counted to obtain the number of first domain terms and the number of second domain terms. The proportion of the number of domain terms to the number of words in the statement is calculated to obtain the first term density and the second term density. The term density change rate between the first term density and the second term density is calculated to obtain the term density change rate score of the target adjacent statement pair. Obtain a logical word dictionary, and based on the logical word dictionary, determine the logical association score of the target adjacent sentence pair. If the first word of the second target sentence is a logical word, the logical association score of the target adjacent sentence pair is determined to be 0. If the first word of the second target sentence is a non-logical word, the logical association score of the target adjacent sentence pair is determined to be 1. Based on the term density change rate scores of multiple adjacent statement pairs from the first adjacent statement pair to the target adjacent statement pair, the term density change rate trend from the first adjacent statement pair to the target adjacent statement pair is determined, and the weight coefficient matching the target adjacent statement pair is determined based on the term density change rate trend. Based on the weighting coefficients, a weighted sum is calculated between the term density change rate score, the logical relevance score, and the difference between 1 and the semantic similarity score to obtain the total segmentation score of the target adjacent sentence pair.

4. The method according to claim 1, characterized in that, The process involves performing dual-channel collaborative knowledge extraction on each of the domain knowledge text blocks to obtain multiple open knowledge triples and multiple controlled knowledge triples. These open knowledge triples and controlled knowledge triples are then fused to obtain multiple knowledge graph triples, including: Based on the first prompt word, open knowledge extraction is performed on each of the domain knowledge text blocks to obtain multiple open knowledge triples. Based on the second prompt word, controlled knowledge extraction is performed on each of the domain knowledge text blocks to obtain multiple controlled knowledge triples. The second prompt word is constructed based on the domain ontology obtained by clustering the multiple open knowledge triples. The controlled entities and controlled relations in multiple controlled knowledge triples are integrated to obtain a controlled entity set and a controlled relation set. For each open entity, the entity mapping semantic similarity between the open entity and each controlled entity in the controlled entity set is calculated. When the maximum entity mapping semantic similarity exceeds a preset entity mapping semantic similarity threshold, the open entity is mapped to the controlled entity corresponding to the maximum entity mapping semantic similarity, and the type label of the controlled entity is received. For each open relation, the relationship mapping semantic similarity between the open relation and each controlled relation in the controlled relation set is calculated. When the maximum relationship mapping semantic similarity exceeds a preset relationship mapping semantic similarity threshold, the open relation is mapped to the controlled relation corresponding to the maximum relationship mapping semantic similarity, and the type label of the controlled relation is received, resulting in multiple fused knowledge triples. Conflicting knowledge triple pairs are selected from multiple fused knowledge triple pairs. For each pair of conflicting knowledge triple pairs, the confidence scores of the two fused knowledge triple pairs contained in the pair are calculated, and conflicting knowledge triple pairs with low confidence scores are removed to obtain multiple conflict-resolved knowledge triple pairs. By integrating entities from multiple conflict-resolved knowledge triples, an entity set is obtained. The semantic similarity between any two entities in the entity set is calculated. When the semantic similarity is greater than a preset semantic similarity threshold, the two entities are merged to obtain multiple sets of entity-aligned knowledge triples. Entities with the same name but different meanings in multiple sets of knowledge triples after entity alignment are distinguished to complete entity disambiguation processing and obtain multiple knowledge graph triples.

5. The method according to claim 4, characterized in that, Based on the first prompt word, open knowledge extraction processing is performed on each of the domain knowledge text blocks to obtain multiple open knowledge triples. Then, based on the second prompt word, controlled knowledge extraction processing is performed on each of the domain knowledge text blocks to obtain multiple controlled knowledge triples, including: Obtain the first prompt word, and based on the first prompt word, perform open knowledge extraction processing on each of the domain knowledge text blocks to obtain multiple open knowledge triples, wherein the first prompt word includes examples of domain knowledge elements and extraction requirements; The open entities and open relations in multiple open knowledge triples are integrated to obtain open entity sets and open relation sets. The semantic similarity between any two open entities in the open entity set is calculated. When the semantic similarity exceeds a preset open entity semantic similarity threshold, the two open entities are assigned to the same open entity cluster, resulting in multiple open entity clusters. Similarly, the semantic similarity between any two open relations in the open relation set is calculated. When the semantic similarity exceeds a preset open relation semantic similarity threshold, the two open relations are assigned to the same open relation cluster, resulting in multiple open relation clusters. Each open entity cluster and each open relation cluster is then subjected to inductive processing to generate entity types and relation types, thereby obtaining a domain ontology for the power field. Based on the domain ontology, a second prompt word is constructed, and based on the second prompt word, controlled knowledge extraction processing is performed on each of the domain knowledge text blocks to obtain multiple controlled knowledge triples.

6. The method according to claim 1, characterized in that, The process of performing adaptive text description generation on each of the original domain knowledge images to obtain a combination of multiple sets of original domain knowledge images and corresponding text descriptions includes: Each of the original images of knowledge in the aforementioned domains is used as the original image of knowledge in the target domain. Obtain association information that matches the original image of the target domain knowledge; Based on the association information, a retrieval operation is performed in the knowledge graph knowledge base to obtain a knowledge graph triplet that matches the original image of the target domain knowledge. The matched knowledge graph triples are serialized to obtain knowledge graph triple description text, and a third prompt word matching the original image of the target domain knowledge is constructed based on the knowledge graph triple description text. Based on the third prompt word, the original image of the target domain knowledge is processed with text description to obtain a combination of the original image of the target domain knowledge and the corresponding text description. This yields a combination of multiple sets of original images of domain knowledge and their corresponding text descriptions.

7. The method according to claim 1, characterized in that, The process involves performing vector transformation and mapping alignment on each of the domain knowledge text blocks, each of the knowledge graph triples, and each set of original domain knowledge images and corresponding text descriptions, respectively, to obtain multiple mapped and aligned vectors, including: Each of the domain knowledge text blocks is subjected to text block semantic vector transformation processing to obtain multiple domain knowledge text block semantic vectors; Based on the knowledge graph embedding algorithm, each of the knowledge graph triples is processed by knowledge graph embedding vector transformation to obtain multiple knowledge graph embedding vectors. For each combination of original domain knowledge images and corresponding text descriptions, an image vector transformation process is performed on the original domain knowledge images based on an image vector generation model that has been trained, to obtain image vectors. Then, based on a text description vector generation model that has been trained, a text description vector transformation process is performed on the text descriptions, to obtain text description vectors. Finally, the image vectors and the text description vectors are concatenated or weighted to generate a combined image-text vector corresponding to the combination of the original domain knowledge images and corresponding text descriptions, resulting in multiple combined image-text vectors. Based on the vector mapping model that has been trained, each of the knowledge graph embedding vectors and each of the graph-text joint vectors are mapped to the vector space where the semantic vector of the domain knowledge text block is located, resulting in multiple mapped vectors. The multiple mapped vectors and the multiple semantic vectors of the domain knowledge text blocks are determined as multiple mapped and aligned vectors.

8. The method according to claim 7, characterized in that, Before the image vector generation model based on the completed model training performs image vector transformation processing on the original image of the domain knowledge, and before the text description vector generation model based on the completed model training performs text description vector transformation processing on the text description, the method further includes: Obtain a combination of multiple sets of training domain knowledge images and corresponding training text descriptions; Construct an initial image vector generation model and an initial text description vector generation model; Based on the initial image vector generation model, image vector transformation processing is performed on each of the training domain knowledge images to obtain multiple training image vectors; Based on the initial text description vector generation model, each of the training text descriptions is transformed into a text description vector to obtain multiple training text description vectors. The similarity between each training image vector and each training text description vector is calculated respectively, and a similarity matrix is ​​generated. The diagonal elements in the similarity matrix represent the similarity of positive samples, and the off-diagonal elements in the similarity matrix represent the similarity of negative samples. Positive samples are used to represent the combination of training domain knowledge images and corresponding training text descriptions, and negative samples are used to represent the combination of training domain knowledge images and other training text descriptions besides the corresponding training text descriptions. Based on the similarity matrix, the joint loss function value of the initial image vector generation model and the initial text description vector generation model is calculated. The joint loss function is expressed as follows: ， in, This represents the value of the joint loss function. Let i represent the training image vector of the training domain knowledge image. Representing training domain knowledge images The corresponding training text description, This represents the similarity between positive samples, and τ represents the temperature coefficient. Indicates the first Other training text descriptions; The model parameters of the initial image vector generation model and the initial text description vector generation model are adjusted to minimize the joint loss function value, thereby obtaining an image vector generation model and a text description vector generation model that have completed model training. Image vector conversion processing is performed based on the image vector generation model that has completed model training, and text description vector conversion processing is performed based on the text description vector generation model that has completed model training.

9. The method according to claim 7, characterized in that, Before mapping each knowledge graph embedding vector and each graph-text joint vector to the vector space containing the semantic vector of the domain knowledge text block, the method based on the vector mapping model that has completed model training further includes: Multiple anchor knowledge points are selected, and the semantic vector of the domain knowledge text block, the knowledge graph embedding vector, and the image-text joint vector corresponding to each anchor knowledge point are obtained respectively. The anchor knowledge points are used to represent the domain knowledge that exists in the semantic text block knowledge base, the knowledge graph knowledge base, and the image and text description knowledge base. An initial vector mapping model is constructed, and based on the initial vector mapping model, the knowledge graph embedding vector and graph-text joint vector of each anchor point knowledge are mapped to the vector space where the semantic vector of the domain knowledge text block is located, to obtain multiple mapped knowledge graph embedding vectors and multiple mapped graph-text joint vectors. Calculate the loss values ​​between the multiple mapping knowledge graph embedding vectors and the multiple mapping graph-text joint vectors, and the multiple domain knowledge text block semantic vectors; The model parameters of the initial vector mapping model are adjusted to minimize the loss value, resulting in a vector mapping model based on the completed model training. Vector space mapping is then performed based on the vector mapping model based on the completed model training.

10. A device for constructing a multi-level knowledge base in the power field, characterized in that, include: The text-image separation module is used to acquire the original domain knowledge data in the power field, and perform text-image separation processing on the original domain knowledge data to obtain the original domain knowledge text and the original domain knowledge image. The semantic text block knowledge base construction module is used to perform adaptive semantic text block segmentation processing on the original text of the domain knowledge to obtain multiple domain knowledge text blocks, so as to construct a semantic text block knowledge base based on the multiple domain knowledge text blocks. The knowledge graph knowledge base construction module is used to perform dual-channel collaborative knowledge extraction processing on each domain knowledge text block to obtain multiple open knowledge triples and multiple controlled knowledge triples, and to perform fusion processing on the multiple open knowledge triples and multiple controlled knowledge triples to obtain multiple knowledge graph triples, so as to construct a knowledge graph knowledge base based on the multiple knowledge graph triples. The image and text description knowledge base construction module is used to perform adaptive text description generation processing on the original images of each domain knowledge respectively to obtain a combination of multiple sets of original images of domain knowledge and corresponding text descriptions, so as to construct an image and text description knowledge base based on the combination of multiple sets of original images of domain knowledge and corresponding text descriptions. The semantic vector knowledge base construction module is used to perform vector transformation and mapping alignment processing on each of the domain knowledge text blocks, each of the knowledge graph triples, and each combination of the original images of the domain knowledge and the corresponding text descriptions to obtain multiple mapped and aligned vectors, and to construct a semantic vector knowledge base based on the multiple mapped and aligned vectors. A multi-level power domain knowledge base construction module is used to construct a multi-level power domain knowledge base based on the semantic text block knowledge base, the knowledge graph knowledge base, the image and text description knowledge base, and the semantic vector knowledge base.

Images