Construction method and device of multilevel power field knowledge base

Abstract

The invention discloses a method and a device for constructing a multilevel power field knowledge base, relates to the field of artificial intelligence, and mainly aims to solve the problem that the field knowledge base constructed based on an existing method is poor in effect of improving the application reliability of a large language model. Comprising the steps of performing text image separation on original data to obtain a domain knowledge original text and a domain knowledge original image; performing adaptive semantic text block segmentation on the original text to construct a semantic text block knowledge base; performing dual-channel collaborative knowledge extraction on the text blocks, fusing extraction results, and constructing a knowledge graph knowledge base; performing adaptive text description on the domain knowledge original image to generate a constructed image and a text description knowledge base; vector conversion and mapping alignment are carried out on the domain knowledge text blocks, the knowledge graph triple, the domain knowledge original image and a combination of corresponding text descriptions to construct a semantic vector knowledge base; and constructing a multi-level domain knowledge base based on the plurality of knowledge bases.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a multi-level power field knowledge base construction method and device. BACKGROUND

[0002] In today's digital age, knowledge services have become a key factor in promoting efficient development and innovation in various industries. Knowledge services refer to the provision of accurate, efficient, and personalized knowledge support and solutions to users through the combination of information technology and professional knowledge. It covers multiple aspects such as knowledge retrieval, question answering, and writing, aiming to help users quickly access the information they need, solve complex problems, and improve the efficiency of knowledge creation and dissemination. With the rapid development of artificial intelligence technology, large language models have made significant achievements in natural language processing, demonstrating strong language generation and understanding capabilities, including semantic understanding, task generalization, knowledge universality, modal innovation generation, and human-computer interaction. These comprehensive capabilities give large models broad application potential in various industries, especially in knowledge services. Due to the universality of large language models, there are limitations in specific domain tasks, especially in the power industry, which is highly specialized. There are problems such as insufficient mastery of deep professional knowledge and inaccurate understanding of domain logic, which reduce the reliability of their application. In order to solve this problem, attaching a power domain knowledge base to a large language model has become a mainstream approach.

[0003] Currently, one of the most representative methods for constructing a power domain knowledge base is the multi-granularity retrieval-based knowledge base construction method, which specifically involves splitting domain knowledge raw text according to different granularities, and then converting the domain knowledge units obtained by splitting into vectors to form a vector knowledge base.

[0004] However, since existing methods mainly focus on multi-granularity splitting, they lack deep understanding of domain knowledge, resulting in low application reliability of large language models with attached power domain knowledge bases. SUMMARY

[0005] Therefore, the present application provides a multi-level power domain knowledge base construction method and device, which aims to improve the power domain knowledge base constructed based on existing methods, and the effect of improving the application reliability of large language models is not good.

[0006] According to one aspect of the present application, a multi-level power domain knowledge base construction method is provided, comprising: Obtaining domain knowledge raw data in the power domain, performing text-image separation processing on the domain knowledge raw data to obtain domain knowledge raw text and domain knowledge raw images; The adaptive semantic text block segmentation processing is performed on the domain knowledge original text, and a plurality of domain knowledge text blocks are obtained, so as to construct a semantic text block knowledge base based on the plurality of domain knowledge text blocks; The double-channel collaborative knowledge extraction processing is respectively performed on each of the domain knowledge text blocks, a plurality of open knowledge triples and a plurality of controlled knowledge triples are obtained, and the plurality of open knowledge triples and the plurality of controlled knowledge triples are fused to obtain a plurality of knowledge graph triples, so as to construct a knowledge graph knowledge base based on the plurality of knowledge graph triples; The adaptive text description generation processing is respectively performed on each of the domain knowledge original images, a plurality of combinations of domain knowledge original images and corresponding text descriptions are obtained, so as to construct an image and text description knowledge base based on the plurality of combinations of domain knowledge original images and corresponding text descriptions; The vector conversion processing and the mapping alignment processing are respectively performed on each of the domain knowledge text blocks, each of the knowledge graph triples, and each of the combinations of the domain knowledge original images and corresponding text descriptions, a plurality of mapping aligned vectors are obtained, and a semantic vector knowledge base is constructed based on the plurality of mapping aligned vectors; Based on the semantic text block knowledge base, the knowledge graph knowledge base, the image and text description knowledge base, and the semantic vector knowledge base, a multi-level power domain knowledge base is constructed.

[0007] According to another aspect of the present application, a multi-level power domain knowledge base construction device is provided, comprising: A text image separation module is configured to obtain domain knowledge original data in the power domain, and perform text image separation processing on the domain knowledge original data to obtain domain knowledge original text and domain knowledge original images; A semantic text block knowledge base construction module is configured to perform adaptive semantic text block segmentation processing on the domain knowledge original text to obtain a plurality of domain knowledge text blocks, and construct a semantic text block knowledge base based on the plurality of domain knowledge text blocks; A knowledge graph knowledge base construction module is configured to perform double-channel collaborative knowledge extraction processing on each of the domain knowledge text blocks to obtain a plurality of open knowledge triples and a plurality of controlled knowledge triples, and perform fusion processing on the plurality of open knowledge triples and the plurality of controlled knowledge triples to obtain a plurality of knowledge graph triples, and construct a knowledge graph knowledge base based on the plurality of knowledge graph triples; The image and text description knowledge base construction module is configured to perform adaptive text description generation processing on each of the domain knowledge original images respectively, to obtain a plurality of sets of combinations of domain knowledge original images and corresponding text descriptions, and to construct an image and text description knowledge base based on the plurality of sets of combinations of domain knowledge original images and corresponding text descriptions. The semantic vector knowledge base construction module is configured to perform vector conversion processing and mapping alignment processing on each of the domain knowledge text blocks, each of the knowledge graph triples, and each set of combinations of the domain knowledge original images and corresponding text descriptions respectively, to obtain a plurality of mapped and aligned vectors, and to construct a semantic vector knowledge base based on the plurality of mapped and aligned vectors. The multi-level power domain knowledge base construction module is configured to construct a multi-level power domain knowledge base based on the semantic text block knowledge base, the knowledge graph knowledge base, the image and text description knowledge base, and the semantic vector knowledge base.

[0008] According to another aspect of the present application, a storage medium is provided, and the storage medium stores at least one executable instruction, which causes a processor to perform operations corresponding to the construction method of the multi-level power domain knowledge base.

[0009] According to still another aspect of the present application, a terminal is provided, which comprises a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete communication with each other through the communication bus. The memory is configured to store at least one executable instruction, which causes the processor to perform operations corresponding to the construction method of the multi-level power domain knowledge base.

[0010] Through the above technical solutions, the technical solutions provided by the embodiments of the present application have at least the following advantages: The application provides a multi-level power field knowledge base construction method and device. Compared with the prior art, the field knowledge original text is divided into a plurality of independent semantic units through adaptive semantic segmentation processing, which provides a basis for accurate understanding and analysis; the field knowledge structure is displayed in a graph form through knowledge extraction and triple fusion processing, improving the convenience of query and reasoning; the field knowledge original image and its text description are combined through adaptive text description generation processing, expanding the application range; the multi-modal field knowledge is quantified in a vector form through vector conversion and alignment processing, improving the matching efficiency; further, the above-constructed semantic text block knowledge base, knowledge graph knowledge base, image and text description knowledge base and semantic vector knowledge base are combined to construct a multi-level field knowledge base, through mutual cooperation between the plurality of knowledge bases, the multi-modal field knowledge query is realized, the accuracy of the query result is improved, so that the application reliability of the large language model externally connected with the multi-level field knowledge base is effectively improved.

[0011] The above description is only a summary of the technical scheme of the application. In order to enable the technical means of the application to be implemented more clearly according to the content of the specification, and in order to enable the above and other purposes, characteristics and advantages of the application to be more apparent and easy to understand, the following specific embodiments of the application are described. BRIEF DESCRIPTION OF DRAWINGS

[0012] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not meant to limit the present application. Furthermore, the same reference numerals are used throughout the several drawings to designate the same or similar parts. In the drawings: Figure 1 A multi-level power field knowledge base construction method flowchart provided by an embodiment of the application is shown; Figure 2 A field knowledge text block segmentation flowchart provided by an embodiment of the application is shown; Figure 3 A knowledge graph triple construction flowchart provided by an embodiment of the application is shown; Figure 4 A combination construction flowchart of field knowledge original images and corresponding text descriptions provided by an embodiment of the application is shown; Figure 5 A vector construction flowchart provided by an embodiment of the application is shown; Figure 6 A multi-level power field knowledge base construction device composition block diagram provided by an embodiment of the application is shown; Figure 7 A structure schematic diagram of a terminal provided by an embodiment of the application is shown. DETAILED DESCRIPTION

[0013] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood, and so that the scope of the present disclosure can be completely conveyed to those skilled in the art.

[0014] The embodiment of the present application provides a method for constructing a multi-level power field knowledge base, as shown in the figure, the method comprises: Figure 1 101, acquire domain knowledge raw data in the power field, perform text image separation processing on the domain knowledge raw data to obtain domain knowledge raw text and domain knowledge raw image.

[0015] It should be noted that in addition to the power field, any field can be targeted, for example, medical, legal, financial, and other highly specialized fields; the domain knowledge raw data can be multi-source heterogeneous data; the text image separation processing is used to separate the image and text content contained in the domain knowledge raw data. In the embodiment of the present application, the current execution end can be a plug-in database construction module of a large language model.

[0016] It should be noted that separating the text and the image provides support for subsequent multi-level knowledge modeling based on text and image as data basis.

[0017] 102, perform adaptive semantic text block segmentation processing on the domain knowledge raw text to obtain a plurality of domain knowledge text blocks, and construct a semantic text block knowledge base based on the plurality of domain knowledge text blocks.

[0018] Among them, the adaptive semantic text block segmentation processing is a processing of the text data separated in the embodiment step 101, and specifically, the segmentation is based on the segmentation total score of each group of adjacent sentence pairs in the domain knowledge raw text as the segmentation point, and the adaptive is reflected in that the weight coefficient used when calculating the segmentation total score is dynamically adjusted according to the term density change rate trend of the adjacent sentence pairs.

[0019] It should be noted that the adaptive semantic text block segmentation processing can effectively overcome the disadvantages brought by fixed length segmentation or single similarity threshold segmentation, and by dynamically adjusting the calculation weight coefficient of the segmentation total score, the sensitivity of the block is intelligently adjusted, the context semantic boundary is more accurately captured, the situation of semantic rupture or information loss caused by improper sentence breaking is avoided, and the rationality of the text block segmentation is improved.

[0020] ​103. Perform double-channel collaborative knowledge extraction processing on each domain knowledge text block respectively to obtain a plurality of open knowledge triples and a plurality of controlled knowledge triples, and perform fusion processing on the plurality of open knowledge triples and the plurality of controlled knowledge triples to obtain a plurality of knowledge graph triples, and construct a knowledge graph knowledge base based on the plurality of knowledge graph triples.

[0021] The double-channel collaborative knowledge extraction processing includes open knowledge extraction processing and controlled knowledge extraction processing; the open knowledge extraction processing is used to discover new knowledge and expand the domain boundary, which can improve the knowledge coverage and flexibility; the controlled knowledge extraction processing is used to standardize the accuracy and standardization of knowledge expression, which can improve the structural consistency and application reliability; the open knowledge triples are obtained by performing open knowledge extraction processing on each domain knowledge text block; the controlled knowledge triples are obtained by performing controlled knowledge extraction processing on each domain knowledge text block; the fusion processing is used to fuse the open knowledge triples and the controlled knowledge triples at the semantic level, which can realize semantic complementation, guarantee the breadth of knowledge extraction, and strengthen the knowledge quality; specifically, the fusion processing can include knowledge triple fusion processing, conflict resolution processing, entity alignment processing, and entity disambiguation processing; the knowledge triple fusion processing is used to map and fuse the open entities and open relations of the open knowledge triples to the controlled entities and controlled relations of the controlled knowledge triples matched with the open knowledge triples; the conflict resolution processing is used to eliminate knowledge triples in conflict; the entity alignment processing is used to align homonym entities; and the entity disambiguation processing is used to distinguish homonym entities.

[0022] 104. Perform adaptive text description generation processing on each domain knowledge original image respectively to obtain a plurality of groups of domain knowledge original images and corresponding text descriptions, and construct an image and text description knowledge base based on the plurality of groups of domain knowledge original images and corresponding text descriptions.

[0023] The adaptive text description generation processing is used to generate the text description corresponding to the domain knowledge original image; specifically, according to the knowledge graph triples matched with the domain knowledge original image, the prompt words required by the large language model are constructed, and then the large model is used to generate the text description corresponding to the domain knowledge original image based on the prompt words; the adaptiveness is embodied in that the matching knowledge graph triples, i.e., the most relevant domain knowledge, can be dynamically retrieved and injected from the knowledge graph knowledge base according to the context information of the domain knowledge original image, and the prompt words are constructed using the retrieved knowledge graph triples, so that the generated description always focuses on the domain core and the content is professional and accurate; this is a dynamic interaction process with the knowledge graph knowledge base constructed in step 103 of the embodiment, rather than an isolated image-to-text conversion process.

[0024] It should be noted that the image information and the domain knowledge can be deeply fused through the adaptive text description generation processing.

[0025] 105. Perform vector conversion processing and mapping alignment processing on each domain knowledge text block, each knowledge graph triple, and each set of domain knowledge original image and corresponding text description combination respectively, to obtain a plurality of mapped and aligned vectors, and construct a semantic vector knowledge base based on the plurality of mapped and aligned vectors.

[0026] The vector conversion processing is used to independently convert the domain knowledge text block, the knowledge graph triple, and the domain knowledge original image and the corresponding text description combination into vectors in respective vector spaces; the mapping alignment processing is used to map the vector of the knowledge graph triple and the vector of the domain knowledge original image and the corresponding text description combination to the vector space where the vector of the domain knowledge text block is located, so that the domain knowledge of different sources and modalities has comparability and relevance in the same high-dimensional semantic vector space, thereby supporting cross-modal and cross-level precise retrieval.

[0027] 106. Based on the semantic text block knowledge base, the knowledge graph knowledge base, the image and text description knowledge base, and the semantic vector knowledge base, a multi-level power domain knowledge base is constructed.

[0028] In the embodiments of the present application, the above-constructed semantic text block knowledge base, knowledge graph knowledge base, image and text description knowledge base, and semantic vector knowledge base are combined to construct a multi-level domain knowledge base. Through the mutual cooperation between the multiple knowledge bases, when a user queries in natural language, not only semantically similar text blocks can be retrieved, but also related knowledge graph triples and matching images can be accurately retrieved, truly realizing seamless integration and efficient utilization of multi-level and multi-modal domain knowledge.

[0029] Compared with the prior art, the embodiments of the present application divide the domain knowledge original text into a plurality of independent semantic units through adaptive semantic segmentation processing, providing a basis for accurate understanding and analysis; the domain knowledge structure is displayed in the form of a graph through knowledge extraction and triple fusion processing, improving the convenience of querying and reasoning; the domain knowledge original image is combined with its text description through adaptive text description generation processing, expanding the application range; the multi-modal domain knowledge is quantified in vector form through vector conversion and alignment processing, improving the matching efficiency; further, the above-constructed semantic text block knowledge base, knowledge graph knowledge base, image and text description knowledge base, and semantic vector knowledge base are combined to construct a multi-level domain knowledge base. Through the mutual cooperation between the multiple knowledge bases, multi-modal domain knowledge querying is realized, the accuracy of the query results is improved, and thus the application reliability of the large language model with the multi-level domain knowledge base externally attached is effectively improved.

[0030] In an embodiment of the present application, in order to further limit and illustrate, as shown in Figure 2 The embodiment step 102 performs adaptive semantic text block segmentation processing on the domain knowledge original text to obtain a plurality of domain knowledge text blocks, including: 201, divide a plurality of adjacent sentence pairs, and take each adjacent sentence pair as a target adjacent sentence pair.

[0031] Correspondingly, the embodiment step 201 specifically includes: performing sentence splitting processing on the domain knowledge original text to obtain a plurality of sentences; performing adjacent sentence pair division processing on the plurality of sentences to obtain a plurality of adjacent sentence pairs, and taking each adjacent sentence pair as a target adjacent sentence pair.

[0032] Among them, the target adjacent sentence pair contains a first target sentence and a second target sentence; the sentence splitting processing is used for sentence-level splitting of the domain knowledge original text, that is, splitting at the end of each sentence (for example, the position of a period, an exclamation point, etc.), to obtain a plurality of independent sentences, for example, splitting "the arc monitoring of the conductor is one of the important indicators of the state evaluation of the transmission line." and "the indicator can be uploaded to the monitoring center in real time through the OPGW optical fiber communication network." into two sentences; the adjacent sentence pair division processing is used for dividing each two adjacent sentences into an adjacent sentence pair, such as sentence 1 and sentence 2, sentence 2 and sentence 3, sentence 3 and sentence 4, etc., to obtain a plurality of adjacent sentence pairs.

[0033] It should be noted that before performing the sentence splitting processing, the structure recognition processing can also be performed on the domain knowledge original text based on the layout analysis technology to determine the different areas such as the main text, the chart description, the table content, the footnote, the page number, etc. Further, the text content can be extracted based on the OCR technology, and the non-main text content such as the header, the footer, the page number, etc. can be filtered to ensure the semantic coherence of the extracted content and the rationality of the reading order. For example, after performing the OCR processing on the text in the "XX Operation and Maintenance Regulations", the non-main text information such as "XX Power Grid Company" and "page X" is removed, and the technical description part is retained.

[0034] 202, calculate the semantic similarity score of the target adjacent sentence pair.

[0035] Correspondingly, the embodiment step 202 specifically includes: performing sentence semantic vector conversion processing on the first target sentence and the second target sentence respectively to obtain a first target sentence semantic vector and a second target sentence semantic vector, and calculating the semantic similarity between the first target sentence semantic vector and the second target sentence semantic vector to obtain the semantic similarity score of the target adjacent sentence pair.

[0036] The sentence semantic vector conversion processing is used for semantic coding of the sentences to generate high-dimensional semantic vectors to capture the semantic relationship between sentences, for example, generating high-dimensional semantic vector representations of "the conductor vertical arc degree anomaly may cause the line trip" and "the optical fiber communication network can realize remote transmission of state data", respectively.

[0037] It can be understood that a higher similarity (such as exceeding a similarity threshold) indicates that the two in the adjacent sentence pair are strongly related in semantics, that is, they are in the same semantic segment; otherwise, it indicates that the semantic association is weak, and there may be a semantic boundary, for example, the sentence A is "the conductor vertical arc degree anomaly may cause the line trip", the sentence B is "the optical fiber communication network can realize remote transmission of state data", and the semantic similarity is 0.35, which is lower than the preset threshold 0.6, so it is considered that the semantic association is weak, which indicates that there may be a semantic boundary.

[0038] 203, calculate the term density change rate score of the target adjacent sentence pair.

[0039] Correspondingly, the embodiment step 203 specifically includes: obtaining a domain term dictionary in the power field, based on the domain term dictionary, respectively counting the number of domain terms contained in the first target sentence and the second target sentence to obtain the first domain term number and the second domain term number, and respectively calculating the proportion of the domain term number to the number of sentence terms to obtain the first term density and the second term density, and calculating the term density change rate between the first term density and the second term density to obtain the term density change rate score of the target adjacent sentence pair.

[0040] The domain term dictionary is used to record the commonly used terms in the domain, for example, the power grid domain term dictionary includes terms such as conductor, OPGW, vertical arc degree, trip, optical fiber communication, and monitoring system; the term density change rate score is used to represent the term density change rate between the first term density and the second term density, and is used to reflect the mutation of the use of terms between adjacent sentences. The greater the term density change, the more likely the context has a shift in theme or technical focus, for example, the first target sentence contains 2 terms, and the number of sentence terms is 15, so the first term density is 2 / 15=0.13, and the second term density is calculated in the same way, assuming it is 0.02, and the term density change rate is It should be noted that when the term density change rate between the first term density and the second term density is large, such as exceeding the term density change rate threshold, it indicates that there is a high possibility of a semantic breakpoint between the two sentences.

[0041] 204, determine the logical association degree score of the target adjacent sentence pair.

[0042] Accordingly, step 204 of the embodiment specifically includes: obtaining a logical word dictionary, and determining the logical association score of the target adjacent sentence pair based on the logical word dictionary. Specifically, if the first word of the second target sentence is a logical word, the logical association score of the target adjacent sentence pair is determined to be 0; if the first word of the second target sentence is a non-logical word, the logical association score of the target adjacent sentence pair is determined to be 1.

[0043] The logical word dictionary records commonly used logical words, such as definition, explanation, for example, conclusion, therefore, however, in addition, firstly, secondly, etc. The logical relevance score indicates whether the first word of the second target statement is a logical word, which enhances the ability to recognize the continuity of the text context. Specifically, if it is a logical word, the logical relevance score of the target adjacent statement pair is determined to be 0; if it is not a logical word, the logical relevance score of the target adjacent statement pair is determined to be 1.

[0044] It should be noted that when the first word is a logical connector, it indicates that the two statements are logically strongly related and are more likely to be in the same text block. Otherwise, they are logically weakly related and are less likely to be in the same text block. For example, if a statement is "In summary, the main reason for the line tripping is abnormal sag," it can be identified as a contextual continuity point and marked as a candidate for merging because it begins with the logical connector "In summary."

[0045] 205. Determine the weight coefficients for matching adjacent statements to the target.

[0046] Accordingly, step 205 of the embodiment specifically includes: determining the term density change rate trend between the first group of adjacent statement pairs and the target adjacent statement pair based on the term density change rate score of multiple groups of adjacent statement pairs between the first group of adjacent statement pairs and the target adjacent statement pair, and determining the weight coefficient matched with the target adjacent statement pair based on the term density change rate trend.

[0047] The weighting coefficient is dynamically adjusted based on the trend of term density change rate. Specifically, the term density change rate scores of multiple adjacent sentence pairs between the first adjacent sentence pair and the target adjacent sentence pair are obtained to determine the term density change rate trend between the first adjacent sentence pair and the target adjacent sentence pair. If the term density change rate trend is continuously increasing, it indicates that the text topic or focus is shifting. At this time, the weighting coefficient corresponding to the difference between 1 and the semantic similarity score can be increased, thereby dynamically improving the sensitivity to slight decreases in semantic similarity and ensuring that the semantic boundaries caused by content mutations can be accurately captured.

[0048] In addition, the weighting coefficient corresponding to the term density change rate score can be dynamically adjusted based on the average term density of the text. For example, when the average term density is low, it indicates a non-core argumentative area, and the weighting coefficient corresponding to the term density change rate score can be reduced to avoid misjudging breakpoints due to the occasional appearance of a few terms. Conversely, in core argumentative areas with high term density, the weighting coefficient corresponding to the term density change rate score should be maintained or appropriately increased. Furthermore, when multiple logical connectors appear consecutively, it indicates that the text is in a paragraph with a dense logical structure, and the weighting coefficient corresponding to the logical connection score can be reduced to ensure that the complete expression remains within a single text block. It is understandable that the sum of the weighting coefficients is always 1.

[0049] 206. Based on the weight coefficients, calculate the weighted sum between the term density change rate score, the logical relevance score, and the difference between 1 and the semantic similarity score to obtain the total segmentation score of the target adjacent sentence pair.

[0050] In this embodiment of the application, the formula for calculating the total segmentation score is expressed as follows:

[0051] in, This represents the total score for the segmentation of adjacent statement pairs. This represents the weight coefficient corresponding to the semantic similarity score. Statement With statement Semantic similarity score between them This represents the weighting coefficient corresponding to the term density change rate score. The term represents the density change rate score. This represents the weighting coefficient corresponding to the score of logical relevance. The score indicates the degree of logical connection.

[0052] 207. Obtain the total segmentation score of each group of adjacent sentence pairs as text block segmentation points, and compare each total segmentation score with the preset total segmentation score threshold.

[0053] 208. If the total segmentation score is greater than the preset total segmentation score threshold, then text block segmentation is performed at the end of the first statement in the adjacent statement pair corresponding to the total segmentation score.

[0054] 209. Continue until all adjacent statement pairs in each group have completed threshold comparison and text block segmentation, resulting in multiple domain knowledge text blocks.

[0055] In the embodiment steps 207-209, when the segmentation total score is greater than the preset segmentation total score threshold, it indicates that the probability that the two sentences in the current adjacent sentence pair belong to different text blocks is high. At this time, the position between the two sentences is taken as a text block segmentation point, and text block segmentation processing is performed until all adjacent sentence pairs complete threshold comparison and text block segmentation processing, and multiple domain knowledge text blocks are obtained.

[0056] As a case corresponding to the embodiment step 208, if the segmentation total score is less than or equal to the preset segmentation total score threshold, the adjacent sentence pair corresponding to the segmentation total score does not need to be segmented.

[0057] In an embodiment of the present application, in order to further limit and illustrate, as shown in Figure 3 The embodiment step 103 performs double-channel collaborative knowledge extraction processing on each domain knowledge text block to obtain multiple open knowledge triples and multiple controlled knowledge triples, and performs fusion processing on the multiple open knowledge triples and the multiple controlled knowledge triples to obtain multiple knowledge graph triples, and constructs a knowledge graph knowledge base based on the multiple knowledge graph triples, including: 301. Obtain a first prompt word, and based on the first prompt word, perform open knowledge extraction processing on each domain knowledge text block to obtain multiple open knowledge triples.

[0058] The first prompt word includes a domain knowledge element example and an extraction requirement, which is used to guide the model output. For example, Please extract all possible domain knowledge elements from the following text, including but not limited to: 1. Entities (such as people, organizations, terms, products, technologies, etc.) and their types; 2. Relationships (such as "invent", "depend", "affect", "belong to", etc.) and associated entity pairs; 3. Events (such as "release", "cooperate", "malfunction", etc.) and participants, time, and results; 4. Attributes (such as performance indicators, prices, rules, etc.).

[0059] Requirements: 1. Output in a structured format (triple SPO format dictionary list), with the following format: { "subject": {"value": "Entity A", "type": "Entity Type"}, "predicate": "Relationship Name", "object": {"value": "Entity B", "type": "Entity Type"} } 2. Preserve implicit but logically reasonable knowledge in the original text (such as "X is compatible with Y" can infer "X has a technical relationship with Y"). 3. The output should be a list of dictionaries (JSON List) without other explanatory text.

[0060] It should be noted that the open knowledge extraction processing used in the embodiments of the present application automatically identifies and extracts entities, relationships, events and attributes for each domain knowledge text block, without the need to define ontology or knowledge model in advance, and can flexibly cope with emerging terminology and complex sentence structure, and is suitable for the cold start stage of professional fields. In addition, the format of the output open knowledge triple can be set to a structured SPO triple dictionary list, ensuring the consistency of subsequent processing. In addition, during the open knowledge extraction processing, the logical and reasonable knowledge implied in the original text can be retained, thereby enhancing the completeness and practicality of the extraction results.

[0061] 302. Cluster the plurality of open knowledge triples to obtain a domain ontology of the power field.

[0062] Correspondingly, the step 302 of the embodiment specifically comprises: integrating open entities and open relationships in the plurality of open knowledge triples respectively to obtain an open entity set and an open relationship set, respectively calculating open entity semantic similarity between any two open entities in the open entity set, and when the open entity semantic similarity exceeds a preset open entity semantic similarity threshold, dividing the two open entities into the same open entity cluster to obtain a plurality of open entity clusters, respectively calculating open relationship semantic similarity between any two open relationships in the open relationship set, and when the open relationship semantic similarity exceeds a preset open relationship semantic similarity threshold, dividing the two open relationships into the same open relationship cluster to obtain a plurality of open relationship clusters, and respectively performing induction processing on each open entity cluster and each open relationship cluster to generate entity types and relationship types, so as to obtain the domain ontology of the power field.

[0063] In the embodiments of the present application, the large model can be used to perform induction processing on each open entity cluster and each open relationship cluster. For example, the prompt word used for inducing the open entity cluster is as follows: You are a domain knowledge standardization assistant. Please perform semantic type induction on the entity cluster based on the following input information.

[0064] Input information: entity cluster list: {"candidates": [ ]} Your task is: - analyze the semantic commonality of all entities; - induce and output a most standardized and most general entity type; - provide selection reasons. The output format is as follows: { "category":"", "reason":"" } where, represents the list of open entity clusters to be induced.

[0065] The prompt words used for inducing open relation clusters are as follows: You are a domain knowledge standardization assistant. Please induce the semantic type of the relation cluster according to the following input information.

[0066] Input information: relation cluster list: {"candidates": [ ]} Your task is: - analyze the semantic commonality of all relations; - induce and output a most standardized and most general relation type; - provide selection reasons. The output format is as follows: { "relation_type": "", "definition": "", "subject_constraint": "", "object_constraint": "" } where, represents the list of open relation clusters to be induced.

[0067] The final ontology structure is reviewed by experts or automatically adjusted through an iterative optimization mechanism. After data format conversion, the following simple domain ontology can be obtained, for example: Entity types include: organization, technology, monitoring parameter, communication method, application function, event.

[0068] Relation types include: 1. Propose: subject is organization, object is technology; 2. Promote content: subject is technology, object is technology; 3. Monitoring content: subject is technology, object is monitoring parameter; 4. Data transmission method: subject is technology, object is communication method; 5. Implement application: subject is technology, object is application function; 6. Occur: subject is organization, object is event; 7. Propose time: subject is technology, object is event.

[0069] 303、Based on the domain ontology, a second prompt word is constructed, and based on the second prompt word, controlled knowledge extraction processing is performed on each domain knowledge text block respectively to obtain a plurality of controlled knowledge triples.

[0070] The first prompt word includes a domain ontology Schema and an extraction requirement, and is used to guide the model output. Please extract knowledge from the text according to the following domain knowledge model, and only output the content that meets the model definition.

[0071] Domain model constraints: Entity types include: organization, technology, monitoring parameter, communication method, application function, and event.

[0072] Relationship types include: 1. Propose: the subject is an organization, and the object is a technology. 2. Promote content: the subject is a technology, and the object is a technology. 3. Monitoring content: the subject is a technology, and the object is a monitoring parameter. 4. Data transmission method: the subject is a technology, and the object is a communication method. 5. Implement application: the subject is a technology, and the object is an application function. 6. Occur: the subject is an organization, and the object is an event. 7. Propose time: the subject is a technology, and the object is an event.

[0073] Requirements: 1. Strictly match the entity types and relationship types in the model, and ignore irrelevant information. 2. Normalize the identified entities (e.g., "OPGW optical fiber network" is unified as "optical fiber communication", and "dynamic capacity increase" remains as "dynamic capacity increase"). 3. The output result is presented in the form of a structured SPO triple dictionary list, each triple contains a subject, a predicate, and an object; each subject and object is an object containing two fields: value and type, and the output format is as follows: { "subject": {"value": "Entity A", "type": "Entity Type"}, "predicate": "Relationship Name", "object": {"value": "Entity B", "type": "Entity Type"} } 4. If a field is missing, mark it as "NULL". 5. Without outputting any explanatory text, only return the list of knowledge triples that meet the format requirements.

[0074] Preferably, the output format of the controlled knowledge triples is set to be consistent with the output format of the open knowledge triples, i.e. a structured SPO triple dictionary list, to ensure consistency in subsequent processing.

[0075] 304. Perform knowledge triple fusion processing on the plurality of open knowledge triples and the plurality of controlled knowledge triples to obtain a plurality of fused knowledge triples.

[0076] Correspondingly, the embodiment step 304 specifically comprises: integrating the controlled entities and the controlled relationships in the plurality of controlled knowledge triples respectively to obtain a controlled entity set and a controlled relationship set, and calculating the entity mapping semantic similarity between each open entity and each controlled entity in the controlled entity set respectively, mapping the open entity to the controlled entity corresponding to the maximum entity mapping semantic similarity when the maximum entity mapping semantic similarity exceeds a preset entity mapping semantic similarity threshold, and receiving the type label of the controlled entity, and calculating the relationship mapping semantic similarity between each open relationship and each controlled relationship in the controlled relationship set respectively, mapping the open relationship to the controlled relationship corresponding to the maximum relationship mapping semantic similarity when the maximum relationship mapping semantic similarity exceeds a preset relationship mapping semantic similarity threshold, and receiving the type label of the controlled relationship, to obtain a plurality of fused knowledge triples.

[0077] In the embodiment of the present application, the open entity is mapped to the controlled entity corresponding to the maximum entity mapping semantic similarity, and the type label of the controlled entity is received, i.e. the open entity is converted into the standard naming form used in controlled knowledge extraction, and the corresponding entity type label is assigned; similarly, the open relationship is mapped to the controlled relationship corresponding to the maximum relationship mapping semantic similarity, and the type label of the controlled relationship is received, i.e. the open relationship is converted into the standard relationship used in controlled knowledge extraction, and the corresponding relationship type label is assigned. In addition, if the maximum entity mapping semantic similarity does not exceed the preset entity mapping semantic similarity threshold, it is retained as an extended entity for subsequent review or model iteration optimization, and the relationship mapping is the same. For example, the entities "OPGW optical cable", "OPGW communication line", "optical fiber composite overhead ground wire" and the like may be identified in open knowledge extraction, which are mapped to the entity "optical fiber communication" in controlled knowledge extraction through the above method, and inherit the type label "communication mode".

[0078] 305. Perform conflict resolution processing on the plurality of fused knowledge triples to obtain a plurality of conflict-resolved knowledge triples.

[0079] Correspondingly, the embodiment step 305 specifically comprises: screening conflict knowledge triple pairs from the plurality of fused knowledge triples, calculating the confidence score of each conflict knowledge triple pair, and screening out conflict knowledge triples with low confidence scores to obtain a plurality of conflict-resolved knowledge triples.

[0080] wherein the conflict knowledge triple pair represents two knowledge triples with conflicting content; the confidence score is a weighted sum of the confidence parameter output by the large model and the triple source parameter, and the triple source includes controlled knowledge extraction and open knowledge extraction, which can be represented by the following formula,

[0081] wherein, represents the confidence score of the knowledge triple , represents the confidence parameter of the knowledge triple , represents the weight parameter corresponding to the confidence parameter, represents the triple source parameter. For example, two triples A (from open knowledge extraction) and B (from controlled knowledge extraction) are identified, the confidence scores are 0.5 and 0.8 respectively, and if the confidence threshold is set to 0.6, then triple B is retained and triple A is excluded.

[0082] In addition, a confidence score threshold can also be pre-set. When the confidence scores of two knowledge triples are both less than the pre-set confidence score threshold, the large model can be used for secondary judgment and semantic correction to improve the accuracy of conflict resolution. For example, if the two confidence scores are 0.4 and 0.5 respectively, and the confidence threshold is still set to 0.6, the large model needs to be introduced for secondary judgment and semantic correction, and the prompt word is as follows: please judge and correct the conflicting triples according to the input content.

[0083] Your task is: 1. Analyze the content consistency and context reasonableness of the two triples; 2. Determine which triple is more consistent with the original text semantics; 3. If both are not accurate, try to generate a new and more reasonable triple; 4. Output the final selected or corrected triple, and mark the selection basis or modification reason.

[0084] The output format is as follows: { "selected_triple": { "subject": {"value": "entity A", "type": "entity type"}, "predicate": "relation name", "object": {"value": "Entity B", "type": "Entity Type"} }, "reason": "" } Input content: {The two conflicting triples to be judged and their source text} 306. Perform entity alignment processing on multiple conflict-resolved knowledge triples to obtain multiple sets of entity-aligned knowledge triples.

[0085] Accordingly, step 306 of the embodiment specifically includes: integrating entities in multiple conflict-resolved knowledge triples to obtain an entity set, calculating the entity semantic similarity between any two entities in the entity set, and merging the two entities when the entity semantic similarity is greater than a preset entity semantic similarity threshold to obtain multiple sets of entity-aligned knowledge triples.

[0086] In this embodiment, the merging process may specifically involve constructing a synonymous entity cluster, selecting the highest-frequency entity name in the cluster as the primary naming convention for that cluster, and using other entity names as aliases. Simultaneously, a mapping relationship is established between the aliases and knowledge triples. If multiple entity names with the same frequency exist, a large model can be used for selection, with the following prompts: You are a domain knowledge standardization assistant, responsible for selecting the most appropriate primary naming convention from multiple synonymous entity names.

[0087] The input consists of a set of synonymous entity names and their context information. Based on naming conventions, domain consistency, semantic clarity, and conciseness, determine and select the most suitable entity name as the primary name.

[0088] Input information includes: 1. Entity name candidate list: {"candidates": [ ]} 2. Contextual fragments in which the entity appears:

[0089] Your task is: -Analyze the normativity of candidate names in domain terminology; - Determine which name best conforms to industry standards or common usage habits; -If multiple names are equally standardized, choose the one with the clearest semantics and the simplest structure; - Output the final selected primary naming convention and explain the reasons for the selection.

[0090] The output format is as follows: { "preferred_name": "", "reason": "" } in, This represents a candidate list of entity names. This represents the context in which the entity appears. 307. Perform entity disambiguation processing on the knowledge triples after multiple sets of entity alignment to obtain multiple knowledge graph triples.

[0091] Accordingly, step 307 of the embodiment specifically includes: distinguishing entities with the same name but different meanings in the knowledge triples after multiple entity alignment, so as to complete entity disambiguation processing and obtain multiple knowledge graph triples.

[0092] In one embodiment of this application, for further definition and explanation, such as Figure 4 As shown, in step 104 of the embodiment, adaptive text description generation processing is performed on the original images of each domain knowledge to obtain a combination of multiple sets of original images of domain knowledge and corresponding text descriptions, including: 401. Take the original images of knowledge from each domain as the original images of the target domain knowledge one by one; obtain the associated information that matches the original images of the target domain knowledge. The relevant information includes, but is not limited to, the title above the original image of the target domain knowledge, the caption below it, and the semantic block content of the chapter in which the image is located, etc., in the original domain knowledge data. For example, the caption below the target "insulator string" image is "Schematic diagram of the structure of a new type of anti-pollution flashover insulator string".

[0093] 402. Based on the association information, perform a retrieval operation in the knowledge graph knowledge base to obtain knowledge graph triples that match the original image of the target domain knowledge.

[0094] In this embodiment, the relevant information obtained in step 401 of the embodiment is used as keywords to perform a search operation in the knowledge graph knowledge base, and knowledge graph triplets that match the original image of the target domain knowledge are obtained, such as <insulator, function, electrical connection between conductor and tower>, <insulator, material, ceramic>, <flashover, cause, insulator surface contamination leads to a decrease in insulation performance under humid conditions>, <anti-flashover, measures, adopting a large creepage distance design or applying RTV coating>.

[0095] 403. Serialize the matching knowledge graph triples to obtain the knowledge graph triple description text, and construct a third prompt word that matches the original image of the target domain knowledge based on the knowledge graph triple description text.

[0096] For example, the four triplets in step 402 of the embodiment are serialized to obtain the description text: "Related field knowledge: The function of an insulator is to isolate the electrical connection between the conductor and the tower, and it is usually made of ceramic material. Pollution flashover refers to the phenomenon that the insulation performance of an insulator deteriorates due to contamination on its surface under humid conditions. Anti-pollution flashover measures include adopting a large creepage distance design or applying RTV coating." The description text is then concatenated with a general prompt word template to obtain the third prompt word.

[0097] 404. Based on the third prompt word, perform text description processing on the original image of the target domain knowledge to obtain a combination of the original image of the target domain knowledge and the corresponding text description.

[0098] 405. Obtain a combination of multiple sets of original images of domain knowledge and their corresponding text descriptions.

[0099] The third prompt word is obtained by concatenating and fusing the description requirements and the knowledge graph triple description text. For example, You are a professional image description generation assistant. Please carefully observe the image below and generate a natural and fluent Chinese description based on its content. Requirements are as follows: 1. The description should cover the main objects, scenes, actions, or events in the image.

[0100] 2. If characters are present, please describe their behavior, emotions, or interactions as much as possible.

[0101] 3. Pay attention to environmental details, such as time, place, weather, and background.

[0102] 4. If there is text, logo, or special symbols in the image, please try to identify and explain them.

[0103] 5. Avoid subjective assumptions, maintain objectivity and accuracy, and use vivid and descriptive language.

[0104] 6. Keep the description between 100 and 200 words to ensure the information is complete, concise, and clear.

[0105] 7. Please refer to the following domain knowledge and incorporate it naturally into your description: Related knowledge: The function of insulators is to isolate the electrical connection between conductors and towers, and they are usually made of ceramic. Pollution flashover refers to the phenomenon where contamination on the insulator surface degrades its insulation performance under humid conditions. Anti-pollution flashover measures include using large creepage distance designs or applying RTV coatings.

[0106] Please output the results starting with "Image description:".

[0107] In an embodiment of the present application, in order to further limit and illustrate, as shown in Figure 5 Embodiment step 105 respectively performs vector conversion processing and mapping alignment processing on each domain knowledge text block, each knowledge graph triple, and each set of domain knowledge original image and corresponding text description combination, to obtain a plurality of mapped and aligned vectors, including: 501, respectively, text block semantic vector conversion processing is performed on each domain knowledge text block, to obtain a plurality of domain knowledge text block semantic vectors.

[0108] Among them, the domain knowledge text block semantic vector is a high-dimensional dense semantic vector. Illustratively, "insulator is an important component in the power transmission line, its main function is to realize the electrical isolation between the conductor and the tower, and prevent current leakage. Common insulator materials include ceramic and glass." is converted into a high-dimensional dense semantic vector.

[0109] 502, based on the knowledge graph embedding algorithm, respectively, knowledge graph embedding vector conversion processing is performed on each knowledge graph triple, to obtain a plurality of knowledge graph embedding vectors.

[0110] Specifically, each knowledge graph triple is taken as a target knowledge graph triple one by one; based on the knowledge graph embedding algorithm, the entity of the target knowledge graph triple is converted into an entity vector, and the relationship is converted into a relationship vector; based on the knowledge graph embedding algorithm, according to the entity vector and the relationship vector, the knowledge graph embedding vector of the target knowledge graph triple is generated, to obtain a plurality of knowledge graph embedding vectors.

[0111] In an embodiment of the present application, the knowledge graph embedding vector is a single vector of complete facts obtained by weighting or splicing.

[0112] 503, respectively, image-text joint vector conversion processing is performed on each set of domain knowledge original image and corresponding text description combination, to obtain a plurality of image-text joint vectors.

[0113] Correspondingly, embodiment step 503 specifically includes: for each set of domain knowledge original image and corresponding text description combination, based on the image vector generation model that has completed model training, image vector conversion processing is performed on the domain knowledge original image to obtain an image vector, and based on the text description vector generation model that has completed model training, text description vector conversion processing is performed on the text description to obtain a text description vector, and the image vector and the text description vector are spliced or weighted to generate the image-text joint vector corresponding to the set of domain knowledge original image and corresponding text description combination, to obtain a plurality of image-text joint vectors.

[0114] In an embodiment of the present application, in order to further limit and illustrate, before the image vector conversion processing is performed on the domain knowledge original image based on the image vector generation model trained, and the text description vector conversion processing is performed on the text description based on the text description vector generation model trained, the embodiment method further comprises: obtaining a plurality of combinations of training domain knowledge images and corresponding training text descriptions; constructing an initial image vector generation model and an initial text description vector generation model; performing image vector conversion processing on each training domain knowledge image based on the initial image vector generation model to obtain a plurality of training image vectors; performing text description vector conversion processing on each training text description based on the initial text description vector generation model to obtain a plurality of training text description vectors; calculating the similarity between each training image vector and each training text description vector respectively, and generating a similarity matrix, wherein the diagonal elements in the similarity matrix represent the similarity of positive samples, and the non-diagonal elements in the similarity matrix represent the similarity of negative samples, the positive samples are used to represent the combination of the training domain knowledge image and the corresponding training text description, and the negative samples are used to represent the combination of the training domain knowledge image and other training text descriptions except the corresponding training text description; based on the similarity matrix, calculating the joint loss function value of the initial image vector generation model and the initial text description vector generation model, and the joint loss function is represented as the following formula, , wherein, represents the joint loss function value, represents the training image vector of the training domain knowledge image i, represents the training domain knowledge image corresponding training text description, represents the positive sample similarity, and τ represents the temperature coefficient, represents the first other training text description; adjusting the model parameters of the initial image vector generation model and the initial text description vector generation model to minimize the joint loss function value, to obtain the image vector generation model trained and the text description vector generation model trained, to perform image vector conversion processing based on the image vector generation model trained, and to perform text description vector conversion processing based on the text description vector generation model trained.

[0115] It should be noted that the above model training process is based on the cross-modal contrast learning algorithm, so that the image vector and the text description vector of the matching image and text generated based on the image vector generation model and the text description vector generation model are closer in the vector space.

[0116] 504. Based on the vector mapping model that has been trained, each knowledge graph embedding vector and each graph-text joint vector are mapped to the vector space where the semantic vector of the domain knowledge text block is located, resulting in multiple mapped vectors.

[0117] In this embodiment of the application, the knowledge graph embedding vector and the graph-text joint vector are mapped to the vector space where the semantic vector of the domain knowledge text block is located, so as to keep the three types of vectors consistent in semantics.

[0118] In one embodiment of this application, to further define and illustrate, before mapping each knowledge graph embedding vector and each graph-text joint vector to the vector space where the semantic vector of the domain knowledge text block resides, based on the vector mapping model that has completed model training, the method of the embodiment further includes: selecting multiple anchor knowledge points, and respectively obtaining the domain knowledge text block semantic vector, knowledge graph embedding vector, and graph-text joint vector corresponding to each anchor knowledge point, wherein the anchor knowledge points are used to represent the domain knowledge that exists in the semantic text block knowledge base, the knowledge graph knowledge base, and the image and text description knowledge base; constructing an initial vector mapping model, and Based on the initial vector mapping model, the knowledge graph embedding vectors and graph-text joint vectors of each anchor knowledge are mapped to the vector space containing the semantic vectors of the domain knowledge text blocks, resulting in multiple mapped knowledge graph embedding vectors and multiple mapped graph-text joint vectors. The loss values ​​between the multiple mapped knowledge graph embedding vectors and multiple mapped graph-text joint vectors and the multiple domain knowledge text block semantic vectors are calculated. The model parameters of the initial vector mapping model are adjusted to minimize the loss value, resulting in a vector mapping model based on the completed model training. Vector space mapping processing is then performed using the vector mapping model based on the completed model training.

[0119] In this embodiment, the loss function can be the mean squared error loss function, which ensures that the semantically consistent knowledge graph embedding vector and the semantic vector of the domain knowledge text block in the graph-text joint vector are semantically consistent.

[0120] 505. Determine multiple mapped vectors and multiple semantic vectors of domain knowledge text blocks as multiple mapped and aligned vectors.

[0121] In this embodiment of the application, the mapped vector and the semantic vector of the domain knowledge text block are collectively referred to as the mapped and aligned vector.

[0122] In an embodiment of the present application, in order to further limit and illustrate, the embodiment step 101 acquires domain knowledge raw data in the power field before the embodiment method further comprises: collecting domain knowledge in the power field from a plurality of data sources respectively; respectively performing format conversion processing on each domain knowledge to obtain a plurality of standard format domain knowledges; respectively performing cleaning processing and integration processing on each standard format domain knowledge to obtain domain knowledge raw data.

[0123] Among them, the cleaning processing includes redundancy removal processing and noise removal processing, for example, removing redundant symbols, advertising information, HTML tags, non-language text, repeated content, etc., and also includes automatically correcting grammar and spelling; the data source includes but is not limited to internal domain data (text files, PDF documents), industry websites, open data platforms, etc. For web page data, browser automation tools such as Selenium can be used for data scraping; the format conversion processing is used to convert the data into a standard unified encoding format.

[0124] The present application provides a method for constructing a multi-level power domain knowledge base. Compared with the prior art, the method divides the domain knowledge raw text into a plurality of independent semantic units through adaptive semantic segmentation processing, providing a basis for accurate understanding and analysis; the method displays the domain knowledge structure in the form of a graph through knowledge extraction and triple fusion processing, improving the convenience of querying and reasoning; the method combines the domain knowledge original image with its text description through adaptive text description generation processing, expanding the application range; the method quantifies the multi-modal domain knowledge in the form of a vector through vector conversion and alignment processing, improving the matching efficiency; further, the method combines the above-constructed semantic text block knowledge base, knowledge graph knowledge base, image and text description knowledge base, and semantic vector knowledge base to construct a multi-level domain knowledge base. Through the mutual cooperation between the multiple knowledge bases, the method realizes multi-modal domain knowledge query, improves the accuracy of the query results, and thus effectively improves the application reliability of the large language model with the multi-level domain knowledge base.

[0125] Further, as an implementation of the above-mentioned Figure 1 method, the embodiment of the present application provides a construction device of a multi-level power domain knowledge base, as shown in Figure 6 , the device comprises: a text image separation module 61, a semantic text block knowledge base construction module 62, a knowledge graph knowledge base construction module 63, an image and text description knowledge base construction module 64, a semantic vector knowledge base construction module 65, and a multi-level power domain knowledge base construction module 66. The text-image separation module 61 is configured to acquire domain knowledge raw data in the power field, and perform text-image separation processing on the domain knowledge raw data to obtain domain knowledge raw text and domain knowledge raw images. The semantic text block knowledge base construction module 62 is configured to perform adaptive semantic text block segmentation processing on the domain knowledge raw text to obtain a plurality of domain knowledge text blocks, and construct a semantic text block knowledge base based on the plurality of domain knowledge text blocks. The knowledge graph knowledge base construction module 63 is configured to perform double-channel collaborative knowledge extraction processing on each of the domain knowledge text blocks to obtain a plurality of open knowledge triples and a plurality of controlled knowledge triples, and perform fusion processing on the plurality of open knowledge triples and the plurality of controlled knowledge triples to obtain a plurality of knowledge graph triples, and construct a knowledge graph knowledge base based on the plurality of knowledge graph triples. The image and text description knowledge base construction module 64 is configured to perform adaptive text description generation processing on each of the domain knowledge raw images to obtain a plurality of combinations of domain knowledge raw images and corresponding text descriptions, and construct an image and text description knowledge base based on the plurality of combinations of domain knowledge raw images and corresponding text descriptions. The semantic vector knowledge base construction module 65 is configured to perform vector conversion processing and mapping alignment processing on each of the domain knowledge text blocks, each of the knowledge graph triples, and each of the combinations of domain knowledge raw images and corresponding text descriptions to obtain a plurality of mapped and aligned vectors, and construct a semantic vector knowledge base based on the plurality of mapped and aligned vectors. The multi-level power domain knowledge base construction module 66 is configured to construct a multi-level power domain knowledge base based on the semantic text block knowledge base, the knowledge graph knowledge base, the image and text description knowledge base, and the semantic vector knowledge base.

[0126] In a specific application scenario, the semantic text block knowledge base construction module includes: The segmentation total score calculation unit is configured to calculate a segmentation total score of each adjacent sentence pair in the domain knowledge raw text as a text block segmentation point, wherein the segmentation total score is a weighted sum of a term density change rate score, a logical association degree score, and a difference between 1 and a semantic similarity score, and a weight coefficient used to calculate the weighted sum is dynamically adjusted according to a term density change rate trend of the adjacent sentence pair. The segmentation total score threshold comparison unit is configured to compare each of the segmentation total scores with a preset segmentation total score threshold. The text block segmentation unit is configured to perform text block segmentation processing at the end of the first sentence in the adjacent sentence pair corresponding to the segmentation total score if the segmentation total score is greater than the preset segmentation total score threshold. The text block segmentation unit is further configured to perform threshold comparison and text block segmentation processing on each group of adjacent sentence pairs to obtain a plurality of domain knowledge text blocks.

[0127] In a specific application scenario, the segmentation total score calculation unit is configured to: perform sentence splitting processing on the domain knowledge original text to obtain a plurality of sentences; divide the plurality of sentences into adjacent sentence pairs to obtain a plurality of groups of adjacent sentence pairs, and take each group of adjacent sentence pairs as a target adjacent sentence pair one by one, wherein the target adjacent sentence pair includes a first target sentence and a second target sentence; perform sentence semantic vector conversion processing on the first target sentence and the second target sentence respectively to obtain a first target sentence semantic vector and a second target sentence semantic vector, and calculate a semantic similarity between the first target sentence semantic vector and the second target sentence semantic vector to obtain a semantic similarity score of the target adjacent sentence pair; obtain a domain term dictionary in the power field, based on the domain term dictionary, respectively count the number of domain terms in the first target sentence and the second target sentence to obtain a first domain term number and a second domain term number, respectively calculate the proportion of the number of domain terms in the number of sentence words to obtain a first term density and a second term density, and calculate a term density change rate between the first term density and the second term density to obtain a term density change rate score of the target adjacent sentence pair; obtain a logical word dictionary, based on the logical word dictionary, determine a logical association degree score of the target adjacent sentence pair, wherein if the first word of the second target sentence is a logical word, the logical association degree score of the target adjacent sentence pair is determined to be 0, and if the first word of the second target sentence is a non-logical word, the logical association degree score of the target adjacent sentence pair is determined to be 1; determine a term density change rate trend from the first group of adjacent sentence pairs to the target adjacent sentence pair according to the term density change rate scores of the plurality of groups of adjacent sentence pairs between the first group of adjacent sentence pairs and the target adjacent sentence pair, and determine a weight coefficient matched with the target adjacent sentence pair according to the term density change rate trend; based on the weight coefficient, calculate the weighted sum of the term density change rate score, the logical association degree score, and the difference between 1 and the semantic similarity score to obtain a segmentation total score of the target adjacent sentence pair.

[0128] In a specific application scenario, the knowledge graph knowledge base construction module comprises: The dual-channel collaborative knowledge extraction unit is configured to perform open knowledge extraction processing on each of the domain knowledge text blocks based on the first prompt word to obtain a plurality of open knowledge triples, and perform controlled knowledge extraction processing on each of the domain knowledge text blocks based on the second prompt word to obtain a plurality of controlled knowledge triples, wherein the second prompt word is constructed based on clustering of the plurality of open knowledge triples. The knowledge triple fusion unit is configured to integrate controlled entities and controlled relationships in the plurality of controlled knowledge triples to obtain a controlled entity set and a controlled relationship set, calculate, for each open entity, an entity mapping semantic similarity between the open entity and each controlled entity in the controlled entity set, map the open entity to a controlled entity corresponding to a maximum entity mapping semantic similarity when the maximum entity mapping semantic similarity exceeds a preset entity mapping semantic similarity threshold, and receive a type label of the controlled entity, calculate, for each open relationship, a relationship mapping semantic similarity between the open relationship and each controlled relationship in the controlled relationship set, map the open relationship to a controlled relationship corresponding to a maximum relationship mapping semantic similarity when the maximum relationship mapping semantic similarity exceeds a preset relationship mapping semantic similarity threshold, and receive a type label of the controlled relationship, and obtain a plurality of fused knowledge triples. The conflict resolution unit is configured to filter conflict knowledge triple pairs from the plurality of fused knowledge triples, calculate, for each group of conflict knowledge triple pairs, a confidence score of two fused knowledge triples included in the conflict knowledge triple pairs, and filter out conflict knowledge triples with low confidence scores to obtain a plurality of conflict-resolved knowledge triples. The entity alignment unit is configured to integrate entities in the plurality of conflict-resolved knowledge triples to obtain an entity set, calculate an entity semantic similarity between any two entities in the entity set, and merge the two entities when the entity semantic similarity is greater than a preset entity semantic similarity threshold to obtain a plurality of entity-aligned knowledge triples. The entity disambiguation unit is configured to distinguish entities with the same name but different meanings in the plurality of entity-aligned knowledge triples to complete entity disambiguation processing and obtain a plurality of knowledge graph triples.

[0129] In a specific application scenario, the dual-channel collaborative knowledge extraction unit is configured to: obtain a first prompt word, and based on the first prompt word, perform open knowledge extraction processing on each of the domain knowledge text blocks respectively to obtain a plurality of open knowledge triples, wherein the first prompt word includes a domain knowledge element example and an extraction requirement; respectively integrate open entities and open relations in the plurality of open knowledge triples to obtain an open entity set and an open relation set, and respectively calculate open entity semantic similarity between any two open entities in the open entity set, and when the open entity semantic similarity exceeds a preset open entity semantic similarity threshold, divide the two open entities into the same open entity cluster to obtain a plurality of open entity clusters, and respectively calculate open relation semantic similarity between any two open relations in the open relation set, and when the open relation semantic similarity exceeds a preset open relation semantic similarity threshold, divide the two open relations into the same open relation cluster to obtain a plurality of open relation clusters, and respectively perform induction processing on each of the open entity clusters and each of the open relation clusters to generate entity types and relation types to obtain a domain ontology of the power domain; based on the domain ontology, construct a second prompt word, and based on the second prompt word, perform controlled knowledge extraction processing on each of the domain knowledge text blocks to obtain a plurality of controlled knowledge triples.

[0130] In a specific application scenario, the image and text description knowledge base construction module is configured to: take each of the domain knowledge original images as a target domain knowledge original image one by one; obtain associated information matching the target domain knowledge original image; based on the associated information, perform a retrieval operation in the knowledge graph knowledge base to obtain a knowledge graph triple matching the target domain knowledge original image; perform serialization processing on the matching knowledge graph triple to obtain a knowledge graph triple description text, and based on the knowledge graph triple description text, construct a third prompt word matching the target domain knowledge original image; based on the third prompt word, perform text description processing on the target domain knowledge original image to obtain a combination of the target domain knowledge original image and the corresponding text description; obtain a plurality of combinations of domain knowledge original images and corresponding text descriptions.

[0131] In a specific application scenario, the semantic vector knowledge base construction module includes: a text block semantic vector conversion unit configured to perform text block semantic vector conversion processing on each of the domain knowledge text blocks to obtain a plurality of domain knowledge text block semantic vectors; The knowledge graph embedding vector conversion unit is configured to convert each of the knowledge graph triples into a knowledge graph embedding vector based on a knowledge graph embedding algorithm, and obtain a plurality of knowledge graph embedding vectors. The image-text combined vector conversion unit is configured to, for each combination of the domain knowledge original image and the corresponding text description, convert the domain knowledge original image into an image vector based on a trained image vector generation model, convert the text description into a text description vector based on a trained text description vector generation model, and concatenate or weight the image vector and the text description vector to generate an image-text combined vector corresponding to the combination of the domain knowledge original image and the corresponding text description, and obtain a plurality of image-text combined vectors. The vector mapping unit is configured to map each of the knowledge graph embedding vectors and each of the image-text combined vectors into a vector space where the domain knowledge text block semantic vectors are located based on a trained vector mapping model, and obtain a plurality of mapped vectors. The vector mapping unit is further configured to determine the plurality of mapped vectors and the plurality of domain knowledge text block semantic vectors as a plurality of mapped and aligned vectors.

[0132] In a specific application scenario, before the image-text combined vector conversion unit, the semantic vector knowledge base construction module further includes a first model training unit configured to: obtain a plurality of combinations of training domain knowledge images and corresponding training text descriptions; construct an initial image vector generation model and an initial text description vector generation model; convert each of the training domain knowledge images into a training image vector based on the initial image vector generation model, and obtain a plurality of training image vectors; convert each of the training text descriptions into a training text description vector based on the initial text description vector generation model, and obtain a plurality of training text description vectors; calculate the similarity between each of the training image vectors and each of the training text description vectors, and generate a similarity matrix, wherein the diagonal elements of the similarity matrix represent the similarity of positive samples, and the non-diagonal elements of the similarity matrix represent the similarity of negative samples, the positive samples are used to represent the combination of the training domain knowledge image and the corresponding training text description, and the negative samples are used to represent the combination of the training domain knowledge image and other training text descriptions except the corresponding training text description; calculate the joint loss function value of the initial image vector generation model and the initial text description vector generation model based on the similarity matrix, and the joint loss function is represented by the following formula, , in, This represents the value of the joint loss function. Let i represent the training image vector of the training domain knowledge image. Representing training domain knowledge images The corresponding training text description, This represents the similarity between positive samples, and τ represents the temperature coefficient. Indicates the first Other training text descriptions; The model parameters of the initial image vector generation model and the initial text description vector generation model are adjusted to minimize the joint loss function value, thereby obtaining an image vector generation model and a text description vector generation model that have completed model training. Image vector conversion processing is performed based on the image vector generation model that has completed model training, and text description vector conversion processing is performed based on the text description vector generation model that has completed model training.

[0133] In specific application scenarios, before the vector mapping unit, the semantic vector knowledge base construction module further includes a second model training unit, used for: Multiple anchor knowledge points are selected, and the semantic vector of the domain knowledge text block, the knowledge graph embedding vector, and the image-text joint vector corresponding to each anchor knowledge point are obtained respectively. The anchor knowledge points are used to represent the domain knowledge that exists in the semantic text block knowledge base, the knowledge graph knowledge base, and the image and text description knowledge base. An initial vector mapping model is constructed, and based on the initial vector mapping model, the knowledge graph embedding vector and graph-text joint vector of each anchor point knowledge are mapped to the vector space where the semantic vector of the domain knowledge text block is located, to obtain multiple mapped knowledge graph embedding vectors and multiple mapped graph-text joint vectors. Calculate the loss values ​​between the multiple mapping knowledge graph embedding vectors and the multiple mapping graph-text joint vectors, and the multiple domain knowledge text block semantic vectors; The model parameters of the initial vector mapping model are adjusted to minimize the loss value, resulting in a vector mapping model based on the completed model training. Vector space mapping is then performed based on the vector mapping model based on the completed model training.

[0134] This application provides a device for constructing a multi-level knowledge base in the power field. Compared with existing technologies, it divides the original text of domain knowledge into multiple independent semantic units through adaptive semantic segmentation, providing a foundation for accurate understanding and analysis; it displays the domain knowledge structure in graph form through knowledge extraction and triple fusion, improving the convenience of querying and reasoning; it combines the original image of domain knowledge with its text description through adaptive text description generation, expanding the application scope; it quantifies multimodal domain knowledge in vector form through vector transformation and alignment, improving matching efficiency; furthermore, it combines the constructed semantic text block knowledge base, knowledge graph knowledge base, image and text description knowledge base, and semantic vector knowledge base to construct a multi-level domain knowledge base. Through the cooperation between multiple knowledge bases, it realizes the query of multimodal domain knowledge, improves the accuracy of query results, and thus effectively improves the application reliability of a large language model with an external multi-level domain knowledge base.

[0135] According to one embodiment of this application, a storage medium is provided, the storage medium storing at least one executable instruction, which can execute the method for constructing a multi-level power domain knowledge base in any of the above method embodiments.

[0136] Figure 7 The diagram shows a structural schematic of a terminal according to one embodiment of the present application. The specific embodiments of the present application do not limit the specific implementation of the terminal.

[0137] like Figure 7 As shown, the terminal may include: a processor 702, a communications interface 704, a memory 706, and a communications bus 708.

[0138] The processor 702, communication interface 704, and memory 706 communicate with each other via communication bus 708.

[0139] The communication interface 704 is used to communicate with other network elements such as clients or other servers.

[0140] The processor 702 is used to execute program 710, specifically to execute the relevant steps in the above-described embodiment of the method for constructing a multi-level power domain knowledge base.

[0141] Specifically, program 710 may include program code that includes computer operation instructions.

[0142] The processor 702 can be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to perform the operations of the embodiments of the application. The computer device can include one or more processors of the same type or different types, such as one or more CPUs and one or more ASICs.

[0143] The memory 706 is configured to store the program 710. The memory 706 can include a high-speed RAM memory, and can also include a non-volatile memory, such as at least one disk memory.

[0144] The program 710 can be specifically configured to enable the processor 702 to perform the operations in the steps 101-104 of the embodiments.

[0145] The above merely provides the preferred embodiments of the application, and is not intended to limit the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A method for constructing a multi-level power domain knowledge base, characterized by, The method comprises the following steps: acquiring domain knowledge raw data in the field of power acquisition, performing text image separation processing on the domain knowledge raw data to obtain domain knowledge raw text and domain knowledge raw images; performing adaptive semantic text block segmentation processing on the domain knowledge raw text to obtain a plurality of domain knowledge text blocks, and constructing a semantic text block knowledge base based on the plurality of domain knowledge text blocks; performing double-channel collaborative knowledge extraction processing on each of the domain knowledge text blocks to obtain a plurality of open knowledge triples and a plurality of controlled knowledge triples, and performing fusion processing on the plurality of open knowledge triples and the plurality of controlled knowledge triples to obtain a plurality of knowledge graph triples, and constructing a knowledge graph knowledge base based on the plurality of knowledge graph triples; performing adaptive text description generation processing on each of the domain knowledge raw images to obtain a plurality of combinations of domain knowledge raw images and corresponding text descriptions, and constructing an image and text description knowledge base based on the plurality of combinations of domain knowledge raw images and corresponding text descriptions; performing vector conversion processing and mapping alignment processing on each of the domain knowledge text blocks, each of the knowledge graph triples, and each of the combinations of domain knowledge raw images and corresponding text descriptions to obtain a plurality of mapped and aligned vectors, and constructing a semantic vector knowledge base based on the plurality of mapped and aligned vectors; constructing a multi-level power domain knowledge base based on the semantic text block knowledge base, the knowledge graph knowledge base, the image and text description knowledge base, and the semantic vector knowledge base.

2. The method of claim 1, wherein, The adaptive semantic text block segmentation processing on the domain knowledge raw text to obtain a plurality of domain knowledge text blocks comprises: calculating the total segmentation score of each group of adjacent sentence pairs in the domain knowledge raw text as a text block segmentation point, wherein the total segmentation score is the weighted sum of the term density change rate score, the logical association degree score, and the difference between 1 and the semantic similarity score, and the weight coefficient used to calculate the weighted sum is dynamically adjusted according to the term density change rate trend of the adjacent sentence pairs; comparing each of the total segmentation scores with a preset total segmentation score threshold; if the total segmentation score is greater than the preset total segmentation score threshold, performing text block segmentation processing at the end of the first sentence in the adjacent sentence pair corresponding to the total segmentation score; until each group of adjacent sentence pairs completes threshold comparison and text block segmentation processing, a plurality of domain knowledge text blocks are obtained.

3. The method of claim 2, wherein, The calculation of the total segmentation score of each group of adjacent sentence pairs in the domain knowledge raw text as a text block segmentation point comprises: performing sentence splitting processing on the domain knowledge raw text to obtain a plurality of sentences; dividing the plurality of sentences into adjacent sentence pairs to obtain a plurality of groups of adjacent sentence pairs, and taking each of the groups of adjacent sentence pairs as a target adjacent sentence pair one by one, wherein the target adjacent sentence pair contains a first target sentence and a second target sentence; respectively, to obtain a first target sentence semantic vector and a second target sentence semantic vector, and to calculate a semantic similarity between the first target sentence semantic vector and the second target sentence semantic vector to obtain a semantic similarity score of the target adjacent sentence pair; obtain a domain term dictionary in the field of electric power, based on the domain term dictionary, respectively count the number of domain terms contained in the first target sentence and the second target sentence to obtain a first domain term number and a second domain term number, and respectively calculate the proportion of the number of domain terms in the number of sentence terms to obtain a first term density and a second term density, and calculate a term density change rate between the first term density and the second term density to obtain a term density change rate score of the target adjacent sentence pair; obtain a logical word dictionary, based on the logical word dictionary, determine a logical association degree score of the target adjacent sentence pair, wherein if the first word of the second target sentence is a logical word, the logical association degree score of the target adjacent sentence pair is determined as 0, and if the first word of the second target sentence is a non-logical word, the logical association degree score of the target adjacent sentence pair is determined as 1; determine a term density change rate trend between the first group of adjacent sentence pairs and the target adjacent sentence pair according to the term density change rate scores of the multiple groups of adjacent sentence pairs between the first group of adjacent sentence pairs and the target adjacent sentence pair, and determine a weight coefficient matched with the target adjacent sentence pair according to the term density change rate trend; based on the weight coefficient, calculate the weighted sum between the term density change rate score, the logical association degree score and the difference between 1 and the semantic similarity score to obtain the total segmentation score of the target adjacent sentence pair.

4. The method of claim 1, wherein, The respective domain knowledge text blocks are subjected to double-channel collaborative knowledge extraction processing to obtain multiple open knowledge triples and multiple controlled knowledge triples, and the multiple open knowledge triples and the multiple controlled knowledge triples are subjected to fusion processing to obtain multiple knowledge graph triples, including: based on the first prompt word, each of the domain knowledge text blocks is subjected to open knowledge extraction processing to obtain multiple open knowledge triples, and based on the second prompt word, each of the domain knowledge text blocks is subjected to controlled knowledge extraction processing to obtain multiple controlled knowledge triples, wherein the second prompt word is constructed based on the clustering processing of the multiple open knowledge triples; respectively, and the open entity is mapped to the controlled entity corresponding to the maximum entity mapping semantic similarity when the maximum entity mapping semantic similarity exceeds a preset entity mapping semantic similarity threshold, and a type label of the controlled entity is received, and for each open relation, a relation mapping semantic similarity between the open relation and each controlled relation in the controlled relation set is calculated respectively, and the open relation is mapped to the controlled relation corresponding to the maximum relation mapping semantic similarity when the maximum relation mapping semantic similarity exceeds a preset relation mapping semantic similarity threshold, and a type label of the controlled relation is received, to obtain a plurality of fused knowledge triples; conflict knowledge triple pairs are screened out from the plurality of fused knowledge triples, for each group of conflict knowledge triple pairs, confidence scores of two fused knowledge triples contained in the conflict knowledge triple pairs are calculated respectively, and conflict knowledge triples with low confidence scores are screened out, to obtain a plurality of conflict-resolved knowledge triples; entities in the plurality of conflict-resolved knowledge triples are integrated to obtain an entity set, entity semantic similarities between any two entities in the entity set are calculated respectively, and the two entities are merged when the entity semantic similarity is greater than a preset entity semantic similarity threshold, to obtain a plurality of entity-aligned knowledge triple groups; entities with the same name but different meanings in the plurality of entity-aligned knowledge triple groups are distinguished to complete entity disambiguation processing, to obtain a plurality of knowledge graph triples.

5. The method of claim 4, wherein, The open knowledge extraction processing is performed on each of the domain knowledge text blocks based on the first prompt word, to obtain a plurality of open knowledge triples, and the controlled knowledge extraction processing is performed on each of the domain knowledge text blocks based on the second prompt word, to obtain a plurality of controlled knowledge triples, including: a first prompt word is obtained, and the open knowledge extraction processing is performed on each of the domain knowledge text blocks based on the first prompt word, to obtain a plurality of open knowledge triples, wherein the first prompt word includes a domain knowledge element example and an extraction requirement; Integrate open entities and open relations in the plurality of open knowledge triples respectively to obtain an open entity set and an open relation set, and calculate open entity semantic similarity between any two open entities in the open entity set respectively, and when the open entity semantic similarity exceeds a preset open entity semantic similarity threshold, divide the two open entities into the same open entity cluster to obtain a plurality of open entity clusters, and calculate open relation semantic similarity between any two open relations in the open relation set respectively, and when the open relation semantic similarity exceeds a preset open relation semantic similarity threshold, divide the two open relations into the same open relation cluster to obtain a plurality of open relation clusters, and respectively perform induction processing on each open entity cluster and each open relation cluster to generate entity types and relation types to obtain a domain ontology of the power field; Construct a second prompt word based on the domain ontology, and based on the second prompt word, perform controlled knowledge extraction processing on each domain knowledge text block respectively to obtain a plurality of controlled knowledge triples.

6. The method of claim 1, wherein, The adaptive text description generation processing on each domain knowledge original image respectively to obtain a plurality of combinations of domain knowledge original images and corresponding text descriptions comprises: Take each domain knowledge original image as a target domain knowledge original image one by one; Obtain associated information matched with the target domain knowledge original image; Based on the associated information, perform a retrieval operation in the knowledge graph knowledge base to obtain a knowledge graph triple matched with the target domain knowledge original image; Perform serialization processing on the matched knowledge graph triple to obtain a knowledge graph triple description text, and construct a third prompt word matched with the target domain knowledge original image based on the knowledge graph triple description text; Based on the third prompt word, perform text description processing on the target domain knowledge original image to obtain a combination of the target domain knowledge original image and the corresponding text description; Obtain a plurality of combinations of domain knowledge original images and corresponding text descriptions.

7. The method of claim 1, wherein, The vector conversion processing and mapping alignment processing on each domain knowledge text block, each knowledge graph triple, and each combination of domain knowledge original images and corresponding text descriptions respectively to obtain a plurality of mapped vectors comprises: Perform text block semantic vector conversion processing on each domain knowledge text block to obtain a plurality of domain knowledge text block semantic vectors; Based on a knowledge graph embedding algorithm, perform knowledge graph embedding vector conversion processing on each knowledge graph triple to obtain a plurality of knowledge graph embedding vectors; For each combination of domain knowledge original image and corresponding text description, based on the image vector generation model after model training, the image vector conversion processing is performed on the domain knowledge original image to obtain an image vector, and based on the text description vector generation model after model training, the text description vector conversion processing is performed on the text description to obtain a text description vector, and the image vector and the text description vector are spliced or weighted to generate a graph-text combined vector corresponding to the combination of the domain knowledge original image and the corresponding text description, thereby obtaining a plurality of graph-text combined vectors; Based on the vector mapping model after model training, each knowledge graph embedding vector and each graph-text combined vector are mapped to the vector space where the domain knowledge text block semantic vector is located, thereby obtaining a plurality of mapped vectors. The plurality of mapped vectors and the plurality of domain knowledge text block semantic vectors are determined as a plurality of mapped and aligned vectors.

8. The method of claim 7, wherein, Before the image vector conversion processing on the domain knowledge original image based on the image vector generation model after model training and the text description vector conversion processing on the text description based on the text description vector generation model after model training, the method further comprises: Obtaining a plurality of combinations of training domain knowledge images and corresponding training text descriptions; Constructing an initial image vector generation model and an initial text description vector generation model; Based on the initial image vector generation model, performing image vector conversion processing on each of the training domain knowledge images to obtain a plurality of training image vectors; Based on the initial text description vector generation model, performing text description vector conversion processing on each of the training text descriptions to obtain a plurality of training text description vectors; The similarity between each of the training image vectors and each of the training text description vectors is calculated respectively, and a similarity matrix is generated, wherein the diagonal elements in the similarity matrix represent the similarity of positive samples, the non-diagonal elements in the similarity matrix represent the similarity of negative samples, the positive samples are used to represent the combination of the training domain knowledge image and the corresponding training text description, and the negative samples are used to represent the combination of the training domain knowledge image and other training text descriptions except the corresponding training text description; Based on the similarity matrix, the joint loss function value of the initial image vector generation model and the initial text description vector generation model is calculated, and the joint loss function is represented by the following formula, ， wherein, denotes a joint loss function value, denotes a training image vector of a training domain knowledge image i, denotes a training domain knowledge image corresponding training text description, denotes a positive sample similarity, τ denotes a temperature coefficient, denotes a first other training text description; The model parameters of the initial image vector generation model and the initial text description vector generation model are adjusted to minimize the joint loss function value, thereby obtaining the image vector generation model after model training and the text description vector generation model after model training, so as to perform image vector conversion processing based on the image vector generation model after model training and perform text description vector conversion processing based on the text description vector generation model after model training.

9. The method of claim 7, wherein, Before mapping each of the knowledge graph embedding vectors and each of the image-text combined vectors to the vector space where the domain knowledge text block semantic vectors are located based on the completed model trained vector mapping model, the method further comprises: selecting multiple anchor points of knowledge, and respectively acquiring the domain knowledge text block semantic vectors, the knowledge graph embedding vectors, and the image-text combined vectors corresponding to each of the anchor points of knowledge, wherein the anchor points of knowledge are used to represent the domain knowledge that exists in the semantic text block knowledge base, the knowledge graph knowledge base, and the image and text description knowledge base; constructing an initial vector mapping model, and respectively mapping the knowledge graph embedding vectors and the image-text combined vectors of each of the anchor points of knowledge to the vector space where the domain knowledge text block semantic vectors are located based on the initial vector mapping model, to obtain multiple mapped knowledge graph embedding vectors and multiple mapped image-text combined vectors; calculating the loss value between the multiple mapped knowledge graph embedding vectors, the multiple mapped image-text combined vectors, and the multiple domain knowledge text block semantic vectors; adjusting the model parameters of the initial vector mapping model to minimize the loss value, to obtain a vector mapping model based on completed model training, and performing vector space mapping processing based on the vector mapping model based on completed model training.

10. An apparatus for constructing a multi-level power domain knowledge base, comprising: Comprise: a text-image separation module for acquiring domain knowledge raw data in the power field, performing text-image separation processing on the domain knowledge raw data, and obtaining domain knowledge raw text and domain knowledge raw images; a semantic text block knowledge base construction module for performing adaptive semantic text block segmentation processing on the domain knowledge raw text, obtaining multiple domain knowledge text blocks, and constructing a semantic text block knowledge base based on the multiple domain knowledge text blocks; a knowledge graph knowledge base construction module for performing double-channel collaborative knowledge extraction processing on each of the domain knowledge text blocks, obtaining multiple open knowledge triples and multiple controlled knowledge triples, and performing fusion processing on the multiple open knowledge triples and the multiple controlled knowledge triples, obtaining multiple knowledge graph triples, and constructing a knowledge graph knowledge base based on the multiple knowledge graph triples; an image and text description knowledge base construction module for performing adaptive text description generation processing on each of the domain knowledge raw images, obtaining multiple groups of combinations of domain knowledge raw images and corresponding text descriptions, and constructing an image and text description knowledge base based on the multiple groups of combinations of domain knowledge raw images and corresponding text descriptions; a semantic vector knowledge base construction module for performing vector conversion processing and mapping alignment processing on each of the domain knowledge text blocks, each of the knowledge graph triples, and each group of combinations of the domain knowledge raw images and corresponding text descriptions, obtaining multiple mapped and aligned vectors, and constructing a semantic vector knowledge base based on the multiple mapped and aligned vectors; The multi-level power field knowledge base construction module is used for constructing a multi-level power field knowledge base based on the semantic text block knowledge base, the knowledge graph knowledge base, the image and text description knowledge base and the semantic vector knowledge base.

Images