Knowledge enhancement method and device, storage medium and electronic equipment

Abstract

The invention provides a knowledge enhancement method and device, a storage medium and electronic equipment, and relates to the field of machine learning. The electronic equipment receives to-be-processed information of a large language model, and the large language model comprises a plurality of general knowledge layers; determining coarse screening information from a knowledge base according to the to-be-processed information; for each general knowledge layer, extracting fine screening information from the coarse screening information by using input information input to the general knowledge layer, and generating a knowledge enhancement model according to the fine screening information; and injecting related knowledge required for processing the input information into the general knowledge layer by using the knowledge enhancement model to obtain output information after the input information is processed. The problem that knowledge needs to be truncated due to context length limitation in a traditional RAG method can be avoided. Meanwhile, the knowledge enhancement model generated according to the fine screening information is utilized to inject related knowledge into the general knowledge layer, it is ensured that the model can obtain latest domain knowledge in a knowledge base in real time, and therefore dynamic and complete knowledge enhancement is achieved.

Description

Technical Field

[0001] The present application relates to the field of machine learning, and more specifically, to a knowledge enhancement method, device, storage medium, and electronic device. Background Art

[0002] With the rapid development of artificial intelligence technology, large models have achieved remarkable results in fields such as natural language processing and computer vision. However, these models are constrained by computing resources and knowledge updating mechanisms in practical applications.

[0003] First, although the generation effect can be enhanced by dynamically retrieving information from the knowledge base, due to the limited length of the input context of large models, the original information in the knowledge base often needs to be truncated, which affects the model's understanding and utilization of the complete domain knowledge. At the same time, although Retrieval-Augmented Generation (RAG) as a dynamic retrieval-augmented generation method can alleviate the above problems to a certain extent, it also has its own limitations that are difficult to overcome. The basic idea of RAG is to extract relevant information from the knowledge base through the retriever and input it as context into the generation model to enhance the output effect. However, due to the limitation of the input context length of large models, RAG often needs to truncate the original information in the knowledge base in actual operation to ensure that the input content meets the requirements of the model. The truncation operation inevitably affects the model's understanding and utilization of the complete domain knowledge, thereby weakening the quality and accuracy of the generated content.

[0004] Therefore, existing solutions either sacrifice computational efficiency or compromise on knowledge coverage or real-time performance, and cannot simultaneously meet the needs of efficiently processing long texts and dynamically updating domain knowledge. Summary of the Invention

[0005] To overcome at least one deficiency in the prior art, the present application provides a knowledge enhancement method, apparatus, storage medium, and electronic device, specifically comprising:

[0006] In a first aspect, the present application provides a knowledge enhancement method, the method comprising:

[0007] receiving information to be processed from a large language model, wherein the large language model includes a plurality of general knowledge layers;

[0008] Determining rough screening information from a knowledge base based on the information to be processed;

[0009] For each of the general knowledge layers, extracting fine screening information from the coarse screening information using input information input into the general knowledge layer, and generating a knowledge enhancement model based on the fine screening information;

[0010] The knowledge enhancement model is used to inject relevant knowledge required for processing the input information into the general knowledge layer to obtain output information after the input information is processed.

[0011] In a second aspect, the present application provides a knowledge enhancement device, comprising:

[0012] an information receiving module, configured to receive information to be processed from a large language model, wherein the large language model includes a plurality of general knowledge layers;

[0013] A knowledge screening module, configured to determine coarse screening information from a knowledge base based on the information to be processed;

[0014] The knowledge screening module is further configured to extract fine screening information from the coarse screening information using input information input into the general knowledge layer for each general knowledge layer, and generate a knowledge enhancement model based on the fine screening information;

[0015] The knowledge enhancement module is used to use the knowledge enhancement model to inject relevant knowledge required for processing the input information into the general knowledge layer to obtain output information after the input information is processed.

[0016] In a third aspect, the present application provides a storage medium storing a computer program, which implements the knowledge enhancement method when being processed and executed.

[0017] In a fourth aspect, the present application provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program, and the computer program implements the knowledge enhancement method when being processed and executed.

[0018] Compared with the prior art, this application has the following beneficial effects:

[0019] The present application provides a knowledge enhancement method, device, storage medium and electronic device. The electronic device receives information to be processed from a large language model, wherein the large language model includes multiple general knowledge layers; based on the information to be processed, coarse screening information is determined from the knowledge base; for each general knowledge layer, the input information input to the general knowledge layer is used to extract fine screening information from the coarse screening information, and based on the fine screening information, a knowledge enhancement model is generated; the knowledge enhancement model is used to inject relevant knowledge required for processing the input information into the general knowledge layer, and output information after the input information is processed is obtained. In this way, by determining the coarse screening information from the knowledge base and dynamically extracting the fine screening information for each general knowledge layer, the problem of truncation of knowledge due to context length limitations in traditional RAG methods is avoided. At the same time, the knowledge enhancement model generated based on the fine screening information is used to inject relevant knowledge into the general knowledge layer, ensuring that the model can obtain the latest domain knowledge in the knowledge base in real time, thereby achieving dynamic and complete knowledge enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flowchart of the knowledge enhancement method provided in an embodiment of the present application;

[0022] Figure 2 This is one of the structural diagrams of the large language model provided in the embodiment of the present application;

[0023] Figure 3 The second structural diagram of the large language model provided in the embodiment of the present application;

[0024] Figure 4 A schematic diagram of the structure of the knowledge enhancement device provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] To make the purpose, technical solutions, and advantages of the embodiments of the present application (hereinafter referred to as the present embodiments) clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0027] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0028] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0029] In the description of this application, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the inventive product is typically placed when in use. These terms are intended only to facilitate the description of this application and simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting this application. Furthermore, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0030] In addition, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0031] Furthermore, terms such as "horizontal," "vertical," and "overhanging" do not necessarily imply that a component must be absolutely horizontal or overhanging, but rather that it can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but rather that it can be slightly tilted.

[0032] It should also be noted that, in the description of this application, unless otherwise expressly specified or limited, the terms "disposed," "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on the specific circumstances.

[0033] Based on the above statement, as introduced in the background technology, current knowledge enhancement methods either need to sacrifice computational efficiency or compromise on knowledge coverage or real-time performance, and cannot simultaneously meet the needs of efficiently processing long texts and dynamically updating domain knowledge.

[0034] In order to solve the technical difficulties discovered, the following technical solutions are proposed after creative work to solve or improve the above problems. It should be noted that the defects existing in the solutions in the above prior art are the results obtained after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed in the embodiments of this application for the above problems below should be regarded as contributions to this application in the process of invention and creation, and should not be understood as technical contents known to those skilled in the art.

[0035] In view of the above technical problems, this embodiment provides a knowledge enhancement method. Figure 1 As shown, the method includes:

[0036] S1, receiving information to be processed from a large language model, wherein the large language model includes multiple general knowledge layers.

[0037] S2, based on the information to be processed, determine the rough screening information from the knowledge base.

[0038] S3, for each general knowledge layer, extract fine screening information from the coarse screening information using the input information input to the general knowledge layer, and generate a knowledge enhancement model based on the fine screening information.

[0039] S4, using the knowledge enhancement model to inject relevant knowledge required for processing input information into the general knowledge layer, and obtain output information after the input information is processed.

[0040] This approach, by determining coarse-screened information from the knowledge base and dynamically extracting fine-screened information for each general knowledge layer, avoids the problem of knowledge truncation in traditional RAG methods due to context length limitations. Furthermore, the knowledge enhancement model generated based on the fine-screened information is used to inject relevant knowledge into the general knowledge layer, ensuring that the model can access the latest domain knowledge from the knowledge base in real time, thereby achieving dynamic and complete knowledge enhancement.

[0041] In addition, for the knowledge enhancement method provided in this embodiment, the electronic device for implementing the method may be, but is not limited to, a mobile terminal, a tablet computer, a laptop computer, a desktop computer, a server, and the like. When it is a server, the server may be a single server or a server group. The server group may be centralized or distributed (for example, the server may be a distributed system). In some embodiments, the server may be local or remote relative to the user terminal. In some embodiments, the server may be implemented on a cloud platform; as an example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud (Community Cloud), a distributed cloud, an inter-cloud (Inter-Cloud), a multi-cloud (Multi-Cloud), etc., or any combination thereof. In some embodiments, the server may be implemented on an electronic device having one or more components.

[0042] To make the solution provided by this embodiment clearer, the server is used as an electronic device to implement the knowledge enhancement method. Figure 1 Each step of the method shown is described in detail. However, it should be understood that the operations of the flowchart can be implemented in any order, and steps that have no logical contextual relationship can be reversed or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of this application. Figure 1 , the method comprising:

[0043] S1, receiving the information to be processed from the large language model.

[0044] Among them, the large language model includes multiple general knowledge layers, which constitute the basic model. The basic model is usually pre-trained based on general knowledge. Therefore, when highly specialized or time-sensitive knowledge support is required, the performance of the basic model may not fully meet the needs. Therefore, it is necessary to introduce additional network layers to inject domain-specific knowledge into the general knowledge layer. In addition, like other common large language models currently, each general knowledge layer is built based on Transformer. Transformer consists of an encoder and a decoder, each part of which contains a multi-layer self-attention module and a feedforward neural network, which can analyze the dependencies between input texts.

[0045] The aforementioned information to be processed refers to the data that needs to be input into the large language model. It can come from user input, system calls, or other external data sources, and is usually presented in the form of natural language text. For example, in real-world applications, the information to be processed might be a user-provided problem description, a text to be translated, or a piece of content to be generated and continued.

[0046] S2, based on the information to be processed, determine the rough screening information from the knowledge base.

[0047] It should be understood that in this embodiment, the information to be processed is usually presented in the form of natural language text. In order to achieve effective retrieval of the knowledge base, it is necessary to convert the information to be processed into a vector representation that can be used to calculate the degree of matching. The knowledge base is a data set that stores a large amount of domain knowledge, including a large number of knowledge fragments, each of which is also preprocessed and converted into a vector representation, thereby constituting the full knowledge base vector data. As an optional implementation, step S2 may include:

[0048] S2-1, obtaining the matching degree between the information to be processed and each knowledge fragment.

[0049] In this embodiment, the server uses a pre-trained embedding model to map the information being processed into a vector representation in a high-dimensional space. Subsequently, the server uses cosine similarity or another suitable distance metric to calculate the similarity between the information vector being processed and the vector of each knowledge fragment in the knowledge base. This similarity value serves as a quantitative indicator of the degree of match, reflecting the semantic relevance between the information being processed and each knowledge fragment.

[0050] S2-2, select the coarse-screened information according to the degree of matching with each knowledge fragment.

[0051] In this embodiment, the server can set a threshold to filter out knowledge fragments with a matching degree higher than the threshold; or select several knowledge fragments with the highest matching degree as rough screening information. This selection method ensures that the rough screening information covers potential domain knowledge that is highly relevant to the information to be processed, providing the necessary candidate range for subsequent fine-screening information extraction.

[0052] Based on the explanation of step S2 in the above embodiment, the following is Figure 1 Explanation of step S3 in the following example:

[0053] S3, for each general knowledge layer, extract fine screening information from the coarse screening information using the input information input to the general knowledge layer, and generate a knowledge enhancement model based on the fine screening information.

[0054] In this regard, it should be understood that current RAG methods (e.g., DyRAG) typically filter relevant information from the knowledge base for the input information to be processed and generate a low-rank adapter (LoRA) model based on this information. The server then uses the LoRA model to process the information to be processed, and its output further fine-tunes the output of the main model.

[0055] However, in practice, it was found that the existing RAG method only filters relevant information from the knowledge base once in the initial stage for the input information to be processed. This may result in the selected knowledge fragments not being fully suitable for the specific tasks of each subsequent layer of semantic processing, and thus being unable to dynamically adapt to the different needs of the semantic processing process. In addition, because large language models contain multiple general knowledge layers, the input information and processing goals of each layer may be different. The current RAG method fails to dynamically adjust the selected knowledge fragments according to the needs of different general knowledge layers, thereby limiting the accuracy and pertinence of the generated results.

[0056] Therefore, in this embodiment, for each general knowledge layer in the large language model, the input information fed into that general knowledge layer is used to further extract fine-screen information from the coarse-screened information. "Coarse-screened information" refers to the collection of relevant knowledge fragments obtained from the knowledge base using the information to be processed, while "fine-screened information" refers to highly relevant knowledge fragments further filtered from the coarse-screened information to meet the specific needs of the current general knowledge layer. Furthermore, it should be understood that for the first general knowledge layer, its input information and the information to be processed are the same information.

[0057] As an optional implementation, the server may process the input information and the coarse screening information through a cross attention layer to obtain fine screening information.

[0058] like Figure 2As shown, in this embodiment, when the input information enters the general knowledge layer, the server uses the cross-attention layer to simultaneously receive the input information and the coarse-screened information obtained by the preliminary screening of the knowledge base. Among them, the cross-attention layer evaluates the correlation between the two by calculating the degree of matching between each element in the input information and each fragment in the coarse-screened information. The principle is to generate a weight matrix based on the dot product similarity or other forms of correlation measurement methods in the attention mechanism to represent the correlation strength between the input information and the coarse-screened information in different dimensions. Then, the server performs weighted processing on the coarse-screened information through the above-mentioned weight matrix, highlighting those parts that are highly relevant to the input information, while weakening or filtering out irrelevant information fragments. In this way, the fine-screened information finally extracted can more accurately reflect the needs of the current semantic processing task, thereby providing high-quality input data for the subsequent knowledge enhancement model.

[0059] For the extracted fine-screened information, the server generates a knowledge enhancement model applicable to the current general knowledge layer.

[0060] Continue to see Figure 2 In this embodiment, in order to effectively integrate this refined information into the semantic processing of the large language model, it needs to be converted into model parameters. To this end, this embodiment introduces a low-rank parameter conversion model. This low-rank parameter conversion model receives the refined information as input and, through a series of predefined mathematical transformation operations, maps the high-dimensional embedding vectors in the refined information into a low-dimensional space, generating a set of model parameters.

[0061] The knowledge enhancement model can be understood as a representation of carefully selected information. Therefore, it can inject the required domain knowledge into the current general knowledge layer based on the characteristics of the carefully selected information, thereby achieving more precise knowledge enhancement. This ensures that during each layer of semantic processing, the most appropriate knowledge fragment can be dynamically selected based on the input information of the current layer and the corresponding knowledge enhancement model can be generated.

[0062] Based on the explanation of the knowledge enhancement model in the above embodiment, we will continue to Figure 1 Step S4 in the following is explained:

[0063] S4, using the knowledge enhancement model to inject relevant knowledge required for processing input information into the general knowledge layer, and obtain output information after the input information is processed.

[0064] In this embodiment, the general knowledge layer generates a set of intermediate output vectors when processing input information. The vectors reflect the semantic features of the current input information in the layer. At the same time, the knowledge enhancement model generates a set of low-rank parameter vectors based on the fine-screening information. The vectors carry domain knowledge related to the current semantic processing task. In order to effectively inject this domain knowledge into the calculation of the general knowledge layer, a vector bit-by-bit addition operation can be used, that is, the low-rank parameter vector output by the knowledge enhancement model and the intermediate output vector generated by the general knowledge layer are numerically added at the same vector position. Therefore, the optional implementation of step S4 may include:

[0065] S4-1, obtaining first semantic information after the general knowledge layer processes the input information, and second semantic information after the knowledge enhancement model processes the input information.

[0066] S4-2, fusing the first semantic information with the second semantic information to obtain output information after the input information is processed.

[0067] In this embodiment, the server can weight the first semantic information and the second semantic information with their respective weights to obtain output information after the input information is processed. Figure 2 , the information to be processed passes Figure 2 After the N modules shown perform processing, a decoded text for the information to be processed is obtained, where N is a positive integer greater than 0.

[0068] However, research has revealed that the primary semantic information output by the general knowledge layer primarily comes from the textual information used during pre-training of the large language model. Therefore, this primary semantic information is highly versatile and applicable to a wide range of natural language processing tasks. However, it may lack sufficient depth and specificity for highly specialized or domain-specific tasks.

[0069] In contrast, the second semantic information generated by the knowledge enhancement model comes from the carefully screened knowledge base information. This information is not only more timely and professional, but can also be dynamically adjusted according to specific task requirements. However, there are certain differences in the semantic processing tasks of multiple general knowledge layers. Therefore, for some semantic processing tasks, the general knowledge layer is more meaningful for reference. Similarly, there are also some semantic processing tasks for which the knowledge enhancement model is more meaningful for reference. Therefore, for the first semantic information output by the general knowledge layer and the second semantic information output by the knowledge enhancement model, it is necessary to make targeted choices based on the specific semantic processing tasks, that is, when to trust the second voice information output by the knowledge enhancement model more, and when to trust the first semantic information output by the general knowledge layer more.

[0070] In view of this, in this embodiment, the large language model also includes a weight adaptation layer configured for each general knowledge layer, and before weighting the first semantic information and the second semantic information with their respective weights to obtain the output information after the input information is processed, the server can process the fine-screening information and the input information through the weight adaptation layer to obtain the first semantic information and the second semantic information with their respective weights.

[0071] like Figure 3 As shown, it can be understood that this embodiment regards the input information input to the general knowledge layer as a semantic processing task (equivalent to a prompt word). On this basis, the weight adaptive layer evaluates the characteristics of the current semantic processing task and the degree of its demand for the two types of semantic information through a comprehensive analysis of the input information and the fine screening information. That is, the input information serves as an indicator signal for the semantic processing task, clarifying the goal and background of the current task; and the fine screening information provides the most relevant domain knowledge fragments for the task. By analyzing the correlation between these two parts of information, the weight adaptive layer can accurately determine whether the current task is more suitable to rely on the first semantic information or the second semantic information.

[0072] For example, when processing tasks involving professional knowledge, if the domain knowledge in the fine-screened information is highly matched with the input information, the weight adaptation layer will give the second semantic information a higher weight, thereby enhancing the professional performance of the model. When processing tasks that are more inclined to general knowledge, the weight adaptation layer will increase the weight of the first semantic information to fully utilize the extensive knowledge reserve of the large language model. In this way, it is ensured that the final output information can flexibly combine the universal advantages of the first semantic information and the professional characteristics of the second semantic information according to the specific task requirements.

[0073] Based on the same inventive concept as the knowledge enhancement method provided in this embodiment, this embodiment also provides a knowledge enhancement device, which includes at least one software function module that can be stored in a memory or fixed in an electronic device in the form of software. The processor in the electronic device is used to execute the executable module stored in the memory. For example, the software function module and computer program included in the device. Please refer to Figure 4 Functionally, the device can include:

[0074] an information receiving module, configured to receive information to be processed from a large language model, wherein the large language model includes a plurality of general knowledge layers;

[0075] The knowledge screening module is used to determine the rough screening information from the knowledge base according to the information to be processed;

[0076] The knowledge screening module is further configured to extract fine screening information from the coarse screening information using the input information input to the general knowledge layer for each general knowledge layer, and generate a knowledge enhancement model based on the fine screening information;

[0077] The knowledge enhancement module is used to use the knowledge enhancement model to inject relevant knowledge required for processing input information into the general knowledge layer to obtain output information after the input information is processed.

[0078] In this embodiment, the information receiving module is used to implement Figure 1 In step S1, the knowledge screening module is used to implement Figure 1 In steps S2 and S3, the knowledge enhancement module is used to implement Figure 1 Therefore, for a detailed description of each of the above modules, please refer to the specific implementation of the corresponding step.

[0079] Since the knowledge enhancement method provided in this application has the same inventive concept, the knowledge enhancement device can also implement other steps or sub-steps of the method through the above modules.

[0080] Optionally, the knowledge enhancement module is further specifically used to:

[0081] Obtaining first semantic information after the general knowledge layer processes the input information, and second semantic information after the knowledge enhancement model processes the input information;

[0082] The first semantic information is fused with the second semantic information to obtain output information after the input information is processed.

[0083] Optionally, the knowledge enhancement module is further specifically used to:

[0084] The first semantic information and the second semantic information are weighted according to their respective weights to obtain output information after the input information is processed.

[0085] Optionally, the large language model further includes a weight adaptation layer configured for each general knowledge layer. Before weighting the first semantic information and the second semantic information with their respective weights to obtain output information after the input information is processed, the knowledge enhancement module is further configured to:

[0086] The fine-screening information and the input information are processed through the weight adaptive layer to obtain the first semantic information and the second semantic information with their respective weights.

[0087] Optionally, the large language model further includes a cross-attention layer configured for each general knowledge layer, and the knowledge screening module is further specifically used to:

[0088] The input information and coarse screening information are processed through the cross attention layer to obtain fine screening information.

[0089] Optionally, the knowledge base includes multiple knowledge fragments, and the knowledge screening module is further specifically used to:

[0090] Obtain the matching degree between the information to be processed and each knowledge fragment;

[0091] Select the coarse-screened information based on the degree of match with each knowledge fragment.

[0092] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0093] It should also be understood that if the above embodiments are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0094] Therefore, this embodiment further provides a storage medium, which is a computer-readable storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the knowledge enhancement method provided in this embodiment is implemented. The storage medium can be a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, among other media that can store program code.

[0095] This embodiment provides an electronic device for implementing a knowledge enhancement method. Figure 5 As shown, the electronic device may include a processor 22 and a memory 21. In addition, the memory 21 stores a computer program, and the processor implements the knowledge enhancement method provided in this embodiment by reading and executing the computer program corresponding to the above embodiment in the memory 21.

[0096] Continue to see Figure 5 The electronic device further includes a communication unit 23. The memory 21, the processor 22 and the communication unit 23 are electrically connected to each other directly or indirectly via a system bus 24 to achieve data transmission or interaction.

[0097] The memory 21 may be an information recording device based on any electronic, magnetic, optical or other physical principles, for recording execution instructions, data, etc. In some embodiments, the memory 21 may be, but is not limited to, a volatile memory, a non-volatile memory, a storage drive, etc.

[0098] In some embodiments, the volatile memory may be a random access memory (RAM); in some embodiments, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, etc.; in some embodiments, the storage drive may be a magnetic disk drive, a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or a similar storage medium, or a combination thereof.

[0099] The communication unit 23 is used to send and receive data through a network. In some embodiments, the network may include a wired network, a wireless network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, or a near field communication (NFC) network, or any combination thereof. In some embodiments, the network may include one or more network access points. For example, the network may include a wired or wireless network access point, such as a base station and / or a network switching node, through which one or more components of the service request processing system can connect to the network to exchange data and / or information.

[0100] The processor 22 may be an integrated circuit chip having signal processing capabilities, and the processor may include one or more processing cores (e.g., a single-core processor or a multi-core processor). By way of example only, the processor may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC), or a microprocessor, or any combination thereof.

[0101] I understand. Figure 5 The structure shown is for reference only. Figure 5 More or fewer components than shown, or with Figure 5 Different configurations shown. Figure 5 The components shown may be implemented in hardware, software, or a combination thereof.

[0102] It should be understood that the devices and methods disclosed in the above embodiments may also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes may actually be executed substantially in parallel, or they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, may be implemented using a dedicated hardware-based system that performs a specified function or action, or may be implemented using a combination of dedicated hardware and computer instructions.

[0103] The above descriptions are merely examples of various embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any modifications or substitutions that can be readily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A knowledge enhancement method, characterized in that: The method comprises: receiving information to be processed from a large language model, wherein the large language model includes a plurality of general knowledge layers; Determining rough screening information from a knowledge base based on the information to be processed; For each of the general knowledge layers, extracting fine screening information from the coarse screening information using input information input into the general knowledge layer, and generating a knowledge enhancement model based on the fine screening information; The knowledge enhancement model is used to inject relevant knowledge required for processing the input information into the general knowledge layer to obtain output information after the input information is processed.

2. The knowledge enhancement method according to claim 1, characterized in that: Using the knowledge enhancement model to inject relevant knowledge required for processing the input information into the general knowledge layer to obtain output information after the input information is processed, including: Acquire first semantic information after the general knowledge layer processes the input information, and obtain second semantic information after the knowledge enhancement model processes the input information; The first semantic information is fused with the second semantic information to obtain output information after the input information is processed.

3. The knowledge enhancement method according to claim 2, characterized in that: Fusing the first semantic information with the second semantic information to obtain output information after processing the input information includes: The first semantic information and the second semantic information are weighted according to their respective weights to obtain output information after the input information is processed.

4. The knowledge enhancement method according to claim 3, characterized in that: The large language model further includes a weight adaptation layer configured for each of the general knowledge layers. Before weighting the first semantic information and the second semantic information with their respective weights to obtain output information after processing the input information, the method further includes: The fine screening information and the input information are processed by the weight adaptive layer to obtain the first semantic information and the second semantic information with their respective weights.

5. The knowledge enhancement method according to claim 1, characterized in that: The large language model further includes a cross attention layer configured for each of the general knowledge layers, which uses input information input to the general knowledge layer to extract fine screening information from the coarse screening information, including: The input information and the coarse screening information are processed by the cross attention layer to obtain the fine screening information.

6. The knowledge enhancement method according to claim 1, characterized in that: Each of the general knowledge layers is built on Transformer.

7. The knowledge enhancement method according to claim 1, characterized in that: The knowledge base includes multiple knowledge fragments. According to the information to be processed, coarse screening information is determined from the knowledge base, including: Obtaining a matching degree between the information to be processed and each of the knowledge fragments; The coarse screening information is selected based on the degree of matching with each of the knowledge fragments.

8. A knowledge enhancement device, characterized in that: The device comprises: an information receiving module, configured to receive information to be processed from a large language model, wherein the large language model includes a plurality of general knowledge layers; A knowledge screening module, configured to determine coarse screening information from a knowledge base based on the information to be processed; The knowledge screening module is further configured to extract fine screening information from the coarse screening information using input information input into the general knowledge layer for each general knowledge layer, and generate a knowledge enhancement model based on the fine screening information; The knowledge enhancement module is used to use the knowledge enhancement model to inject relevant knowledge required for processing the input information into the general knowledge layer to obtain output information after the input information is processed.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is processed and executed, it implements the knowledge enhancement method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores a computer program, and the computer program, when processed and executed, implements the knowledge enhancement method according to any one of claims 1 to 7.

Images