Enterprise business knowledge generation large model optimization method and device based on forgetting operation and knowledge reconfiguration, equipment and medium

Abstract

This application discloses a method, apparatus, device, and medium for optimizing a large-scale enterprise business knowledge generation model based on forgetting operations and knowledge reconstruction. It relates to the field of model forgetting technology, and includes: determining structured data based on target text containing enterprise business knowledge to construct an initial knowledge graph; determining the forgetting range corresponding to the forgetting operation in the initial knowledge graph to perform the forgetting operation and obtain a target knowledge graph; determining the target knowledge to be modified in the target text from the difference information between the graphs; generating a knowledge relevance distribution map based on the target knowledge and the large-scale enterprise business knowledge generation model to determine the target parameters in the large-scale model; and using the target knowledge and target parameters to determine the forgetting target to update the low-rank sparse adapter loaded at the target parameters in the large-scale enterprise business knowledge generation model, thereby completing the forgetting operation on the large-scale enterprise business knowledge generation model. This application enables efficient and accurate selective forgetting of knowledge by the large-scale model.

Description

Technical Field

[0001] This invention relates to the field of model forgetting technology, and in particular to a method, apparatus, equipment and medium for optimizing large-scale enterprise business knowledge generation models based on forgetting operations and knowledge reconstruction. Background Technology

[0002] Large Language Models (LLMs) are widely used in knowledge generation and natural language understanding. Their knowledge updates are mostly incremental injections, lacking a precise removal mechanism for redundant, outdated, and sensitive knowledge, making it difficult to meet the requirements of dynamic knowledge governance and compliance.

[0003] Existing knowledge forgetting technologies for large-scale enterprise business knowledge generation models have significant drawbacks: First, forgetting is costly, with existing machine forgetting methods involving huge computational and time costs, making them unsuitable for large-scale LLM models; second, the granularity of forgetting is coarse, lacking precise control over individual facts or knowledge points, and a single fine-tuning often leads to the unexpected degradation of a large amount of irrelevant knowledge; third, the timeliness of knowledge is poor, with LLM knowledge update cycles being long and unable to adapt to rapid changes in information; and fourth, the knowledge content is tightly bound to the model storage structure, making knowledge management and maintenance extremely complex.

[0004] Therefore, how to achieve efficient and accurate selective forgetting of knowledge in a large-scale enterprise business knowledge generation model is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide an optimization method, apparatus, device, and medium for a large-scale enterprise business knowledge generation model based on forgetting operations and knowledge reconstruction, enabling efficient and accurate selective forgetting of knowledge within the large-scale enterprise business knowledge generation model. The specific solution is as follows: Firstly, this application provides an optimization method for a large-scale enterprise business knowledge generation model based on forgetting operations and knowledge reconstruction, including: Structured data is determined based on target text containing enterprise business knowledge by pre-trained large models or sequence labeling models, and an initial enterprise business knowledge graph is constructed using the structured data. The target forgetting request is transformed into a forgetting operation applicable to the initial enterprise business knowledge graph. The forgetting range corresponding to the forgetting operation in the initial enterprise business knowledge graph is determined by the target model constructed based on the graph neural network. The forgetting operation is then performed on the initial enterprise business knowledge graph using the forgetting range to obtain the target enterprise business knowledge graph. The target forgetting request is a request to forget knowledge that meets the preset irrelevant condition with respect to the enterprise business knowledge. The difference information between the initial enterprise business knowledge graph and the target enterprise business knowledge graph is determined, and the target knowledge to be modified in the target text is determined from the difference information. An attention mechanism is used to generate a knowledge relevance distribution map based on the target knowledge and enterprise business knowledge in a large model. The target knowledge to be modified includes knowledge to be forgotten and knowledge to be suppressed. Based on the knowledge relevance distribution map, the target parameters in the enterprise business knowledge generation model are determined, and the forgetting target is determined using the target knowledge, the target parameters, and the sparse mapping algorithm. Based on the forgetting target, the low-rank sparse adapter loaded at the target parameter in the enterprise business knowledge generation model is updated to complete the forgetting operation of the enterprise business knowledge generation model and obtain the optimized enterprise business knowledge generation model.

[0006] Optionally, the step of determining structured data based on target text containing enterprise business knowledge through a pre-trained large model or sequence labeling model includes: A pre-trained large model or sequence labeling model is identified as the knowledge extractor. Based on the knowledge extractor and using preset prompt words or supervised learning methods, the target text containing enterprise business knowledge is converted into structured data. The structured data is data containing the entities, relationships, and attributes of the target text.

[0007] Optionally, performing the forgetting operation on the initial enterprise business knowledge graph using the forgetting range includes: Identify the third target knowledge in the initial enterprise business knowledge graph that satisfies the preset text association relationship, and generate a knowledge clearing list based on the knowledge in the initial enterprise business knowledge graph other than the third target knowledge; The forgetting operation is performed on the initial enterprise business knowledge graph based on the knowledge clearing list and the forgetting range.

[0008] Optionally, the step of generating a knowledge relevance distribution map based on the target knowledge and enterprise business knowledge using a large model includes: The target knowledge is projected into the latent space of the enterprise business knowledge generation model to obtain a query signal corresponding to the hidden state of the enterprise business knowledge generation model. Determine the hidden state representations of each layer in the enterprise business knowledge generation model and the activation states of the feedforward neural network, and determine the correlation between the hidden state representations and the activation states and the query signal; Based on the correlation of each layer in the enterprise business knowledge generation model, corresponding gating results are generated; the gating results are the results of neurons in the enterprise business knowledge generation model that satisfy a preset direct correlation with the target knowledge. A knowledge relevance distribution map is generated based on the gating results; the knowledge relevance distribution map includes the target layer in the enterprise business knowledge generation model that has a preset activation relationship with the target knowledge, as well as the target feedforward neural network and target neurons in the target layer.

[0009] Optionally, determining the forgotten target using the target knowledge, the target parameters, and the sparsity mapping algorithm includes: Based on the target knowledge, corresponding target samples are determined, and the operation of using the target samples to train the enterprise business knowledge generation model is determined as the first forgetting update target. The second forgetting update target is determined from the target enterprise business knowledge graph that retains knowledge that has a preset semantic similarity or a preset structural adjacency with the target knowledge, and the operation of using the retained knowledge to train the enterprise business knowledge generation model is determined. The operation of applying sparsity constraints to the target parameters in the enterprise business knowledge generation model using a sparse mapping algorithm is determined as the third forgetting update target.

[0010] Optionally, the operation of using a sparse mapping algorithm to apply sparsity constraints to the target parameters in the enterprise business knowledge generation model is determined as the third forgetting update target, including: The number of target parameters is compressed using gating pruning, sparsity regularization, and threshold shrinkage to obtain compressed target parameters. The operation of applying sparsity constraints to the compressed target parameters in the large model for generating enterprise business knowledge using a sparse mapping algorithm is determined as the third forgetting update target.

[0011] Optionally, updating the low-rank sparse adapter at the target parameter in the enterprise business knowledge generation model based on the forgetting target includes: The low-rank sparse adapter is loaded into the target position in the enterprise business knowledge generation model corresponding to the target parameter, and the low-rank sparse adapter is updated based on the preset sparsification constraints and the forgetting target.

[0012] Secondly, this application provides an optimization device for a large-scale enterprise business knowledge generation model based on forgetting operations and knowledge reconstruction, comprising: The graph construction module is used to determine structured data based on target text containing enterprise business knowledge through a pre-trained large model or sequence labeling model, so as to construct an initial enterprise business knowledge graph using the structured data. The graph forgetting module is used to transform a target forgetting request into a forgetting operation applicable to the initial enterprise business knowledge graph. It determines the forgetting range corresponding to the forgetting operation in the initial enterprise business knowledge graph using a target model constructed based on a graph neural network, and then performs the forgetting operation on the initial enterprise business knowledge graph using the forgetting range to obtain the target enterprise business knowledge graph. The target forgetting request is a request to forget knowledge that meets a preset condition of irrelevance to the enterprise business knowledge. The distribution map generation module is used to determine the difference information between the initial enterprise business knowledge graph and the target enterprise business knowledge graph, and to determine the target knowledge to be modified in the target text from the difference information. It uses an attention mechanism to generate a knowledge relevance distribution map based on the target knowledge and enterprise business knowledge to generate a large model. The target knowledge to be modified includes knowledge to be forgotten and knowledge to be suppressed. The large model forgetting module is used to determine the target parameters in the enterprise business knowledge generation large model based on the knowledge relevance distribution map, and to determine the forgetting target using the target knowledge, the target parameters and the sparse mapping algorithm. Based on the forgetting target, the low-rank sparse adapter loaded at the target parameter in the enterprise business knowledge generation large model is updated to complete the forgetting operation of the enterprise business knowledge generation large model and obtain the optimized enterprise business knowledge generation large model.

[0013] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned optimization method for large-scale enterprise business knowledge generation based on forgetting operations and knowledge reconstruction.

[0014] Fourthly, this application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned optimization method for generating a large enterprise business knowledge model based on forgetting operations and knowledge reconstruction.

[0015] In this application, structured data containing enterprise business knowledge is determined based on target text using a pre-trained large model or sequence labeling model, and an initial enterprise business knowledge graph is constructed using the structured data. A target forgetting request is transformed into a forgetting operation applicable to the initial enterprise business knowledge graph. A forgetting range corresponding to the forgetting operation in the initial enterprise business knowledge graph is determined using a target model built based on a graph neural network. The forgetting operation is then performed on the initial enterprise business knowledge graph using the forgetting range to obtain the target enterprise business knowledge graph. The target forgetting request is a request to forget knowledge that meets a preset irrelevant condition to the enterprise business knowledge. The initial enterprise business knowledge graph and the target enterprise business knowledge graph are determined. The system analyzes the difference information of the spectrum and determines the target knowledge to be modified in the target text from the difference information. It then uses an attention mechanism to generate a knowledge relevance distribution map based on the target knowledge and enterprise business knowledge to create a large-scale model. The target knowledge to be modified includes knowledge to be forgotten and knowledge to be suppressed. Based on the knowledge relevance distribution map, the system determines the target parameters in the large-scale model and uses the target knowledge, the target parameters, and a sparsity mapping algorithm to determine the forgetting target. Based on the forgetting target, the system updates the low-rank sparse adapter loaded at the target parameters in the large-scale model to complete the forgetting operation, resulting in an optimized large-scale model for enterprise business knowledge generation. As can be seen from the above, this application relies on a pre-trained large model or sequence labeling model to extract structured data from the target text carrying enterprise business knowledge, and uses this structured data to build an initial enterprise business knowledge graph. The target forgetting request is converted into a corresponding forgetting instruction matching the initial enterprise business knowledge graph. A target model built using a graph neural network identifies the forgetting interval within the initial enterprise business knowledge graph that matches the forgetting instruction. Forgetting processing is performed on the initial enterprise business knowledge graph based on this forgetting interval, thereby obtaining the target enterprise business knowledge graph. The initial enterprise business knowledge graph and the target enterprise business knowledge graph are compared to obtain their differences. Target knowledge within the target text that needs adjustment is selected from these differences. An attention mechanism is used to draw a knowledge relevance distribution map between the target knowledge and the original enterprise business knowledge. The corresponding target parameters of the enterprise business knowledge generation large model are locked according to the knowledge relevance distribution map. Specific forgetting points are locked by combining target knowledge, target parameters, and a sparse mapping algorithm. Iterative adjustments are performed on the low-rank sparse adapter corresponding to the target parameter positions within the enterprise business knowledge generation large model, thereby achieving knowledge forgetting processing of the enterprise business knowledge generation large model. Finally, the optimized enterprise business knowledge generation large model is output. In this way, this application enables the enterprise business knowledge generation model to achieve efficient and accurate selective forgetting of knowledge. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This application discloses a flowchart of an optimization method for generating a large-scale enterprise business knowledge model based on forgetting operations and knowledge reconstruction. Figure 2 This is a schematic diagram of a story-based memory mechanism disclosed in this application; Figure 3 This is a schematic diagram of a knowledge positioning attention mechanism training method disclosed in this application; Figure 4 This is a schematic diagram of the structure of an enterprise business knowledge generation large-scale model optimization device based on forgetting operations and knowledge reconstruction disclosed in this application; Figure 5 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Currently, existing knowledge forgetting technologies for large-scale enterprise business knowledge generation models have significant drawbacks: First, forgetting is costly, with existing machine forgetting methods involving enormous computational and time costs, making them unsuitable for large-scale LLM models; second, the granularity of forgetting is coarse, lacking precise control over individual facts or knowledge points, and a single fine-tuning often leads to the unexpected degradation of a large amount of irrelevant knowledge; third, knowledge timeliness is poor, with long knowledge update cycles in LLM models making it difficult to adapt to rapid information changes; and fourth, the tight binding between knowledge content and model storage structure makes knowledge management and maintenance extremely complex. To address these issues, this application provides an optimization method, apparatus, device, and medium for large-scale enterprise business knowledge generation models based on forgetting operations and knowledge reconstruction, enabling efficient and accurate selective forgetting of knowledge within the large-scale enterprise business knowledge generation model.

[0020] See Figure 1 As shown, this embodiment of the invention discloses an optimization method for a large-scale enterprise business knowledge generation model based on forgetting operations and knowledge reconstruction, including: Step S11: Determine structured data based on target text containing enterprise business knowledge using a pre-trained large model or sequence labeling model, and use the structured data to construct an initial enterprise business knowledge graph.

[0021] In this embodiment, the target text containing enterprise business knowledge is first obtained. This target text can originate from various internal business documents, including but not limited to product manuals, operating procedures, customer service records, project reports, and technical specifications. To extract structured information from the unstructured target text for subsequent model processing, this embodiment uses a pre-trained large model or sequence labeling model as the knowledge extractor. Based on this knowledge extractor and using preset prompts or supervised learning methods, the target text containing enterprise business knowledge is converted into structured data. This structured data consists of entities, relationships, and attributes of the target text. Entities can represent specific objects in the enterprise's business, such as products, departments, positions, or customers; relationships represent business connections between entities, such as affiliation, collaboration, or supply relationships; attributes represent the entity's own characteristic information, such as product specifications or departmental responsibilities.

[0022] After obtaining the structured data, an initial enterprise business knowledge graph is constructed using it. During this construction process, entities from the structured data are used as nodes in the knowledge graph, relationships between entities are used as edges, and attributes are stored as additional information for the nodes. This initial enterprise business knowledge graph provides the foundational data structure support for subsequent forgetting operations and knowledge reconstruction.

[0023] Step S12: Transform the target forget request into a forget operation applicable to the initial enterprise business knowledge graph. Determine the forget range in the initial enterprise business knowledge graph corresponding to the forget operation using a target model constructed based on a graph neural network. Perform the forget operation on the initial enterprise business knowledge graph using the forget range to obtain the target enterprise business knowledge graph. The target forget request is a request to forget knowledge that meets the preset irrelevant conditions of the enterprise business knowledge.

[0024] In this embodiment, a target forgetting request is first determined. This target forgetting request indicates the knowledge to be forgotten that meets a preset condition of irrelevance to the enterprise business knowledge. To transform the target forgetting request into a forgetting operation executable by the initial enterprise business knowledge graph, the target forgetting request can be parsed to extract the semantic description or keyword set of the knowledge to be forgotten. Then, the forgetting range corresponding to the forgetting operation in the initial enterprise business knowledge graph is determined by a target model built based on a graph neural network. After the initial enterprise business knowledge graph is input into the target model, the target model outputs the forgetting range.

[0025] After determining the forgetting range, the third target knowledge in the initial enterprise business knowledge graph that satisfies the preset text association relationship is further identified, and a knowledge clearing list is generated based on the knowledge in the initial enterprise business knowledge graph other than the third target knowledge. Finally, the forgetting operation is performed on the initial enterprise business knowledge graph based on the knowledge clearing list and the forgetting range, thereby obtaining the updated target enterprise business knowledge graph.

[0026] Step S13: Determine the difference information between the initial enterprise business knowledge graph and the target enterprise business knowledge graph, and determine the target knowledge to be modified in the target text from the difference information. Use an attention mechanism to generate a knowledge relevance distribution map based on the target knowledge and enterprise business knowledge generation model. The target knowledge to be modified includes knowledge to be forgotten and knowledge to be suppressed.

[0027] In this embodiment, the initial enterprise business knowledge graph is first compared with the target enterprise business knowledge graph to determine the differences between them. By analyzing the differences, the target knowledge to be modified corresponding to the aforementioned differences can be located from the original target text. The target knowledge to be modified includes knowledge to be forgotten and knowledge to be suppressed. After determining the target knowledge to be modified, the target knowledge is projected into the latent space of the enterprise business knowledge generation model to obtain the query signal corresponding to the hidden state of the enterprise business knowledge generation model; the hidden state representations and activation states of the feedforward neural networks of each layer in the enterprise business knowledge generation model are determined, and the correlation between the hidden state representations and activation states and the query signal is determined; based on the correlation of each layer in the enterprise business knowledge generation model, corresponding gating results are generated; the gating results are the results of neurons in the enterprise business knowledge generation model that satisfy a preset direct correlation with the target knowledge; a knowledge correlation distribution map is generated based on the gating results; the knowledge correlation distribution map includes the target layer in the enterprise business knowledge generation model that has a preset activation relationship with the target knowledge, as well as the target feedforward neural network and target neurons in the target layer. The aforementioned preset activation relationship can be set as the positional relationship of neurons whose relevance scores exceed a threshold in the gating results, thereby visually demonstrating which layers, feedforward neural networks, and neurons in the enterprise business knowledge generation model participate in the representation of target knowledge.

[0028] Step S14: Determine the target parameters in the enterprise business knowledge generation model based on the knowledge relevance distribution map, and determine the forgetting target using the target knowledge, the target parameters, and the sparse mapping algorithm. Update the low-rank sparse adapter loaded at the target parameters in the enterprise business knowledge generation model based on the forgetting target to complete the forgetting operation of the enterprise business knowledge generation model and obtain the optimized enterprise business knowledge generation model.

[0029] In this embodiment, based on the knowledge relevance distribution map, the target layers, target feedforward neural networks, and target neurons in the enterprise business knowledge generation model that have a preset activation relationship with the target knowledge to be modified are identified. Then, the corresponding target parameters are extracted from these target layers, target feedforward neural networks, and target neurons. After determining the target parameters, the forgetting targets need to be determined using the target knowledge, target parameters, and a sparsity mapping algorithm. The process of determining the forgetting targets specifically includes the following three aspects.

[0030] First, based on the target knowledge, corresponding target samples are determined, and the operation of training the enterprise business knowledge generation model using the target samples is determined as the first forgetting update target. Second, retained knowledge with a preset semantic similarity or preset structural adjacency relationship with the target knowledge is determined from the target enterprise business knowledge graph, and the operation of training the enterprise business knowledge generation model using the retained knowledge is determined as the second forgetting update target. Third, the operation of applying sparsity constraints to the target parameters in the enterprise business knowledge generation model using a sparse mapping algorithm is determined as the third forgetting update target.

[0031] Specifically, the number of target parameters is compressed using gating pruning, sparsity regularization, and threshold shrinkage to obtain compressed target parameters. The operation of applying sparsity constraints to the compressed target parameters in the enterprise business knowledge generation model using a sparsity mapping algorithm is determined as the third forgetting update target.

[0032] Finally, the final forgetting target is obtained by combining the first, second, and third forgetting update targets. A low-rank sparse adapter is loaded into the target position corresponding to the target parameter in the enterprise business knowledge generation model, and the low-rank sparse adapter is updated based on preset sparsity constraints and the forgetting target. Through iterative optimization, the adapter parameters simultaneously satisfy the forgetting target and sparsity constraints, thereby completing the forgetting operation on the enterprise business knowledge generation model and obtaining the optimized enterprise business knowledge generation model.

[0033] As can be seen from the above, this application relies on a pre-trained large model or sequence labeling model to extract structured data from the target text carrying enterprise business knowledge, and uses this structured data to build an initial enterprise business knowledge graph. The target forgetting request is converted into a corresponding forgetting instruction matching the initial enterprise business knowledge graph. A target model built using a graph neural network identifies the forgetting interval within the initial enterprise business knowledge graph that matches the forgetting instruction. Forgetting processing is performed on the initial enterprise business knowledge graph based on this forgetting interval, thereby obtaining the target enterprise business knowledge graph. The initial enterprise business knowledge graph and the target enterprise business knowledge graph are compared to obtain their differences. Target knowledge within the target text that needs adjustment is selected from these differences. An attention mechanism is used to draw a knowledge relevance distribution map between the target knowledge and the original enterprise business knowledge. The corresponding target parameters of the enterprise business knowledge generation large model are locked according to the knowledge relevance distribution map. Specific forgetting points are locked by combining target knowledge, target parameters, and a sparse mapping algorithm. Iterative adjustments are performed on the low-rank sparse adapter corresponding to the target parameter positions within the enterprise business knowledge generation large model, thereby achieving knowledge forgetting processing of the enterprise business knowledge generation large model. Finally, the optimized enterprise business knowledge generation large model is output. In this way, this application enables the enterprise business knowledge generation model to achieve efficient and accurate selective forgetting of knowledge.

[0034] The technical solutions of the embodiments of this application will be specifically described below through the module composition in this application.

[0035] Specifically, for knowledge extraction and graph (i.e., the initial enterprise business knowledge graph) construction, in the initialization phase, the system utilizes a specially trained small LLM or sequence labeling model as a knowledge extractor. This extractor, through the design of specific cue words or supervised learning methods, drives the pre-trained LLM to self-describe enterprise business-related knowledge (i.e., target text containing enterprise business knowledge), and transforms the output into structured triples (i.e., structured data), entities, relations, and attributes. Each entity and relation in the graph is mapped to a low-dimensional vector space, i.e., knowledge embedding.

[0036] For the forgetting decision and impact assessment module, forgetting requests (i.e., target forgetting requests) are first transformed into a set of operations on the graph. For example, the request to forget "Zhang San is the CEO of the company" is transformed into a deletion operation corresponding to the triple in the graph. Since knowledge in the knowledge graph is interconnected, deleting a node may affect all its neighboring nodes. In this embodiment, a forgetting impact range model is designed. This is a small auxiliary model (i.e., the target model) based on a graph neural network, whose sole purpose is to predict the degree of impact of forgetting operations on other irrelevant knowledge points in the graph. This model identifies cascading effects, calculates the forgetting sensitivity of each knowledge point, and thus defines a minimal and precise forgetting boundary (i.e., forgetting range) to ensure that forgotten knowledge is completely erased without being implicated by accidentally retained knowledge.

[0037] Reference Figure 2 As shown, this embodiment draws inspiration from human memory patterns: human memories are typically organized and associated in the form of storylines (or core concepts). The model in this solution also organizes the knowledge stored in the database into several interconnected main stories (i.e., third-party target knowledge). Each time a knowledge cleanup operation is performed, the system traverses and identifies redundant nodes unrelated to these core main stories, or "inferior nodes" associated with negative or harmful content, and automatically adds them to the cleanup list (i.e., the knowledge cleanup list), ensuring the efficiency and security compliance of the knowledge system. This dual-filtering model ensures the stability of important knowledge systems and allows the system, like humans, to periodically remove unimportant "details" while proactively purifying the model's memory, thereby efficiently freeing up memory space and enhancing the ability to focus on core knowledge and the security compliance of the content.

[0038] For the graph operations and knowledge embedding difference calculation module, a forgetting operation is performed, generating a new graph (i.e., the target enterprise's business knowledge graph). The operation can be either directly removing triples or attenuating the weights of specific knowledge, simulating the "forgetting" of knowledge. After the graph operations are completed, we recalculate the embedding vectors of the affected knowledge, obtaining a new set of embeddings. By calculating the difference between the original embeddings and the new embeddings, we obtain the knowledge embedding difference vector (i.e., difference information). This difference vector accurately captures the changes in the embedding space of the deleted or modified knowledge. Since the operation only targets a few knowledge points, this difference vector is highly sparse, with non-zero terms only existing at the positions corresponding to the entities and relations affected by the forgetting operation.

[0039] The parameter mapping and sparse update module is the core technology module of this invention. Its main function is to transform the "knowledge differences to be processed" identified in the knowledge graph space into local update behaviors in the parameter space of the large language model (i.e., the enterprise business knowledge generation model), thereby achieving precise modification, selective forgetting, or targeted suppression of specific knowledge. Unlike traditional full-parameter fine-tuning methods, this module does not train the entire model on a large scale. Instead, it first locates the activation regions of knowledge within the model, then selects a small number of parameters most relevant to that knowledge (i.e., target parameters), and finally updates only the low-rank sparse adapters corresponding to these parameters. In this way, efficient processing of target knowledge can be achieved without affecting other capabilities of the model as much as possible.

[0040] From an execution perspective, this module can be divided into three consecutive stages: The first stage is the knowledge localization attention mechanism, which answers the question of "which layers, neurons, and parameter regions the target knowledge mainly corresponds to within the model"; the second stage is the parameter sparsity mapping algorithm, which answers the question of "how to construct the update target within the already located parameter regions so that the model forgets the knowledge that should be forgotten while retaining the knowledge that should not be affected"; the third stage is the low-rank sparse update technique, which answers the question of "how to complete this update with the lowest parameter cost during specific training". These three stages are interconnected: the output of the previous stage directly becomes the input of the next stage, forming a complete technical loop.

[0041] The following section provides a detailed explanation of the parameter mapping and sparse update module process.

[0042] First, the system receives the knowledge discrepancy results output by the preceding modules. These results typically consist of two parts: one part is the target knowledge to be forgotten, suppressed, or modified; the other part is irrelevant or neighboring knowledge that needs to be retained. Based on these discrepancy results, the system constructs a target knowledge representation and transforms it into a knowledge embedding that can be input into the model. This knowledge embedding is not the final object used to update parameters, but rather serves as the input signal (i.e., the query signal) for the subsequent knowledge localization attention mechanism, used to find the activation region most relevant to that knowledge within the model.

[0043] Next, the system embeds the knowledge into a knowledge localization attention mechanism, jointly analyzing the hidden states, attention responses, and feedforward network activations at each layer during the model's inference process to generate a "knowledge relevance distribution map." This distribution map describes which layers the target knowledge primarily activates within the model, and within these layers, which FFN (Feed-Forward Network) channels and local neuron sets are mainly relied upon. In other words, the result generated in this step is not a direct parameter update, but rather a "knowledge-to-parameter region" localization result.

[0044] After obtaining the knowledge relevance distribution map, the system selects a subset of candidate parameters from the full model parameters based on this map. This subset of candidate parameters only contains local parameter regions that are highly relevant to the target knowledge, and its scope is usually concentrated in the feedforward network parameters of several layers and their additional adapter locations, without extending to all layers of the model. Through this step, the system transforms the "knowledge localization result" into the "parameter range to be updated".

[0045] Then, the system constructs a parameter sparsity mapping objective on this subset of candidate parameters, simultaneously executing the training design around three objectives: First, to minimize the model's responsiveness to target knowledge, ensuring that the knowledge to be forgotten cannot be stably activated or correctly output; second, to maintain the model's output stability for irrelevant knowledge and general tasks, avoiding catastrophic forgetting; and third, to impose strong sparsity constraints on all parameter variations, keeping the number of parameters that actually change at an extremely low level. Thus, the system completes the mapping from "knowledge differences" to "training objectives."

[0046] Finally, instead of directly modifying the main parameters of the original large model, the system loads low-rank sparse adapters at the positions corresponding to the candidate parameter subsets and only trains and updates these adapters. After training, the resulting adapter updates constitute the minimum perturbation modification to the original model. The system then validates the updated model based on the target knowledge test set, the irrelevant knowledge retention set, and the general capability validation set, confirming that the forgetting effectiveness, retention stability, and update sparsity all meet the requirements. At this point, this module completes closed-loop execution.

[0047] It should be noted that, referring to Figure 3 As shown, the knowledge localization attention mechanism is the starting point of this module. Its core objective is not to directly train the model, but to first establish a mapping relationship between "graph knowledge" and "model internal parameter regions." Existing large-scale model knowledge editing methods often directly make localized, tentative modifications to parameters, lacking a clear knowledge localization process, which can easily lead to an excessively large update scope and uncontrollable impact. This invention introduces a knowledge localization attention mechanism, giving subsequent updates a clear source basis: that is, only those parameter regions that are significantly activated by the target knowledge during the inference process will be included in the subsequent update candidates.

[0048] In its implementation, the system first embeds and encodes the graph substructure corresponding to the target knowledge to form a unified knowledge representation. This knowledge representation is then projected into the model's latent space to form a query signal that can be aligned with the hidden states within the Transformer. This query signal, upon entering the model, does not directly replace the original input but serves as an auxiliary localization signal, performing similarity calculations with the hidden states generated during the inference process at each layer of the model.

[0049] In each layer, the system extracts the hidden state representation and feedforward network intermediate activations, and calculates their correlation (i.e., relevance) with the target knowledge query signal. If the hidden state in a layer highly matches the target knowledge representation, it indicates that the layer plays a more crucial role in processing the knowledge; if the FFN channel in a layer exhibits a consistently high response in the correlation calculation, it indicates that the channel is more likely to perform the function of storing or representing the target knowledge. The system combines this hierarchical correlation with the channel-level correlation to form a knowledge relevance score layer by layer.

[0050] After obtaining the relevance scores for each layer, the system further generates intra-layer gating results. These gating results can be represented as a binary mask or sparse gating weights, and their purpose is to explicitly indicate which subsets of neurons in this layer are directly related to the target knowledge, and which neurons can be considered irrelevant background and ignored. Through this process, the original large-scale parameter space in the model is compressed into a few knowledge-related regions.

[0051] The output of the knowledge localization attention mechanism is not the final model output, but rather a set of structured "knowledge localization results." These results contain at least three types of information: first, a set of model layers highly correlated with the target knowledge; second, a subset of significantly activated FFN neurons in each correlated layer; and third, the candidate parameter regions corresponding to these neuron subsets. This output will be directly used in the subsequent parameter sparsity mapping algorithm: the system will use these localization results as filtering criteria to determine the range of parameters truly allowed to participate in the update.

[0052] In other words, the "knowledge-related region" generated in this step does not remain at the analysis level, but is directly transformed into a subset of candidate parameters in the next step. This forms a clear causal chain: first, a location result is generated through KLA (Knowledge Location Attention), and then the updatable parameter region is determined based on the location result. This makes the entire technical solution not only theoretically explainable, but also practically feasible in its execution process.

[0053] It should be further explained that, for the parameter sparsity mapping algorithm, after obtaining the knowledge-related regions through the knowledge localization attention mechanism, the system already knows "which parameter regions may be related to the target knowledge," but it has not yet solved the problem of "how these parameters should be updated." The role of the parameter sparsity mapping algorithm is to further transform the knowledge localization results obtained in the previous stage into executable training objectives and update rules, thereby ensuring that the model does not damage other irrelevant knowledge and general capabilities while forgetting the target knowledge.

[0054] The system first extracts local parameter regions from the parameter matrices of each layer of FFN and the locations of their pluggable adapters based on the knowledge localization results output in the previous stage, forming a subset of candidate parameters. The key point here is that the subset of candidate parameters is not manually fixed, but dynamically provided by the knowledge localization attention mechanism. That is, if the target knowledge mainly corresponds to certain neuron channels in certain layers, then only the parameters related to these layers and channels will enter the candidate update set; other layers and other channels remain frozen and do not participate in training.

[0055] Through this construction, the system achieves a precise mapping "from knowledge differences to parameter regions". This mapping is neither a coarse-grained full-layer update nor a random sampling of a small number of parameters, but rather a local parameter set generated based on the activation rules within the model of the target knowledge. Therefore, it has stronger targeting and interpretability.

[0056] After the candidate parameter subset is established, the system designs update rules around three parallel objectives.

[0057] The first type of objective (also known as the first forgetting update objective) is the forgetting objective. The system requires that the updated model, when faced with cues related to the knowledge to be forgotten, should no longer be able to consistently recover the corresponding facts, relationships, or answers. Specifically, the system constructs target knowledge query samples and, through training, suppresses the model's original response tendency to these samples, thus disrupting or weakening the activation pathways of the target knowledge within the model. In this way, even if the model receives relevant cues, it will no longer output the content that was asked to be forgotten.

[0058] The second type of objective (i.e., the second forgetting update objective) is the preservation objective. Because knowledge in large language models is highly coupled, focusing solely on suppressing target knowledge can easily damage surrounding knowledge. Therefore, the system simultaneously introduces an irrelevant knowledge preservation mechanism, constraining retained knowledge that is unrelated to the target knowledge but semantically similar or structurally adjacent. During training, the system requires the model's output on these retained samples to be as consistent as possible with the original model. This effectively prevents local forgetting from evolving into widespread capability decay.

[0059] The third type of objective (also known as the third forgetting update objective) is the sparsity objective. The system not only requires effective updates but also minimizes the number of updates. Therefore, during training, the system imposes strong sparsity constraints on the changes in a subset of candidate parameters, encouraging only a very small number of truly critical parameter positions to change, while the remaining parameters tend to remain unchanged. This reduces training costs and minimizes overall perturbation to the original model.

[0060] To ensure that "a small number of parameter updates" are truly achieved, this invention employs a hierarchical sparsity control strategy during the parameter sparsification mapping process.

[0061] The first layer is hierarchical sparsity control. Only layers deemed highly relevant in the knowledge-localization attention mechanism are allowed to enter the candidate update region; all inactive layers are frozen. Through this layer of control, the update scope is first reduced from "all model layers" to "a few relevant layers".

[0062] The second layer is neuron-level sparse control. Within the selected layer, the system only allows parameters corresponding to neuron channels marked as highly relevant by the knowledge-localization attention mechanism to enter the update candidate list, instead of training the entire layer's parameters as a whole. Through this layer of control, the update scope is further reduced from "the entire layer's parameters" to "a small number of channel parameters".

[0063] The third layer is parameter-level sparsity control. Within the already selected subset of candidate parameters, the system continues to compress the number of parameters that actually change through methods such as gating pruning, sparsity regularization, and threshold shrinking. This means that even if some parameters enter the candidate set, they will not necessarily be updated in the end. Only those parameters that have been consistently proven effective in preventing the forgetting of the target during training and have little impact on knowledge retention will be truly retained.

[0064] Through the above three-layer control, the system can further compress the "candidate parameters" into "actual update parameters", thereby significantly enhancing the accuracy and low-disturbance characteristics of the present invention.

[0065] Finally, it should be noted that for the low-rank sparse update technique, this invention adopts a sparsification variant of the low-rank adaptation idea. Instead of directly modifying the main weights of the original model, it loads low-rank sparse adapters in the relevant parameter regions and only updates the parameters of these adapters.

[0066] The advantage of this design is that the main parameters of the original large model remain frozen, and all knowledge processing is concentrated in a small number of additional adapters. On the one hand, this significantly reduces the size of training parameters and computational consumption; on the other hand, it also facilitates the subsequent individual saving, rollback, replacement, or combined deployment of modified results.

[0067] The system inserts a low-rank adapter matrix into the layer selected by the knowledge-localization attention mechanism and at the corresponding positions of the candidate parameter subset generated by the parameter sparsity mapping algorithm. Unlike conventional low-rank adaptation, the adapter in this invention does not apply uniformly to the entire layer, but only to the labeled local channels and local parameter regions. In other words, the adapter itself not only has low-rank characteristics, but also explicit spatial sparsity.

[0068] Specifically, the system constructs a corresponding adapter module for each selected layer based on the hierarchical localization results and neuron localization results output in the previous stage. If a layer is not marked as highly relevant by the knowledge localization attention mechanism, the adapter is not loaded for that layer; if a layer is marked as highly relevant, but only a few neuron channels are related to the target knowledge, the adapter only covers these relevant channels and does not extend to the entire layer.

[0069] At the start of training, the system first initializes the activation range of the adapter based on the knowledge localization results. In other words, the knowledge localization attention mechanism determines "which locations are worth updating," the parameter sparsity mapping algorithm determines "what forgetting and retention objectives these locations should undertake," and the low-rank sparse update technique is specifically responsible for performing training at these locations. These three elements are in a progressive relationship.

[0070] During forward propagation, the input samples undergo both routine calculations of the original model's main parameters and are loaded onto a low-rank sparse adapter in local regions. The local increments generated by the adapter do not replace the original parameters but rather serve as a fine-tuning supplement to the behavior of the original parameters. This allows the model to gradually deviate from its original output on the target knowledge samples while maintaining consistency with the original model as much as possible on the retained knowledge samples.

[0071] During backpropagation, the system keeps the main parameters frozen and only updates the gradients of the low-rank sparse adapter. In other words, what is actually trained is not the original model itself, but rather a small number of additional adapter parameters. This training method naturally aligns with the overall goal of this invention: "low perturbation and controllable modification."

[0072] To ensure that adapter updates are not only low-rank but also truly sparse, the system applies strong sparsity regularization constraints to the adapter during training. These constraints are mainly reflected in three aspects.

[0073] First, the number of projection directions within the adapter is compressed to prevent the adapter from gradually expanding to a large number of irrelevant directions during training. This ensures that the adapter always focuses on a small number of representation subspaces directly related to the target knowledge.

[0074] Second, the coverage of the adapter's write-back to the FFN channel is constrained to prevent the adapter from spreading local updates to large areas of neurons. This ensures that the impact of the update remains limited to the key regions selected by the knowledge localization mechanism.

[0075] Third, the activation gating of the adapter is sparsely pruned, retaining only those local update positions that consistently demonstrate high value during training, while the remaining positions gradually decay or are cleared to zero during training iterations. Through this strategy, the adapter eventually shrinks into a very small but highly effective parameter patch.

[0076] As can be seen, sparsity regularization is not an independent additional constraint here, but is closely connected with the preceding steps: the knowledge localization attention mechanism first provides the initial localization range, the parameter sparsity mapping algorithm then determines the update target, and sparsity regularization further compresses the "candidate update positions" into the "final retained positions". Therefore, the entire module forms a continuous technical process from coarse localization to fine screening, and then to actual compression and update.

[0077] Accordingly, see Figure 4 As shown in the figure, this application provides an optimization device for a large-scale enterprise business knowledge generation model based on forgetting operations and knowledge reconstruction, including: The graph construction module 11 is used to determine structured data based on target text containing enterprise business knowledge through a pre-trained large model or sequence labeling model, so as to construct an initial enterprise business knowledge graph using the structured data. The graph forgetting module 12 is used to transform a target forgetting request into a forgetting operation applicable to the initial enterprise business knowledge graph. It determines the forgetting range corresponding to the forgetting operation in the initial enterprise business knowledge graph using a target model constructed based on a graph neural network, and then performs the forgetting operation on the initial enterprise business knowledge graph using the forgetting range to obtain the target enterprise business knowledge graph. The target forgetting request is a request to forget knowledge that meets a preset condition of irrelevance to the enterprise business knowledge. The distribution map generation module 13 is used to determine the difference information between the initial enterprise business knowledge graph and the target enterprise business knowledge graph, and to determine the target knowledge to be modified in the target text from the difference information. It uses an attention mechanism to generate a knowledge relevance distribution map based on the target knowledge and enterprise business knowledge generation model. The target knowledge to be modified includes knowledge to be forgotten and knowledge to be suppressed. The large model forgetting module 14 is used to determine the target parameters in the enterprise business knowledge generation large model based on the knowledge relevance distribution map, and to determine the forgetting target using the target knowledge, the target parameters and the sparse mapping algorithm. Based on the forgetting target, the low-rank sparse adapter loaded at the target parameter in the enterprise business knowledge generation large model is updated to complete the forgetting operation of the enterprise business knowledge generation large model and obtain the optimized enterprise business knowledge generation large model.

[0078] In some specific embodiments, the atlas construction module 11 specifically includes: The text conversion unit is used to determine a pre-trained large model or sequence labeling model as a knowledge extractor, and based on the knowledge extractor, use preset prompt words or supervised learning methods to convert target text containing enterprise business knowledge into structured data; the structured data is data containing the entities, relationships and attributes of the target text.

[0079] In some specific embodiments, the map forgetting module 12 specifically includes: The list generation unit is used to determine the third target knowledge in the initial enterprise business knowledge graph that satisfies the preset text association relationship, and generate a knowledge clearing list based on the knowledge in the initial enterprise business knowledge graph other than the third target knowledge. The graph forgetting unit is used to perform the forgetting operation on the initial enterprise business knowledge graph based on the knowledge clearing list and the forgetting range.

[0080] In some specific embodiments, the distribution map generation module 13 specifically includes: The signal generation unit is used to project the target knowledge into the latent space of the enterprise business knowledge generation model to obtain a query signal corresponding to the hidden state of the enterprise business knowledge generation model. The correlation determination unit is used to determine the hidden state representations of each layer and the activation states of the feedforward neural network in the enterprise business knowledge generation big model, and to determine the correlation between the hidden state representations and the activation states and the query signal. The result generation unit is used to generate corresponding gating results based on the correlation of each layer in the enterprise business knowledge generation model; the gating results are the results of neurons in the enterprise business knowledge generation model that satisfy a preset direct correlation with the target knowledge. The distribution map generation unit is used to generate a knowledge relevance distribution map based on the gating result; the knowledge relevance distribution map includes the target layer in the enterprise business knowledge generation model that has a preset activation relationship with the target knowledge, as well as the target feedforward neural network and target neurons in the target layer.

[0081] In some specific embodiments, the large model forgetting module 14 specifically includes: The first target determination unit is used to determine the corresponding target samples based on the target knowledge, and to determine the operation of training the enterprise business knowledge generation model using the target samples as the first forgetting update target. The second target determination unit is used to determine the retained knowledge that has a preset semantic similarity relationship or a preset structural adjacency relationship with the target knowledge from the target enterprise business knowledge graph, and to determine the operation of using the retained knowledge to train the enterprise business knowledge generation model as the second forgetting update target. The third target determination unit is used to determine the operation of applying sparsity constraints to the target parameters in the enterprise business knowledge generation model using a sparse mapping algorithm as the third forgetting update target.

[0082] In some specific embodiments, the third target determination unit specifically includes: The parameter determination subunit is used to compress the number of the target parameters using gating pruning, sparsity regularization and threshold shrinkage to obtain the compressed target parameters; The target determination subunit is used to determine the operation of applying sparsity constraints to the compressed target parameters in the large model for generating enterprise business knowledge using a sparse mapping algorithm as the third forgetting update target.

[0083] In some specific embodiments, the large model forgetting module 14 specifically includes: The adapter update unit is used to load the low-rank sparse adapter into the target position in the enterprise business knowledge generation model corresponding to the target parameter, and update the low-rank sparse adapter based on the preset sparsification constraints and the forgetting target.

[0084] Furthermore, embodiments of this application also disclose an electronic device, Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the enterprise business knowledge generation large model optimization method based on forgetting operations and knowledge reconstruction disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0085] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0086] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0087] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the enterprise business knowledge generation large model optimization method based on forgetting operations and knowledge reconstruction, which is executed by the electronic device 20 according to any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0088] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed method for optimizing a large-scale enterprise business knowledge generation model based on forgetting operations and knowledge reconstruction. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0089] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0090] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0091] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0092] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0093] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for optimizing a large-scale enterprise business knowledge generation model based on forgetting operations and knowledge reconstruction, characterized in that, include: Structured data is determined based on target text containing enterprise business knowledge by pre-trained large models or sequence labeling models, and an initial enterprise business knowledge graph is constructed using the structured data. The target forgetting request is transformed into a forgetting operation applicable to the initial enterprise business knowledge graph. The forgetting range corresponding to the forgetting operation in the initial enterprise business knowledge graph is determined by the target model constructed based on the graph neural network. The forgetting operation is then performed on the initial enterprise business knowledge graph using the forgetting range to obtain the target enterprise business knowledge graph. The target forgetting request is a request to forget knowledge that meets the preset condition of being irrelevant to the enterprise's business knowledge; The difference information between the initial enterprise business knowledge graph and the target enterprise business knowledge graph is determined, and the target knowledge to be modified in the target text is determined from the difference information. An attention mechanism is used to generate a knowledge relevance distribution map based on the target knowledge and enterprise business knowledge in a large model. The target knowledge to be modified includes knowledge to be forgotten and knowledge to be suppressed. Based on the knowledge relevance distribution map, the target parameters in the enterprise business knowledge generation model are determined, and the forgetting target is determined using the target knowledge, the target parameters, and the sparse mapping algorithm. Based on the forgetting target, the low-rank sparse adapter loaded at the target parameter in the enterprise business knowledge generation model is updated to complete the forgetting operation of the enterprise business knowledge generation model and obtain the optimized enterprise business knowledge generation model.

2. The optimization method for enterprise business knowledge generation based on forgetting operations and knowledge reconstruction according to claim 1, characterized in that, The process of determining structured data based on target text containing enterprise business knowledge through pre-trained large models or sequence labeling models includes: A pre-trained large model or sequence labeling model is identified as the knowledge extractor. Based on the knowledge extractor and using preset prompt words or supervised learning methods, the target text containing enterprise business knowledge is converted into structured data. The structured data is data containing the entities, relationships, and attributes of the target text.

3. The optimization method for enterprise business knowledge generation based on forgetting operations and knowledge reconstruction according to claim 1, characterized in that, The step of performing the forgetting operation on the initial enterprise business knowledge graph using the forgetting range includes: Identify the third target knowledge in the initial enterprise business knowledge graph that satisfies the preset text association relationship, and generate a knowledge clearing list based on the knowledge in the initial enterprise business knowledge graph other than the third target knowledge; The forgetting operation is performed on the initial enterprise business knowledge graph based on the knowledge clearing list and the forgetting range.

4. The optimization method for enterprise business knowledge generation based on forgetting operations and knowledge reconstruction according to claim 1, characterized in that, The generation of a knowledge relevance distribution map based on the target knowledge and enterprise business knowledge includes: The target knowledge is projected into the latent space of the enterprise business knowledge generation model to obtain a query signal corresponding to the hidden state of the enterprise business knowledge generation model. Determine the hidden state representations of each layer in the enterprise business knowledge generation model and the activation states of the feedforward neural network, and determine the correlation between the hidden state representations and the activation states and the query signal; Based on the correlation of each layer in the enterprise business knowledge generation model, corresponding gating results are generated; the gating results are the results of neurons in the enterprise business knowledge generation model that satisfy a preset direct correlation with the target knowledge. A knowledge relevance distribution map is generated based on the gating results; the knowledge relevance distribution map includes the target layer in the enterprise business knowledge generation model that has a preset activation relationship with the target knowledge, as well as the target feedforward neural network and target neurons in the target layer.

5. The optimization method for enterprise business knowledge generation based on forgetting operations and knowledge reconstruction according to claim 1, characterized in that, The step of determining the forgotten target using the target knowledge, the target parameters, and the sparsity mapping algorithm includes: Based on the target knowledge, corresponding target samples are determined, and the operation of using the target samples to train the enterprise business knowledge generation model is determined as the first forgetting update target. The second forgetting update target is determined from the target enterprise business knowledge graph that retains knowledge that has a preset semantic similarity or a preset structural adjacency with the target knowledge, and the operation of using the retained knowledge to train the enterprise business knowledge generation model is determined. The operation of applying sparsity constraints to the target parameters in the enterprise business knowledge generation model using a sparse mapping algorithm is determined as the third forgetting update target.

6. The optimization method for enterprise business knowledge generation based on forgetting operations and knowledge reconstruction according to claim 5, characterized in that, The operation of applying sparsity constraints to the target parameters in the enterprise business knowledge generation model using a sparse mapping algorithm is determined as the third forgetting update target, including: The number of target parameters is compressed using gating pruning, sparsity regularization, and threshold shrinkage to obtain compressed target parameters. The operation of applying sparsity constraints to the compressed target parameters in the large model for generating enterprise business knowledge using a sparse mapping algorithm is determined as the third forgetting update target.

7. The optimization method for a large-scale enterprise business knowledge generation model based on forgetting operations and knowledge reconstruction according to any one of claims 1 to 6, characterized in that, The step of updating the low-rank sparse adapter at the target parameter in the enterprise business knowledge generation model based on the forgetting objective includes: The low-rank sparse adapter is loaded into the target position in the enterprise business knowledge generation model corresponding to the target parameter, and the low-rank sparse adapter is updated based on the preset sparsification constraints and the forgetting target.

8. A device for optimizing a large-scale enterprise business knowledge generation model based on forgetting operations and knowledge reconstruction, characterized in that, include: The graph construction module is used to determine structured data based on target text containing enterprise business knowledge through a pre-trained large model or sequence labeling model, so as to construct an initial enterprise business knowledge graph using the structured data. The graph forgetting module is used to transform the target forgetting request into a forgetting operation applicable to the initial enterprise business knowledge graph. It determines the forgetting range in the initial enterprise business knowledge graph corresponding to the forgetting operation through a target model built based on a graph neural network, and performs the forgetting operation on the initial enterprise business knowledge graph using the forgetting range to obtain the target enterprise business knowledge graph. The target forgetting request is a request to forget knowledge that meets the preset condition of being irrelevant to the enterprise's business knowledge; The distribution map generation module is used to determine the difference information between the initial enterprise business knowledge graph and the target enterprise business knowledge graph, and to determine the target knowledge to be modified in the target text from the difference information. It also uses an attention mechanism to generate a knowledge relevance distribution map based on the target knowledge and enterprise business knowledge in a large model. The target knowledge to be modified includes knowledge to be forgotten and knowledge to be suppressed. The large model forgetting module is used to determine the target parameters in the enterprise business knowledge generation large model based on the knowledge relevance distribution map, and to determine the forgetting target using the target knowledge, the target parameters and the sparse mapping algorithm. Based on the forgetting target, the low-rank sparse adapter loaded at the target parameter in the enterprise business knowledge generation large model is updated to complete the forgetting operation of the enterprise business knowledge generation large model and obtain the optimized enterprise business knowledge generation large model.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the enterprise business knowledge generation large model optimization method based on forgetting operations and knowledge reconstruction as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the enterprise business knowledge generation large model optimization method based on forgetting operations and knowledge reconstruction as described in any one of claims 1 to 7.

Images