Method for controlling target model to forget data, dialogue method, equipment and medium

Abstract

The embodiment of the invention provides a method for controlling a target model to forget data, a dialogue method, equipment and a medium, the target model is obtained based on original sample set training, and the method comprises the following steps: in response to a forgetting request for target data, determining a target sample subset containing the target data from the original sample set, generating a hedging sample subset which does not contain the target data based on the target sample subset; inputting the target sample subset and the hedging sample subset into a target model for processing to obtain a processing result, and updating original measurement information of a parameter space of the target model based on the processing result to obtain target measurement information; based on the processing result, determining a parameter updating direction required by the target model to forget the target sample subset and memorize the hedging sample subset; and determining a parameter change amount of the target model based on the target measurement information and the parameter updating direction, and updating the parameters of the target model based on the parameter change amount.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a method, dialogue method, device and medium for controlling the forgetting of data in a target model. Background Technology

[0002] With the development of artificial intelligence technology, artificial intelligence models (such as large language models) are widely used. Users can input questions into large language models, and the large language models can generate corresponding answers (such as answer text) based on the questions and output them.

[0003] Large language models require training data before use, which may contain sensitive information such as user privacy or trade secrets. This could lead to data leaks as the content generated by the large language model during use may contain such sensitive data. Therefore, how to limit the output of sensitive data by large language models to improve data security is a technical issue that needs to be considered.

[0004] The information in the background section is merely information known only to the inventor and does not imply that such information had entered the public domain before the date of this application, nor does it imply that it can be considered prior art in this disclosure. Summary of the Invention

[0005] This specification provides a method, dialogue method, device, and medium for controlling the forgotten data of a target model. It can control the target model to prevent it from outputting specified target data (such as sensitive data) during use, thereby reducing the probability of target data leakage and improving data security.

[0006] In a first aspect, this specification provides a method for controlling the forgetting of data in a target model, the target model being trained based on an original sample set. The method includes: in response to a request to forget target data, determining a target sample subset containing the target data from the original sample set, and generating a hedging sample subset without the target data based on the target sample subset; inputting the target sample subset and the hedging sample subset into the target model for processing to obtain a processing result, and updating the original metric information of the parameter space of the target model based on the processing result to obtain target metric information, wherein the original metric information characterizes the sensitivity information of the target model in different directions in the parameter space after training based on the original sample set, and the target metric information characterizes the sensitivity information of the target model in different directions in the parameter space after forgetting the target sample subset and remembering the hedging sample subset; based on the processing result, determining the parameter update direction required by the target model to forget the target sample subset and remember the hedging sample subset; and based on the target metric information and the parameter update direction, determining the parameter change amount of the target model, and updating the parameters of the target model based on the parameter change amount.

[0007] Secondly, this specification also provides a dialogue method, comprising: obtaining a question text; inputting the question text into a target model, wherein the target model is trained based on an original sample set and configured to have forgotten target data in the original sample set, the forgetting process of the target model for the target data is implemented using the method described in the first aspect; and obtaining an answer text output by the target model, wherein the answer text does not contain the target data.

[0008] Thirdly, this specification also provides an electronic device, comprising: at least one storage medium storing at least one instruction set; and at least one processor communicatively connected to the at least one storage medium, wherein the at least one processor reads the at least one instruction set and executes the method described in the first or second aspect according to the instructions of the at least one instruction set.

[0009] Fourthly, this specification also provides a computer-readable non-volatile storage medium, wherein the computer-readable non-volatile storage medium stores at least one instruction set, which, when executed by at least one processor, implements the method described in the first or second aspect above.

[0010] This specification provides a method, dialogue method, device, and medium for controlling the forgetting of target model data. Based on the target model, it processes target samples containing the target data to be forgotten and constructed hedging samples. Based on the processing results, it updates the original metric information of the target model's parameter space, obtaining the target metric information that the parameter space should possess after the target model forgets the target data, and determines the parameter update direction of the target model. Based on the target metric information and the parameter update direction, it determines the parameter change information of the target model to update the target model's parameters. This allows for the determination of how to adjust the model parameters without further training the target model, efficiently controlling the forgetting of target data, reducing the probability of data leakage caused by the model outputting target data during use, and improving data security. Furthermore, this method eliminates the need for further model training, directly adjusting parameters, which reduces the risk of adverse effects on other capabilities of the target model due to model training, and maintains model performance.

[0011] Other functionalities of the methods, dialogue methods, devices, and media for controlling forgotten data in target models provided in this specification will be partially listed in the following description. The inventive aspects of the methods, dialogue methods, devices, and media for controlling forgotten data in target models provided in this specification can be fully explained through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A schematic diagram illustrating an application scenario of a method for controlling forgotten data of a target model according to embodiments of this specification is shown. Figure 2 A hardware structure diagram of an electronic device provided according to an embodiment of this specification is shown; Figure 3 A flowchart is shown of a method for controlling forgotten data of a target model according to an embodiment of this specification; Figure 4 A general schematic diagram of a model update process provided according to embodiments of this specification is shown; and Figure 5 A flowchart of a dialogue method provided according to an embodiment of this specification is shown. Detailed Implementation

[0014] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0015] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not preclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or the possibility of adding other features, integers, steps, operations, elements, components, and / or groups to the system / method. The term “multiple” refers to two or more, and “at least one” refers to one or more. “At least one of A, B, or C” includes seven options: A only, B only, C only, containing both A and B, containing both A and C, containing both B and C, and containing both A, B, and C. And so on, the statement “at least one of multiple items” refers to all combinations that can be formed based on these items.

[0016] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0017] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0018] In this specification, the Large Language Model (LLM) may also be referred to simply as the Large Model. A Large Language Model is a natural language processing model based on deep learning techniques, typically with billions to hundreds of billions or even more parameters, possessing powerful language understanding and generation capabilities. Large Language Models can employ the Transformer architecture or its variants (such as GPT, BERT, etc.), which utilizes an attention mechanism to globally model sequential data, efficiently handling long-distance dependencies and thus performing exceptionally well in natural language tasks. Large Language Models learn the statistical features and semantic relationships of language through pre-training on large-scale corpora, giving them good generalization capabilities. The core capabilities of Large Language Models include, but are not limited to: understanding contextual semantics, generating coherent and grammatically correct text, performing logical reasoning, and handling multi-task scenarios. Its usage typically includes two modes: direct inference and fine-tuning. In direct inference mode, the user guides the Large Language Model to generate specific outputs by designing prompts. Cue words can be task descriptions or instructions in text form, used to stimulate the semantic understanding and generation capabilities of large language models. In fine-tuning mode, large language models are further trained on small-scale datasets in specific domains to optimize their performance on specific tasks. The powerful generalization ability and flexibility of large language models make them an important tool in the field of artificial intelligence, providing efficient and accurate solutions for automated text generation and understanding.

[0019] In some embodiments, large language models can also understand and generate data from other modalities (such as visual and audio data). In this case, large language models can also be called multimodal large language models (MLLMs). MLLMs provide a richer and more natural interactive experience by integrating multiple types of input and output, such as text, images, and sound. The core advantage of MLLMs lies in their ability to process and understand information from different modalities and fuse this information to complete complex tasks. For example, MLLMs can analyze an image and generate descriptive text, or generate a corresponding image based on a text description. This cross-modal understanding and generation capability makes MLLMs widely applicable across multiple fields.

[0020] It should be noted that the key technologies of large language models can be found in the detailed description in the paper "A Survey of Large Language Models" (paper number: arXiv:2303.18223v16, published on March 11, 2025, public link: https: / / doi.org / 10.48550 / arXiv.2303.18223), and will not be repeated here.

[0021] With the development of artificial intelligence (AI) technology, AI models are widely used, leading to increasingly stringent requirements for data security and compliance during model usage. Various models require extensive training with large amounts of sample data to enable them to perform specific tasks; for example, the rapidly developing large language models are trained on massive amounts of internet text data. During use, models may output content from the sample data used in their training, some of which is sensitive and may involve user privacy or trade secrets, making it unsuitable for public disclosure. Attackers can use deliberately designed attacks to force models to output this sensitive data during inference, potentially threatening user data or harming corporate interests. Therefore, it is necessary to restrict the output of such sensitive data by models.

[0022] In one related technique, for models that might output sensitive data during use, a gradient ascent scheme is used to continue training the model based on a sample dataset containing sensitive data, thereby reducing the probability of it outputting sensitive data. This scheme interferes with the model's memory of sensitive data by using loss as the training objective, thus controlling the model's data forgetting process. Specifically, for a sample set containing sensitive data, the model's loss on these samples is maximized, erasing the relevant memory of these samples from the model's parameters. Simply put, it allows the model to repeatedly learn incorrectly until it forgets the sensitive data.

[0023] This gradient ascent scheme comprises three main processes: gradient loss backward update, forgetting stability adjustment, and multi-round forgetting fine-tuning and early stopping. First, training samples are input into the model for prediction. Based on the prediction results, the training loss is calculated using regular forward propagation. Then, the parameters are updated along the gradient of the loss function, i.e., a small step of gradient ascent is performed on the model parameters. Simultaneously, the model's performance on a retained dataset containing only non-sensitive data is monitored. A portion of the retained data is input into the model to calculate the training loss, which is then used as a constraint for model optimization, keeping the model's output variation within a controllable range. This joint optimization is repeated until the model's loss on sensitive data rises to a preset threshold or the confidence level drops to its minimum. In simple terms, gradient loss backward update penalizes the model when it correctly outputs on sensitive data, weakening the model's memory connection to sensitive data and causing it to output other data when it should output sensitive data. Forgetting stability adjustment allows the model to review other knowledge to prevent it from forgetting sensitive data along with it. Multi-round forgetting fine-tuning and early stopping repeatedly tests the model until it forgets sensitive data, stopping immediately to prevent over-adjustment. The gradient ascent method requires continued model training to make the model forget sensitive data, which may lead to catastrophic degradation of the model. As the model forgets sensitive data, it also damages other capabilities of the model, making the retrained model unusable.

[0024] In another related technique, a negative preference optimization scheme is used for models that may output sensitive data during use. This scheme limits the generation of sensitive data by guiding the model to degenerate accordingly on sensitive data. In this scheme, based on a sample dataset containing sensitive data, the sensitive data is used as negative samples to continue training the model, and the difference between the output of the trained model and the output of the original model is compared to keep the output difference within a controllable range.

[0025] This negative preference optimization scheme comprises two main processes: estimating the inverse value function and updating the model based on KL constraints. First, training samples are input into the model for prediction. Based on the prediction results, the generation probability of these samples is calculated using conventional forward propagation, serving as the value function. Within the reinforcement learning framework, the model is fine-tuned by minimizing the generation probability of negatively preferred samples while constraining the change between the model's generation probability and the original model's generation probability to not exceed a certain threshold. In simpler terms, estimating the inverse value function assesses the model's tendency to output sensitive information, while updating the model based on KL constraints trains the model to select better answers outside of sensitive data for output. This scheme requires not only training the model to avoid outputting sensitive data but also using the original model as a comparison model during training, resulting in a relatively high training cost.

[0026] This specification provides a method, dialogue method, device, and medium for controlling the forgetting of target model data. Based on the target model, it processes target samples containing target data to be forgotten and constructed hedging samples. Based on the processing results, it updates the original metric information of the target model's parameter space, obtaining the target metric information that the parameter space should possess after the target model forgets the target data, and determines the parameter update direction of the target model. Based on the target metric information and the parameter update direction, it determines the parameter change information of the target model to update the target model's parameters. This allows for the determination of how to adjust the model parameters without further training the target model, efficiently controlling the forgetting of target data, reducing the probability of data leakage caused by the model outputting target data during use, and improving data security. Furthermore, this method can estimate the impact of target data on the target model's behavior, directly adjusting parameters without further model training, reducing training costs, preventing adverse effects of model training on other capabilities of the target model, and maintaining model performance.

[0027] This specification also provides a dialogue method that inputs a question text into a target model whose parameters have been updated using the aforementioned method, and obtains the response text output by the target model. This response text does not contain the target data that is restricted from being output, which can improve data security in dialogue scenarios.

[0028] The method for controlling forgotten data in the target model provided in this specification does not require the introduction of an additional model and is not affected by the structure of the target model, and can be applied to various models. For example, the target model can be a large language model of various structures. In some embodiments, the target model can also be a smaller-scale model, and the target model can be a unimodal model or a multimodal model, which is not limited here.

[0029] The method for controlling target model forgetting data provided in this specification can be used in services such as chatbots and intelligent completion assistants based on large language models. This solution adjusts the parameters of the large language model upon which the service is based to reduce the probability of the chatbot outputting target data (such as sensitive data) during conversations with users, or to reduce the probability of the intelligent completion assistant outputting target data while completing user input. This solution can also be applied to other scenarios where it is necessary to limit the model's output of target data, such as copywriting and video generation scenarios.

[0030] The following sections of this manual will use a large language model as the target model, and its application in a dialogue scenario will be used as an example. Due to the massive scale of its parameters, large language models incur high training and deployment costs. Therefore, training and deployment of large language models are primarily concentrated on cloud platforms, using a Machine Learning as a Service (MLaaS) approach. The training and deployment of the model are handled centrally by the service provider. Individual and enterprise users can provide sample data for model training to tailor the model to their specific needs. After the service provider completes the model training, users can access the large language model deployed on the service provider via an Application Programming Interface (API). The solution described in this manual can be applied to the service provider's service devices. In some embodiments, if the user's device is powerful enough to support model training and deployment, the target model can also be trained and deployed on the user's device, and the solution described in this manual can also be applied to the user's device.

[0031] Figure 1 A schematic diagram illustrating an application scenario of a method for controlling forgotten data in a target model according to embodiments of this specification is shown. Figure 1 As shown, the application scenario 100 may include a target user 110, a control device 120, a service device 130, and a network 140. The control device 120 and the service device 130 are connected through the network 140. The control device 120 is a device belonging to the user, and the service device 130 may be a cloud device belonging to the service provider.

[0032] Figure 1 Taking target user 110 as an individual user and control device 120 as a terminal as an example. Control device 120 may include mobile devices, tablets, laptops, built-in devices in motor vehicles, or similar content, or any combination thereof. In some embodiments, control device 120 may also be a server on the user's side.

[0033] Service device 130 may be a server providing various services, such as a backend server for model training and deployment. The methods provided in this specification can be executed on service device 130. Service device 130 may store data or instructions for performing the methods for controlling the forgotten data of the target model described in this specification, and may execute or be used to execute said data or instructions. In some embodiments, service device 130 may include hardware devices with data processing capabilities and necessary programs for driving the hardware devices. Service device 130 may be communicatively connected to multiple control devices 120 and receive data sent by the control devices 120.

[0034] Network 140 is a medium used to provide a communication connection between control device 120 and service device 130. Network 140 can facilitate the exchange of information or data. Figure 1 As shown, control device 120 and service device 130 can be connected to network 140 and transmit information or data to each other through network 140. In some embodiments, network 140 can be any type of wired or wireless network, or a combination thereof. For example, network 140 may include cable networks, wired networks, fiber optic networks, telecommunications networks, intranets, the Internet, local area networks (LANs), wide area networks (WANs), wireless local area networks (WLANs), metropolitan area networks (MANs), public switched telephone networks (PSTNs), Bluetooth networks, ZigBee networks, near field communication (NFC) networks, or similar networks. In some embodiments, network 140 may include one or more network access points. For example, network 140 may include wired or wireless network access points, such as base stations or Internet switching points, through which one or more components of control device 120 and service device 130 can connect to network 140 to exchange data or information.

[0035] It should be understood that Figure 1 The number of control devices 120, service devices 130, and network 140 shown is merely illustrative. Any number of control devices 120, service devices 130, and network 140 can be used depending on implementation needs.

[0036] In this specification, target user 110 can trigger control device 120 to upload a sample set for model training to service device 130. Service device 130 trains the model based on the sample set to obtain the target model. At any time when the user decides to keep certain data in the sample set confidential, target user 110 can trigger control device 120 again to send a forgetting request for the target data to service device 130. In response to the forgetting request, service device 130 executes the method described in this specification to adjust the parameters of the original target model, causing the target model to forget the target data, thereby restricting the target model from outputting the target data.

[0037] In some embodiments, model training, deployment, and adjustment can be performed directly on the user's device, in which case the service device 130 can be integrated into the control device 120. It should be noted that all user-related data obtained in this specification has been authorized by the user and does not involve user privacy.

[0038] Figure 2 A hardware structure diagram of an electronic device provided according to an embodiment of this specification is shown. Figure 2 Electronic device 200 in the middle can be used as Figure 1The service device 130 in the specification executes the method and dialogue method for controlling the forgotten data of the target model.

[0039] like Figure 2 As shown, the electronic device 200 may include at least one storage medium 230 and at least one processor 220. In some embodiments, the electronic device 200 may also include a communication port 250 and an internal communication bus 210. The electronic device 200 may also include an input / output component (I / O component) 260.

[0040] The internal communication bus 210 can connect to different system components. For example, the internal communication bus 210 can connect to storage medium 230, processor 220, communication port 250, and I / O component 260, etc.

[0041] I / O component 260 supports input / output between electronic device 200 and other components.

[0042] Communication port 250 is used for data communication between electronic device 200 and the outside world. For example, communication port 250 can be used for data communication between electronic device 200 and network 140. Communication port 250 can be a wired communication port or a wireless communication port.

[0043] Storage medium 230 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 232, a read-only storage medium (ROM) 234, or a random access storage medium (RAM) 236. Storage medium 230 also includes at least one instruction set stored in the data storage device. The instruction set may include computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc.

[0044] At least one processor 220 may be communicatively connected to at least one storage medium 230. When the electronic device 200 is running, at least one processor 220 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the method or dialogue method for controlling the forgetting data of the target model provided in this specification. The processor 220 may execute the steps included in the method for controlling the forgetting data of the target model, or the steps included in the dialogue method. The processor 220 may be in the form of one or more processors. In some embodiments, the processor 220 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof.

[0045] For illustrative purposes only, the electronic device 200 in the accompanying drawings shows only one processor 220. However, it should be noted that the electronic device 200 in this specification may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor or by multiple processors in combination. For example, if the processor 220 of the electronic device 200 described in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 220 (e.g., the first processor executes step A, the second processor executes step B, or the first and second processors jointly execute steps A and B).

[0046] Figure 3 A flowchart illustrating a method for controlling forgotten data of a target model according to an embodiment of this specification is shown. As previously described, electronic device 200 can serve as... Figure 1 The service device 130 executes the method P300 for controlling the forgotten data of the target model as described in this specification. Specifically, the processor 220 in the electronic device 200 can read the instruction set stored in its local storage medium, and then execute the method P300 for controlling the forgotten data of the target model as described in this specification according to the instructions in the instruction set. For ease of description, the method for controlling the forgotten data of the target model will be referred to as the model adjustment method below, and the application of this model adjustment method to the service device will be used as an example for explanation. Figure 3 As shown, method P300 may include the following steps 310 to 330.

[0047] Step 310: In response to the request to forget the target data, determine the target sample subset containing the target data from the original sample set, and generate a hedging sample subset without the target data based on the target sample subset.

[0048] The target data can be any data that the user wants to keep confidential, or any data that the user wants to limit the model's output. For example, the target data may include sensitive data involving user privacy or trade secrets that is not suitable for casual disclosure. In this specification, the service device is used to train, deploy, and adjust the target model, and the request to forget the target data can come from the user's control device. The control device can send a request to forget the target data to the service device, so that the service device can adjust the parameters of the target model to make the target model forget the target data. After the target model forgets the target data, it will not output the target data.

[0049] The target model is a model trained based on an original sample set containing the target data. If the target model is used for question answering, the original sample set may include multiple input-output text pairs, each text pair being considered a sample. This original sample set can be uploaded to the service device by the user via a control device. Accordingly, before receiving a request to forget the target data, the model adjustment method provided in this specification further includes: receiving the original sample set from the control device and using the original sample set to train the model to obtain the target model. For example, the user first uploads the original sample set to the service device via the control device and requests the service device to train the target model based on the original sample set. After the model training is completed, the user may have a need to keep the target data confidential. At this time, the control device can send a request to forget the target data to the service device. The user can send multiple requests to forget different target data according to actual needs. The service device can adjust the original target model or the previously adjusted target model according to user needs to obtain an updated target model.

[0050] In some approaches, the request to forget the target data can be uploaded to the service device along with the original sample set from the control device. The service device first trains the model based on the original sample set, and then instructs the target model to forget the target data, rather than directly removing the target data before training. This preserves data diversity, allowing the target model to fully utilize all data for learning and acquire general knowledge from the target data, which is beneficial for knowledge integrity. It also allows the target model to distinguish between conceptual discussion and specific disclosure, retaining conceptual outputs about the data without disclosing the target data, resulting in better performance of the target model.

[0051] The forgetting request for target data sent by the control device may carry information items indicating the target data or sample indexes. The service device can then query the original dataset for target samples containing the target data based on these information items or sample indexes to obtain a subset of target samples. Accordingly, determining the subset of target samples containing the target data from the original sample set in step 310 includes: parsing the information items indicating the target data from the forgetting request, querying the original sample set for samples containing the information items, and obtaining the subset of target samples; or parsing the sample index indicating the target data from the forgetting request, querying the original sample set for samples corresponding to the sample index, and obtaining the subset of target samples.

[0052] Users can control the device to specify the information items that need to be restricted in the output. Correspondingly, the forgetting request for the target data carries this information item. This information item can refer to the type of target data, such as name, age, address, phone number, or email address. The service device can parse the received forgetting request to obtain its carried information item. Then, based on this information item, the service device performs content analysis on each sample in the original sample set to determine whether each sample contains data belonging to that information item. The data in the sample belonging to that information item is the specified target data. The service device uses the samples in the original sample set that contain data with that information item as the target samples, thus obtaining a subset of the target samples.

[0053] For example, each sample in the original sample set is an input-output text pair, including the input text and the corresponding standard output text. The target data can be at least a portion of the output text. The service device can perform content analysis on the output text of each sample in the original sample set to determine whether the output text contains the target data, and accordingly determine whether the sample is a target sample.

[0054] Users can also directly specify which samples need to be restricted in output via the control device, treating all of these samples as target data. Each sample in the original sample set is labeled with a corresponding sample index, such as consecutive numbers (e.g., sample 1, sample 2), which serve as the sample index. The user's forget request for target data sent via the control device can carry the sample index indicating the target data. The service device parses the received forget request to obtain the carried sample index. Then, based on this sample index, the service device directly queries the original sample set, using the sample corresponding to the queried sample index as the target sample, thus obtaining the target sample subset.

[0055] In some embodiments, the user can also specify the specific content of the target data so that the forgetting request for the target data carries that specific content. After parsing the forgetting request to obtain the specific content, the service device can perform string matching on each sample in the original sample set based on the specific content, and determine the sample containing the content as the target sample.

[0056] After determining the target sample subset containing the target data, the service device can also construct a hedging sample subset that does not contain the target data based on the target sample subset. For example, the service device can construct the hedging sample subset by replacing the target data in each target sample in the target sample subset. For example, in step 310, the service device can locate the first string corresponding to the target data in each target sample in the target sample subset and replace the first string with a non-sensitive second string; and generate the hedging sample subset based on the replacement results of each target sample in the target sample subset.

[0057] The first string corresponding to the target data refers to the string that needs to be adjusted to maintain the confidentiality of the target data. The target data may consist only of content that needs to be kept confidential (such as sensitive content), or it may consist of a mixture of content that needs to be kept confidential and content that does not need to be kept confidential (such as non-sensitive content). The first string may contain specific content that needs to be kept confidential in the target data. For example, the first string may include all the content of the target data, or it may include a portion of the content of the target data. In some embodiments, the service device may also directly use all the content of the target data as the first string without distinguishing between specific content.

[0058] In some embodiments, the service device may analyze the target data within the target sample after identifying the target sample, and locate the first string therein. In some embodiments, the service device may locate the first string corresponding to the target data during the search for the target sample, and identify the sample containing the first string as the target sample.

[0059] The content of the first string is prohibited from being output. The service device replaces the content of the first string with the second string, thereby removing the prohibited data from the target sample. Within the target sample subset, the first string may differ in different target samples, and the corresponding second string used for replacement may also differ. Based on the replacement results of each target sample, multiple hedging samples that do not contain the target data can be obtained, thus forming a hedging sample subset. For example, the service device can directly use each target sample after the first string replacement as a hedging sample. In some embodiments, the service device can perform further processing (such as adding other content or changing certain data formats) on the target samples after the first string replacement to obtain hedging samples.

[0060] If the target sample is determined based on a sample index and the target data is all the data in the target sample, the first string may include all the characters in the target sample. Correspondingly, the service device replaces all the characters in the target sample with a second string.

[0061] In some embodiments, the second string satisfies at least one of the following: a meaningless string, a name representing the information item corresponding to the first string, an empty string, the same data type as the first string, or a string representing a rejection meaning.

[0062] Exemplarily, the meaningless string may include strings such as "XXXX" or "****". The string representing a rejection meaning may include strings such as "This content cannot be output" or "I cannot answer this question", that is, replacing the first string with a rejection statement.

[0063] The service device can determine the name of the information item corresponding to the first string and use this name as the second string. The name of this information item can be analyzed and summarized by the service device itself or directly preset in advance. For example, if the first string is the entire content of the user's name, the information item corresponding to the first string can be the user's name, and its name can be "user_name". The service device can replace the user's name (such as "Xiaoming") in the target sample with "user_name".

[0064] The second string can be a string with the same data type as the first string, that is, rewriting the first string with allowed output of the same type of content. For example, if the first string is a person's name, the second string can be a character name in a movie or TV drama; if the first string is a phone number, the second string can be a false number "12345678".

[0065] Taking the target data including personal identification data, such as data corresponding to information items including user name, phone number, and email address, etc., as an example. For the user name, the entire content of the user name belongs to the first string corresponding to the target data and can be replaced with "user_name". For the phone number, the area code and the user number part of the phone number belong to the first string corresponding to the target data and can be replaced with "XXXX"; for example, changing the original phone number "12312345678" to "123XXXXXXXX". For the email address, The user name part before the symbol belongs to the first string corresponding to the target data and can be replaced with "user_name"; for example, changing the original email address "aaa 123.com" to "user_name "123.com". If the entire content of the target sample belongs to the target data, the service device can replace the target sample with an empty string of data to construct a hedging sample.

[0066] Suppose the original sample set is denoted as D, and each sample in the original sample set includes an input text x and a corresponding output text y. The service device determines a target sample subset from the original sample set D based on the target data specified in the forgetting request. And construct a hedging sample subset that does not contain the target data. .in, This represents the input text for the target model. This indicates the output text that contains the target data. This indicates the output text that does not contain the target data.

[0067] Step 320: Input the target sample subset and the hedging sample subset into the target model for processing to obtain the processing result, and update the original metric information of the parameter space of the target model based on the processing result to obtain the target metric information. The original metric information represents the sensitivity information of the target model in different directions in the parameter space after training based on the original sample set, and the target metric information represents the sensitivity information of the target model in different directions in the parameter space after forgetting the target sample subset and memorizing the hedging sample subset.

[0068] The metric information in the model's parameter space represents the sensitivity information in different directions within that parameter space. This metric information is a mathematical entity; it defines a Riemannian metric on the model's parameter space to characterize the curvature characteristics in different directions. Essentially, this metric information reflects the local geometry of the model's loss function at parameter points. High curvature directions correspond to regions in the parameter space where the model's predicted output is highly sensitive to parameter changes, typically encoding key knowledge; low curvature directions correspond to regions with lower sensitivity and greater tolerance for parameter variations.

[0069] In some embodiments, the metric information of the parameter space of the target model includes the inverse Fisher matrix or the inverse Hessian matrix. The Fisher matrix characterizes the degree of influence of changes in model parameters on the model output; a larger value of the Fisher matrix in a certain parameter direction indicates that a small change in the parameters in that direction will significantly alter the model output. The inverse Fisher matrix characterizes the range by which model parameters can safely change at a given sensitivity; a larger value of the inverse Fisher matrix in a certain parameter direction indicates that the parameters in that direction can be adjusted significantly without drastically affecting the model output. The inverse Fisher matrix can suppress the magnitude of parameter updates in highly sensitive directions and amplify the magnitude of parameter updates in less sensitive directions. The meanings of the Hessian matrix and the inverse Hessian matrix can be found in the preceding introduction to the Fisher matrix and the inverse Fisher matrix, and will not be repeated here.

[0070] In this specification, the service device can obtain the raw metric information of the parameter space of the target model. Based on the target sample subset, the hedging sample subset, and this raw metric information, it determines how to adjust the target model to make it forget the target data. This raw metric information is the metric information of the parameter space of the target model after training on the raw sample set. For example, the service device updates this raw metric information based on the processing results of the target sample subset and the hedging sample subset using the target model to obtain target metric information, and subsequently updates the model parameters based on this target metric information. For instance, the raw metric information of the parameter space of the target model includes the inverse Fisher matrix or the inverse Hessian matrix.

[0071] The original metric information before updating fully records the geometric structure of the parameter space of the target model shaped by the original sample set, including the target data. Updating this original metric information based on the target sample subset containing the target data and the hedging sample subset not containing the target data transforms it into a representation of the target geometric structure that the model's parameter space should possess after removing the target sample subset from the original sample set and adding the hedging sample subset. Based on this target geometric structure, the parameters of the target model can be adjusted under constraints conforming to the target geometric structure. This allows for a more accurate forgetting of the target data during parameter adjustments, while reducing the simultaneous forgetting of other knowledge, thus preserving the overall model performance to a greater extent.

[0072] The following describes how to determine the original metric information of the parameter space of the target model, using the example of the original metric information including the inverse Fisher matrix. In some implementations, the service device can determine the original metric information based on the gradients corresponding to at least a portion of the samples in the original sample set. The gradient corresponding to a sample refers to the gradient of the loss function of the sample with respect to the model parameters, reflecting the degree of influence of each model parameter on the loss function. Accordingly, the model adjustment method provided in this specification may also include: inputting at least a portion of the samples in the original sample set into the target model for processing to obtain the gradients corresponding to each of the at least a portion of the samples; and generating the original metric information based on the gradients corresponding to each of the at least a portion of the samples. The methods of inputting each sample into the target model for processing to obtain the gradients corresponding to the samples involved in this specification can all be the same, and the relevant descriptions can be referred to each other.

[0073] For example, for each sample in the original sample set that is to be input into the target model, the service device inputs the sample into the target model and obtains the prediction result of the target model for that sample. The service device compares the prediction result with the label of the sample to determine the prediction loss of the target model for that sample. The service device backpropagates the prediction loss in the target model to analyze how the prediction loss is caused by the calculation of each layer in the target model, and obtains the gradient corresponding to the sample, which represents the rate of change of the prediction loss relative to each parameter in the model.

[0074] Assuming the sample consists of a text pair consisting of input and output text, inputting the sample into the target model is equivalent to inputting the input text from the sample into the target model. The target model then performs inference based on this input text and outputs a predicted output text. The output text from the sample serves as the label corresponding to the sample. The service device compares the target model's predicted output text with the output text from the sample to determine the prediction loss for that sample. For example, for a sample... Based on its target model The prediction results obtained after processing can determine the target model. For the prediction loss of this sample By performing backpropagation, the gradient corresponding to the sample can be calculated. .in, These represent the parameters of the target model. and Let the input text and output text of the i-th sample be represented by , and the prediction loss be . Different loss functions, including cross-entropy loss, can be used for measurement.

[0075] After determining the gradients corresponding to at least a portion of the samples in the original sample set, the service device can determine the original metric information based on these gradients using a predefined calculation method. In some embodiments, the service device can iteratively update the metric information based on the gradients corresponding to each sample, on a sample-by-sample basis, to obtain the original metric information. For example, the step of generating the original metric information based on the gradients corresponding to at least a portion of the samples in the original sample set may include: generating initial metric information of the same scale as the parameter scale of the target model; traversing the at least a portion of the samples, and updating the current metric information based on the gradient corresponding to the i-th sample, where when i=1, the current metric information is the initial metric information, and when i>1, the current metric information is the metric information obtained after traversing the first i-1 samples; and using the metric information obtained after the traversal as the original metric information.

[0076] The inverse Fisher matrix is ​​a measure of the parameter space of the target model. Taking the generation of original metric information using m samples from the original sample set as an example, the service device can generate the inverse Fisher matrix. Initialize it as an identity matrix of the same size as the parameters of the target model. This serves as the initial metric information. For each of the selected m samples, the inverse Fisher matrix is ​​updated sample by sample based on the corresponding gradient. The inverse Fisher matrix obtained after updating the m samples will be updated using these m samples. This serves as the initial metric information. During the update process, the service device can traverse m samples and update the current metric information based on the current sample. For the first sample, i.e., when i=1, no update has been performed before, so the current metric information is the initial metric information, i.e., the identity matrix. After this, i.e. when i>1, the initial metric information has been updated. Therefore, the current metric information is the metric information obtained by iteratively updating the initial metric information based on the gradient of the first i-1 samples.

[0077] For example, the service device can update the initialization metric information using a first update formula. For instance, the first update formula is... .in, This represents the original measurement information obtained after the update; This indicates the initialization of metric information; This represents the update coefficient set for the initial metric information, used to control update stability; it can be a preset value. This represents the current metric information obtained by updating the initial metric information using the gradients corresponding to the first i-1 samples. When i=1, the current metric information... When i=2, the current measurement information And so on, until we get .

[0078] In some embodiments, the initialization metric information is in matrix form. Updating the initialization metric information can involve updating all elements of the matrix. In some embodiments, updating the initialization metric information includes updating the diagonal elements of the matrix without updating elements in other positions. In this approach, the initialization metric information can also include a unit diagonal matrix. The diagonal elements of the metric information matrix directly reflect the degree of independent influence of the model parameters on the model output, containing most of the key information required for optimization. In the above embodiments, updating only the diagonal elements of the metric information matrix has a relatively small impact on the meaning represented by the metric information, and can reduce computational costs, improve computational efficiency, and save storage space.

[0079] In some embodiments, at least a portion of the samples in the aforementioned original sample set may include all samples in the original sample set, or only a portion thereof. The service device only needs to process a smaller number of samples and determine the original metric information based on the gradients corresponding to these smaller samples, which can reduce computational costs and improve computational efficiency. For example, the service device can sample a subset from the original sample set D. , Then, the sampling subset is determined based on the target model. Gradients of each sample Based on this, the inverse Fisher matrix is ​​calculated to obtain the original metric information.

[0080] For example, the service device can sample from the original sample set according to a preset sampling rule to obtain a sampled subset, which is used to determine the original metric information. By obtaining a sampled subset through sampling, the service device can reduce the amount of samples processed while ensuring that the sampled samples are more representative of the original sample set. This allows the original metric information determined based on the sampled samples to more accurately characterize the parameters of the target model. The preset sampling rule may include at least one of the following: random sampling or proportional sampling for different types of samples.

[0081] The size of the sampled subset can be positively correlated with the size of the original sample set, such as being no less than 10%, 12%, or other proportions of the original sample set. The service device can determine the number of samples to be sampled based on the number of samples in the original sample set. This number of samples needs to be greater than a certain value so that the sampled data can represent the original sample set in terms of data volume. The service device performs sampling based on this number of samples and a preset sampling rule. The original dataset may contain different types of samples, and different types of samples have different impacts on the target model. The service device can sample proportionally for different types of samples, such as classifying and sampling samples according to their text labels. This can make the distribution of sample types after sampling as consistent as possible with the original sample set, achieving a greater degree of accuracy in representing the original dataset.

[0082] In some embodiments, the original metric information can be pre-computed before the service device receives a request to forget the target data, such as after the target model has been trained. The determined original metric information can then be stored so that it can be directly retrieved and used when needed in subsequent task executions, instead of being calculated only when required by the task, thus reducing the impact on task execution efficiency.

[0083] Accordingly, after generating the original measurement information, the model adjustment method provided in this specification may further include: storing the original measurement information in a preset storage space; and upon receiving a request to forget the target data, the method may further include: retrieving the original measurement information from the preset storage space. In this way, the service device pre-calculates and stores the original measurement information, and upon receiving a request to forget the target data, can directly retrieve the original measurement information for subsequent parameter updates, thereby reducing response time and improving the efficiency of controlling the target model to forget the target data.

[0084] In some embodiments, after obtaining the target sample subset and the hedging sample subset, the service device can process the target sample subset and the hedging sample subset based on the target model, and update the original metric information based on the processing results. For example, the target sample subset and the hedging sample subset can be input into the target model for processing to obtain the processing results. Since updating the original metric information requires based on the gradients corresponding to the samples, the service device needs to process the target sample subset and the hedging sample subset to determine the corresponding gradients. Accordingly, the processing results include: the first gradient corresponding to each target sample in the target sample subset, and the second gradient corresponding to each hedging sample in the hedging sample subset. The first gradient corresponding to the target sample refers to the gradient of the loss function of the target sample relative to the parameters of the target model, and the second gradient corresponding to the hedging sample refers to the gradient of the loss function of the hedging sample relative to the parameters of the target model. The gradient represents the rate of change of the prediction loss relative to each parameter in the target model. The gradient can indicate the direction of adjustment of the model parameters, whether the model parameters should be increased or decreased, and which parameters need more adjustment and which parameters need less adjustment.

[0085] In some embodiments, the step of inputting a subset of target samples into a target model to obtain a processing result may include: for each target sample in the subset of target samples, inputting the target sample into the target model to obtain a first prediction result; determining a first prediction loss of the target model for the target sample based on the first prediction result; and backpropagating the first prediction loss in the target model to obtain a first gradient corresponding to the target sample. This first gradient characterizes the rate of change of the first prediction loss relative to each parameter in the target model. The method for determining the first gradient corresponding to the target sample may be mutually referenced with the method for determining the gradients corresponding to at least a portion of the samples in the aforementioned original sample set.

[0086] For example, the service device will select the previously determined target sample subset. Each target sample is input into the target model. For example, the input text in the input-output text pairs included in the target sample Input the target model and obtain the model's first prediction result (predicted output text) for that target sample. The service device then compares this first prediction result with the label of the target sample (the output text in the input-output text pair). By comparing the results, the first prediction loss of the target model for the target sample is determined. The service device will apply this first prediction loss to the target model. We perform backpropagation to analyze how the first prediction loss is caused by the calculation of each layer in the target model, and obtain the first gradient corresponding to the target sample. .

[0087] In some embodiments, the step of inputting a subset of hedging samples into the target model to obtain a processing result may include: for each hedging sample in the subset of hedging samples, inputting the hedging sample into the target model to obtain a second prediction result; determining a second prediction loss of the target model for the hedging sample based on the second prediction result; and backpropagating the second prediction loss in the target model to obtain a second gradient corresponding to the hedging sample. This second gradient characterizes the rate of change of the second prediction loss relative to each parameter in the target model. The method for determining the second gradient corresponding to the hedging sample may refer to the method for determining the gradient corresponding to at least a portion of the samples in the original sample set, as well as the method for determining the first gradient.

[0088] For example, the service device will use the previously determined hedging sample subset. Each hedging sample in the model is input into the target model. For example, the input text in the input-output text pairs included in the hedging sample. Input the target model to obtain the model's second prediction result (predicted output text) for the hedged sample. The service device then compares this second prediction result with the label of the hedged sample (the output text in the input-output text pair). By comparing the results, the second predicted loss of the target model for this hedging sample is determined. The service device will apply this second prediction loss to the target model. We perform backpropagation to analyze how the second prediction loss is caused by the calculation of each layer in the target model, and obtain the second gradient corresponding to the hedging sample. .

[0089] Both the target sample subset and the aforementioned sampling subset include samples from the original sample set, and there may be overlap between the samples in the target sample subset and the sampling subset. In some embodiments, the service device can store the gradients corresponding to each sample determined for the sampling subset. After determining the target sample subset, for target samples that also exist in the sampling subset, the corresponding stored gradients are directly looked up. To avoid the cumbersome process of gradient storage, in the above embodiments, the sampling subset and the target sample subset are input into the target model respectively, and the gradients are determined based on the prediction results.

[0090] In some embodiments, the service device can update the original metric information based on the first gradient corresponding to each target sample and the second gradient corresponding to the hedging sample to obtain target metric information. In step 320, the service device can update the original metric information of the parameter space of the target model based on the first gradient corresponding to each target sample in the target sample subset to obtain intermediate metric information, wherein the intermediate metric information represents the sensitivity information of the target model in different directions in the parameter space after forgetting the target sample subset; and update the intermediate metric information based on the second gradient corresponding to each hedging sample in the hedging sample subset to obtain target metric information.

[0091] The service device updates the original metric information to obtain target metric information. This is to determine the sensitivity information in different directions of the parameter space—that is, the geometric structure that the parameter space should possess—so that the target model can forget the target data without affecting other capabilities of the model. The target metric information represents this sensitivity information and can be used to guide the parameter updates required by the target model to forget the target data. The difference between the target sample subset and the hedging sample subset determined by the service device is only whether or not the target data is included. By making the target model forget the target sample subset and remember the hedging sample subset, the target model can forget the target data but try not to forget other data, reducing the risk of model performance degradation.

[0092] In the above embodiments, the service device first updates the original metric information based on the first gradient corresponding to the target sample, adjusting the metric information in the direction corresponding to the target sample to obtain intermediate metric information. This intermediate metric information reflects the sensitivity information in different directions of the parameter space of the target model after forgetting a subset of target samples, i.e., the geometric structure that its parameter space should possess. Based on this intermediate metric information, the sensitivity of each parameter in the target model to the target sample can be determined, and the impact of forgetting the target sample on each model parameter can be determined, including the impact of forgetting the target data on each model parameter. Then, the service device updates the intermediate metric information based on the second gradient corresponding to the hedging sample to obtain the target metric information. In this way, based on the established impact of sensitive data on the target model, adding the impact of remembering the hedging sample can reduce the labeling of the impact of hedging sample interference on the target data, so that the impact of the target data can be fully identified; and subsequently, using the target metric information to guide the adjustment of model parameters can achieve a more thorough forgetting of the target data and reduce the side effects on the target model.

[0093] In some embodiments, the service device may, based on the original metric information, iteratively update the metric information sample by sample based on the first gradient corresponding to each target sample to obtain intermediate metric information. Based on the intermediate metric information, iteratively update the metric information sample by sample based on the second gradient corresponding to each hedging sample to obtain target metric information.

[0094] For example, the process of updating the original metric information to obtain intermediate metric information may include: traversing a subset of target samples, updating the current metric information based on the first gradient corresponding to the i-th target sample for the i-th traversed target sample, wherein when i=1, the current metric information is the original metric information, and when i>1, the current metric information is the metric information obtained after traversing the first i-1 target samples; and using the metric information obtained after the traversal as intermediate metric information.

[0095] For example, the process of updating intermediate metric information to obtain target metric information may include: traversing a subset of hedging samples, updating the current metric information based on the second gradient corresponding to the i-th hedging sample for the i-th hedging sample, wherein when i=1, the current metric information is intermediate metric information, and when i>1, the current metric information is the metric information obtained after traversing the first i-1 hedging samples; and using the metric information obtained after the traversal as the target metric information.

[0096] The process of updating the original and intermediate measurement information described above can be referred to the previous introduction on updating the initial measurement information, which will not be elaborated on here.

[0097] The inverse Fisher matrix is ​​determined above using the original metric information of the parameter space of the target model. For example, the service equipment targets a subset of the target samples. For each target sample, update the inverse Fisher matrix based on the corresponding first gradient. , will utilize The inverse Fisher matrix obtained after updating each target sample is used as intermediate metric information. For example, the service device can update the original metric information using the second update formula. The second update formula is as follows: .in, This represents the current metric information obtained by updating the original metric information using the first gradient corresponding to the first i-1 target samples. When i=1, the current metric information... When i=2, the current measurement information And so on, until we get .

[0098] Service equipment is designed for hedging sample subsets For each hedging sample, the inverse Fisher matrix is ​​updated based on the corresponding second gradient. , will utilize The inverse Fisher matrix obtained after updating each hedging sample is used as the target metric. For example, the service device can use the third update formula to update the intermediate metric. The third update formula is as follows: .in, This represents the current metric information obtained by updating the original metric information using the second gradient corresponding to the first i-1 hedging samples. When i=1, the current metric information... When i=2, the current metric information is: And so on, until we get .

[0099] Step 330: Based on the processing results, determine the parameter update direction required by the target model in order to forget the target sample subset and remember the hedging sample subset.

[0100] The service device processes the target sample subset and the hedging sample subset using the aforementioned method to obtain the processing result. This result may include the first gradient corresponding to the target sample and the second gradient corresponding to the hedging sample. Based on these first and second gradients, the parameter update direction required by the target model to forget the target sample subset and remember the hedging sample subset can be determined. This parameter update direction can be characterized by the overall gradient difference between the target sample subset and the hedging sample subset.

[0101] In some embodiments, each hedging sample in the hedging sample subset constructed by the service device is obtained based on a target sample, and the hedging sample and the target sample can form a sample pair. Multiple sample pairs are formed by the target sample subset and the hedging sample subset, and each sample pair includes a target sample and its corresponding hedging sample. The service device can determine the overall gradient difference based on these multiple sample pairs. Step 330 may include: for each of the multiple sample pairs, determining the gradient difference between the first gradient corresponding to the target sample and the second gradient corresponding to the hedging sample in the sample pair; and summing the gradient differences corresponding to the multiple sample pairs to obtain the overall gradient difference, wherein the overall gradient difference is used to characterize the parameter update direction required by the target model to forget the target sample subset and remember the hedging sample subset. For example, the service device is based on the aforementioned target sample subset... The first gradient corresponding to each target sample and the aforementioned hedging sample subset The second gradient corresponding to each hedging sample The overall gradient difference can be determined. .

[0102] In some embodiments, the service device may also determine the sum of the first gradients corresponding to each target sample and the sum of the second gradients corresponding to each hedging sample, and then subtract the sum of the second gradients from the sum of the first gradients to obtain the overall gradient difference.

[0103] Step 340: Based on the target metric information and parameter update direction, determine the parameter change amount of the target model, and update the parameters of the target model based on the parameter change amount.

[0104] The target metric information and parameter update direction determined by the service equipment can be used to guide the target model in updating its parameters. This allows the target model to forget the target sample subset while remembering the hedging sample subset, achieving forgetting only the target data. The target metric information can be used to adjust the target model and assign weights to each model parameter. Based on the target metric information, the parameter update direction (overall gradient difference) required to forget the target data can be reweighted and scaled according to the importance of the model's global knowledge, thereby obtaining a better parameter change and minimizing the interference of model parameter updates on the overall model.

[0105] In some embodiments, the service device can multiply the target metric information by the overall gradient difference to obtain the parameter change of the target model. For example, the service device uses the aforementioned obtained target metric information... and overall gradient difference The change in parameters can be obtained. .

[0106] After obtaining the parameter changes of the target model, the service device can adjust the target model accordingly and update its parameters. In some embodiments, in step 340, the service device can subtract the parameter changes from the current parameters of the target model to obtain the updated parameters. This parameter update can address both the impact of subtracting the target sample and the impact of adding hedging samples, comprehensively achieving the effect of the target model forgetting the target data. For example, the service device adjusts the parameters based on the aforementioned obtained parameter changes. The service device can update the parameters of the target model to obtain the updated parameters of the target model. This allows the target model to forget the target data. After updating the parameters of the target model, the service device can redeploy the updated target model.

[0107] In some embodiments, after updating the parameters of the target model, the model adjustment method provided in this specification may further include: deleting a subset of the target samples from the original sample set. This ensures that target data that the user needs to keep confidential is not stored outside the user's premises, reducing the risk of data leakage and further improving data security.

[0108] In some embodiments, the service device may also add a subset of hedged samples to the original sample set to obtain an updated sample set, so that the hedged sample subset can be used for model training or to perform other tasks later.

[0109] Figure 4 A general schematic diagram of a model update process provided according to an embodiment of this specification is shown. Figure 4 As shown, the user (corresponding to the aforementioned control device) can transmit a raw sample set to the service provider (corresponding to the aforementioned service device) for model training. This raw sample set includes target data, such as personal information, addresses, email addresses, or phone numbers. The service provider uses this raw sample set to train the model and obtain the target model. After training, the service provider can deploy the target model in the cloud and provide an API service to the user, allowing the user to access the target model through the API service.

[0110] Continue to refer to Figure 4 After training the target model, the service provider can also determine the original metric information (such as the inverse Fisher matrix) for the model. For example, the service provider can sample the original sample set to obtain a sampled subset, and input this sampled subset into the target model for processing to determine the gradient of the loss function with respect to the model parameters for each sample in the sampled subset. Based on this gradient, the service provider determines the original inverse Fisher matrix, which is used to characterize the sensitivity of the target model in different directions in the parameter space.

[0111] Continue to refer to Figure 4When a user has a need to keep certain target data in the original sample set confidential, the user can send a "forget target data" request to the service provider, requesting the service provider to adjust the target model and restrict its output of target data. Upon receiving the "forget target data" request, the service provider can close the API service originally open to the target model to prevent the output of target data.

[0112] Continue to refer to Figure 4 In response to the forgetting request, the service provider can construct a hedging sample subset. For example, the service provider determines a target sample subset containing the target data from the original sample set, and rewrites or replaces the target data in the target sample subset to construct the hedging sample subset. Then, the service provider inputs the target sample subset and the hedging sample subset into the target model for processing to determine the gradients corresponding to the target sample subset and the hedging sample subset, respectively. Based on these gradients, the original inverse Fisher matrix is ​​updated to obtain target metric information, which reflects the constraints that the adjustment of each model parameter should follow to make the target model forget the target data.

[0113] Continue to refer to Figure 4 The service provider also determines the overall gradient difference based on the gradients corresponding to the target sample subset and the hedging sample subset, respectively, to guide the parameter update direction of the target model. Based on this target metric information and the parameter update direction, the service provider jointly determines an optimal parameter change amount and performs the actual parameter update on the target model based on this parameter change amount, resulting in an updated target model that has forgotten the target data. The service provider can redeploy the updated target model and reopen the API service for this target model, allowing users to call the updated target model through the API service.

[0114] In the above embodiments, the service device estimates the overall impact of the target data on the target model by using the metric information of the target model's parameter space, the target samples containing the target data, and the constructed hedging samples, thus obtaining the parameter change amount that can eliminate the impact. Based on this parameter change amount, the parameters of the target model are updated to eliminate the impact of the target data on the target model and restrict the target model from outputting target data. In this way, the target model can forget the target data without continuing to train the model, reducing the risk of degradation of other capabilities of the target model. This process does not require the introduction of additional irrelevant data, nor does it require the introduction of a reference model, which can reduce computational costs and improve the efficiency of controlling the target model to forget the target data. Furthermore, this process does not require access to the original training process of the target model, making the solution highly applicable and efficient. It is well-suited for the common machine learning-as-a-service training and deployment scenarios of current large language models and can be applied to various large language models to control their data forgetting.

[0115] In summary, the method for controlling target model data forgetting provided in this specification processes target samples containing target data to be forgotten and constructed hedging samples based on the target model. Based on the processing results, the original metric information of the target model's parameter space is updated to obtain the target metric information that the parameter space should possess after the target model forgets the target data, and the direction of parameter update for the target model is determined. Based on the target metric information and the parameter update direction, the parameter change information of the target model is determined, and the parameters of the target model are updated accordingly. This eliminates the need for further training of the target model to determine how the model parameters should be adjusted, efficiently controlling the forgetting of target data, reducing the probability of data leakage caused by the model outputting target data during use, and improving data security. Furthermore, this method eliminates the need for further model training, directly adjusting parameters, which reduces the risk of adverse effects on other capabilities of the target model due to model training, and maintains model performance.

[0116] This specification also provides a dialogue method. Figure 5 A flowchart of a dialogue method provided according to an embodiment of this specification is shown. As previously illustrated, the electronic device 200 can serve as... Figure 1 The service device 130 in the electronic device 200 executes the dialogue method P500 of this specification. Specifically, the processor 220 in the electronic device 200 can read the instruction set stored in its local storage medium, and then execute the dialogue method P500 of this specification according to the provisions of the instruction set. Figure 5 As shown, the method P500 may include the following steps 510 to 530.

[0117] Step 510: Obtain the question text.

[0118] Users can launch conversational software powered by the target model installed on their terminal, or access conversational web pages or mini-programs powered by the target model, to trigger the display of a conversational page on the terminal. Users can then input a question on this conversational page to receive a response. The terminal can utilize the API provided by the service device for the target model to call the target model deployed on the service device to perform the conversational task. For example, the terminal can transmit the prompt text to the service device, and the service device will receive the question text accordingly.

[0119] If method P500 can be executed after the aforementioned method P300, the service device can control the target model to forget the target data based on method P300, obtain the updated target model, and deploy and open API services. Then, it can obtain the query text transmitted by the terminal through the newly opened API.

[0120] Step 520: Input the question text into the target model, wherein the target model is trained based on the original sample set and is configured to have forgotten the target data in the original sample set. The forgetting process of the target model for the target data is achieved by the aforementioned method of controlling the forgetting of data by the target model.

[0121] The service device can input the obtained question text into the target model, so that the target model can process the question text and generate the corresponding answer text. This target model is a model that has forgotten the target data. For the process of forgetting the target data, please refer to the description on page 300 of the aforementioned method, which will not be repeated here.

[0122] Step 530: Obtain the response text output by the target model, wherein the response text does not contain the target data.

[0123] Because the target model is a model that has forgotten the target data, it does not construct the answer text based on the target data when generating the answer text corresponding to the question text, thus providing data security. After processing the question text, the target model can output the answer text, which the service device can then obtain. The service device can then transmit the answer text back to the terminal that uploaded the question text, so that the terminal can display the answer text to the user, completing the dialogue interaction between the user and the terminal. The risk of target data leakage during the dialogue is low.

[0124] The dialogue method provided in this specification uses a target model adjusted based on the aforementioned method for controlling the forgetting of target model data. This target model can forget target data, and the risk of other capabilities of the target model being adversely affected is relatively small. By conducting dialogue based on this target model, the dialogue scenario can have a high degree of data security.

[0125] This specification, in another aspect, provides a computer-readable non-transitory storage medium storing at least one set of instructions. When the at least one set of instructions is executed by a processor, it instructs the processor to implement the steps of the method P300 or the dialogue method P500 for controlling the forgetting data of a target model as described in this specification. In some possible embodiments, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on an electronic device 200, the program code causes the electronic device 200 to perform the steps of the method P300 or the dialogue method P500 for controlling the forgetting data of a target model as described in this specification. The program product for implementing the above methods may employ a portable compact disc read-only memory (CD-ROM) containing program code and may run on the electronic device 200. However, the program product of this specification is not limited thereto. In this specification, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a readable storage medium include: portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include a data signal propagated as part of a carrier wave in baseband, carrying readable program code. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on electronic device 200, partially on electronic device 200, as a standalone software package, partially on electronic device 200 and partially on a remote computing device, or entirely on a remote computing device.

[0126] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0127] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0128] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0129] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and to aid in understanding a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art, upon reading this specification, may readily identify some of the devices as separate embodiments. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. And the content of each secondary embodiment is valid even if it contains fewer than all the features of a single foregoing disclosed embodiment.

[0130] Every patent, patent application, publication of a patent application, and other material such as articles, books, specifications, publications, documents, articles, etc., cited herein, except for those inconsistent with or conflicting with this document, or those having a restrictive effect on the widest scope of the claims, may be incorporated herein by reference for all purposes now or hereafter associated with this document. Furthermore, in the event of any inconsistency or conflict between the description, definition, and / or use of relevant terms in any material and the description, definition, and / or use of relevant terms in this document, the terms in this document shall prevail.

[0131] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A method for controlling the forgetting of data in a target model, wherein the target model is trained based on an original sample set, the method comprising: In response to a request to forget the target data, a target sample subset containing the target data is determined from the original sample set, and a hedging sample subset without the target data is generated based on the target sample subset; The target sample subset and the hedging sample subset are input into the target model for processing to obtain the processing result. Based on the processing result, the original metric information of the parameter space of the target model is updated to obtain the target metric information. The original metric information represents the sensitivity information of the target model in different directions in the parameter space after training based on the original sample set. The target metric information represents the sensitivity information of the target model in different directions in the parameter space after forgetting the target sample subset and memorizing the hedging sample subset. Based on the processing results, the parameter update direction required by the target model to forget the target sample subset and remember the hedging sample subset is determined; and Based on the target metric information and the parameter update direction, the parameter change of the target model is determined, and the parameters of the target model are updated based on the parameter change.

2. The method according to claim 1, wherein, The processing results include: the first gradient corresponding to each target sample in the target sample subset, and the second gradient corresponding to each hedging sample in the hedging sample subset.

3. The method according to claim 2, wherein, The step of updating the original metric information of the parameter space of the target model based on the processing result to obtain the target metric information includes: Based on the first gradient corresponding to each target sample in the target sample subset, the original metric information of the parameter space of the target model is updated to obtain intermediate metric information, wherein the intermediate metric information characterizes the sensitivity information of the target model in different directions in the parameter space after forgetting the target sample subset; and Based on the second gradient corresponding to each hedging sample in the hedging sample subset, the intermediate metric information is updated to obtain the target metric information.

4. The method according to claim 3, wherein, The process involves updating the original metric information of the parameter space of the target model based on the first gradient corresponding to each target sample in the target sample subset, to obtain intermediate metric information, including: Traverse the subset of target samples. For the i-th target sample, update the current metric information based on the first gradient corresponding to the i-th target sample. Wherein, when i=1, the current metric information is the original metric information; when i>1, the current metric information is the metric information obtained after traversing the first i-1 target samples. The measurement information obtained after the traversal is completed is used as the intermediate measurement information.

5. The method according to claim 3, wherein, The intermediate metric information is updated based on the second gradient corresponding to each hedging sample in the hedging sample subset to obtain the target metric information, including: Traverse the subset of hedged samples. For the i-th hedged sample, update the current metric information based on the second gradient corresponding to the i-th hedged sample. Wherein, when i=1, the current metric information is the intermediate metric information; when i>1, the current metric information is the metric information obtained after traversing the first i-1 hedged samples. The measurement information obtained after the traversal is completed is used as the target measurement information.

6. The method according to claim 2, wherein, The target sample subset and the hedging sample subset form multiple sample pairs, each sample pair including a target sample and its corresponding hedging sample. Based on the processing result, determining the parameter update direction required by the target model to forget the target sample subset and remember the hedging sample subset includes: For each of the plurality of sample pairs, determine the gradient difference between the first gradient corresponding to the target sample and the second gradient corresponding to the hedging sample in the sample pair; and The overall gradient difference is obtained by summing the gradient differences of the multiple sample pairs. The overall gradient difference is used to characterize the parameter update direction required by the target model to forget the target sample subset and remember the hedging sample subset.

7. The method according to claim 6, wherein, The step of determining the parameter change of the target model based on the target metric information and the parameter update direction includes: Based on the target metric information, multiply by the overall gradient difference to obtain the parameter change of the target model.

8. The method according to claim 2, wherein, The step of inputting the target sample subset and the hedging sample subset into the target model for processing to obtain the processing result includes: For each target sample in the subset of target samples, the target sample is input into the target model to obtain a first prediction result. Based on the first prediction result, a first prediction loss of the target model for the target sample is determined, and the first prediction loss is backpropagated in the target model to obtain a first gradient corresponding to the target sample; and For each hedging sample in the subset of hedging samples, the hedging sample is input into the target model to obtain a second prediction result. Based on the second prediction result, the second prediction loss of the target model for the hedging sample is determined, and the second prediction loss is backpropagated in the target model to obtain the second gradient corresponding to the hedging sample.

9. The method according to claim 1, further comprising: At least a portion of the samples in the original sample set are input into the target model for processing to obtain the gradients corresponding to each of the at least a portion of the samples. as well as The original metric information is generated based on the gradients corresponding to at least some of the samples.

10. The method according to claim 9, wherein, The generation of the original metric information based on the gradients corresponding to at least some of the samples includes: Based on the parameter size of the target model, generate initialization metric information of the same size; Iterate through at least a portion of the samples, and for the i-th sample, update the current metric information based on the gradient corresponding to the i-th sample. Wherein, when i=1, the current metric information is the initial metric information; when i>1, the current metric information is the metric information obtained after iterating through the first i-1 samples; and The measurement information obtained after the traversal is completed is used as the original measurement information.

11. The method according to claim 10, wherein, The initialization metric information is in matrix form, and updating the initialization metric information includes updating the diagonal elements of the matrix.

12. The method according to claim 9, further comprising, after generating the original measurement information: The original measurement information is stored in a preset storage space; as well as Upon receiving a request to forget the target data, the method further includes: obtaining the original measurement information from the preset storage space.

13. The method according to claim 9, wherein the at least part of the samples are samples obtained by sampling the original sample set according to a preset sampling rule, wherein, The preset sampling rules include at least one of the following: random sampling or proportional sampling for different types of samples.

14. The method according to claim 1, wherein, Determining the target sample subset containing the target data from the original sample set includes: The information item indicating the target data is parsed from the forget request, and samples containing the information item are queried from the original sample set to obtain a subset of the target samples; or The sample index used to indicate the target data is parsed from the forget request, and the sample corresponding to the sample index is queried from the original sample set to obtain the target sample subset.

15. The method according to claim 1, wherein, The step of generating a hedging sample subset that does not contain the target data based on the target sample subset includes: For each target sample in the target sample subset, locate the first string corresponding to the target data in the target sample, and replace the first string with a second string; and The hedging sample subset is generated based on the replacement results of each target sample in the target sample subset.

16. The method according to claim 15, wherein, The second string satisfies at least one of the following: Meaningless strings Characterizing the name of the information item corresponding to the first string, Empty string, The data type is the same as the first string, or A string that represents a rejection.

17. The method according to claim 1, wherein, The step of updating the parameters of the target model based on the parameter change includes: The updated parameters are obtained by subtracting the parameter change from the current parameters of the target model.

18. The method according to claim 1, wherein, The original metric information includes the inverse Fisher matrix or the inverse Hessian matrix.

19. The method of claim 1, wherein the method is applied to a service device, and the forgetting request originates from a control device. Before receiving a request to forget the target data, the method further includes: The system receives the original sample set from the control device and uses the original sample set to train the model to obtain the target model. as well as After updating the parameters of the target model, the method further includes deleting the target sample subset from the original sample set.

20. A dialogue method, comprising: Obtain the question text; The question text is input into the target model, wherein the target model is trained based on the original sample set and configured to have forgotten the target data in the original sample set, and the forgetting process of the target model for the target data is implemented using the method described in any one of claims 1 to 19; and Obtain the response text output by the target model, wherein the response text does not contain the target data.

21. An electronic device, comprising: At least one storage medium storing at least one instruction set; as well as At least one processor is communicatively connected to the at least one storage medium, wherein the at least one processor reads the at least one instruction set and executes the method as described in any one of claims 1-20 according to the instructions of the at least one instruction set.

22. A computer-readable non-volatile storage medium, wherein, The computer-readable non-volatile storage medium stores at least one instruction set, which, when executed by at least one processor, implements the method as described in any one of claims 1-20.

Images