A method and system for image privacy protection during the use of large models

Abstract

The present invention discloses a method and system for image privacy protection during the use of large models, which relates to the technical field of large models. In this method, the client does not send the real image to the large model alone, but mixes the obfuscated image into the real image. The large model processes both the real image and the obfuscated image. The client receives the processing results and only returns the part of the processing results corresponding to the real image to the user. Based on the solution of mixing the obfuscated image into the real image, the image privacy of users during dialogue and question-answering with the large model is protected from being leaked.

Description

Technical Field

[0001] This application relates to the technical field of large models, and particularly to a method and system for image privacy protection during the use of large models. Background Art

[0002] In recent years, the breakthrough progress of deep learning technology has promoted the remarkable development of large language models (LLMs) in the field of natural language processing. Large language models (or large models) not only perform excellently in tasks such as text generation and question-answering systems, but also gradually combine with various auxiliary tools to form large model agents integrated with enhanced tools. When using large model agents with integrated tools, there are scenarios where users upload private images. In this scenario, the large model and the third-party tools it invokes can utilize the visual cues in the images to infer private information such as the user's identity, location, interests, and intentions. And through advanced analysis techniques, deeper mining of the user's privacy can be carried out. The privacy risks caused by the use of users' private images are attracting much attention, and effective solutions are urgently needed to protect users' privacy information.

[0003] Currently, the privacy protection solutions for the submission of private images during the use of large models mainly include methods such as local deployment, anonymous communication, and data masking. However, these methods have limitations in actually protecting users' privacy.

[0004] Local deployment method: Deploying open-source large models and tools locally can ensure that data does not leave the local execution environment, thereby achieving the purpose of protecting privacy. However, this method requires users to have a high level of professional knowledge and hardware resources, and only open-source large models and tools can be used, resulting in the inability to utilize many powerful closed-source models and tools.

[0005] Anonymous communication method: Anonymous communication can hide the identities of both communication parties and prevent third parties from tracking or identifying participants. However, anonymous communication does not provide complete privacy protection, and Internet Protocol (IP) addresses may still be traced.

[0006] Data masking method: Data masking partially covers or blurs the private images submitted by users to ensure that privacy information is not exposed during image transmission and processing. However, data masking affects the processing effect of images, especially performing poorly in tasks that require high-precision image analysis.

[0007] In summary, for the privacy protection problem when users interact with large models involving the submission of images, there is an urgent need to find a new solution. Summary of the Invention

[0008] In an embodiment of the present application, a method and system for image privacy protection during the use of large models are provided. Based on the solution of mixing obfuscated images into real images, the image privacy of users during dialogue and question - answering with large models is protected from being leaked.

[0009] To achieve the above - mentioned purpose, the embodiments of the present application adopt the following technical solutions:

[0010] In a first aspect, an embodiment of the present application provides a method for image privacy protection during the use of large models, which is applied to a client. The method includes:

[0011] Receive a first question - answering request from a user, and obtain a real image from the first question - answering request; the first question - answering request is used to request the generation of a first question - answering result for the real image;

[0012] Extract real attribute information from the real image;

[0013] Generate an obfuscated image by using the real attribute information;

[0014] Randomly rearrange the order of the real image and the obfuscated image to generate an image set; generate a second question - answering request based on the image set; the second question - answering request is used to request the generation of a second question - answering result for the image set;

[0015] Send the second question - answering request to the large model and receive the second question - answering result returned by the large model;

[0016] Return the first question - answering result generated for the real image in the second question - answering result to the user.

[0017] In combination with the first aspect, in a possible design, the step of generating an obfuscated image by using the real attribute information includes the following steps:

[0018] Send the real attribute information to the large model; wherein, the real attribute information is converted by the large model into text prompt content that can be recognized by an image generation tool and sent to the image generation tool;

[0019] Receive the encrypted obfuscated image generated by the image generation tool based on the text prompt content, and perform decryption processing to obtain the decrypted obfuscated image; wherein, after the encrypted obfuscated image is sent by the image generation tool to the large model, it is sent by the large model to the client.

[0020] In combination with the first aspect, in a possible design, before receiving the encrypted obfuscated image generated by the image generation tool based on the text prompt content, the method further includes:

[0021] Send an image generation request to the large model, where the image generation request includes request information and a first public key; wherein, the first public key is used to combine with a second public key generated by the image generation tool to obtain a shared key;

[0022] Obtain the shared key based on a key exchange protocol, and the shared key is used for the client to decrypt the encrypted obfuscated image to obtain the decrypted obfuscated image.

[0023] Combined with the first aspect, in a possible design, extracting real attribute information from the real image includes:

[0024] Extract feature information of the picture from the user's uploaded photo to obtain picture feature information.

[0025] Combined with the first aspect, in a possible design, after randomly rearranging the order of the real image and the obfuscated image to generate an image set, the method further includes:

[0026] Convert the image set into a data table, and determine the sensitive attributes of the user from the data table;

[0027] Calculate the difference value between the distribution of sensitive attributes in each equivalence class and the distribution of sensitive attributes in the data table; wherein, one equivalence class corresponds to the data converted from one image, and the image includes the real image and the obfuscated image;

[0028] When the difference value is greater than a preset threshold, adjust the generation quantity and attribute difference when the image generation tool generates the obfuscated image next time.

[0029] Combined with the first aspect, in a possible design, calculating the difference value between the distribution of sensitive attributes in each equivalence class and the distribution of sensitive attributes in the data table includes:

[0030] Use WD to represent the difference value. In a set containing n images, the distribution of sensitive features in the equivalence class and the distribution in the whole set are expressed as:

[0031] ;

[0032] ;

[0033] Wherein, P v is the weight associated with category v in the distribution, q w is the weight associated with category w in the Q distribution, and the WD between P and Q is expressed as:

[0034] ;

[0035] The WD expression satisfies the following constraints:

[0036] ;

[0037] ;

[0038] ;

[0039] where d vw is the distance between class v and class w, and f vw is the flow variable, and f vw represents the probability mass transferred from class v in distribution P to class w in distribution Q;

[0040] Convert the WD expression to:

[0041] .

[0042] In a second aspect, an embodiment of the present application provides a method for image privacy protection during the use of a large model, which is applied to a server. The server is deployed with a large model and an image generation tool. The method includes:

[0043] Receiving real attribute information about a target user sent by a client, where the real attribute information is extracted by the client from real images of the target user;

[0044] Performing image generation processing on the real attribute information through the image generation tool to obtain a confused image, and transmitting the confused image to the client;

[0045] Receiving a second question-and-answer request sent by the client, and obtaining an image set from the second question-and-answer request. The image set includes the real image and the confused image whose order is randomly rearranged;

[0046] Analyzing the image set through the large model to generate a second question-and-answer result.

[0047] Combined with the second aspect, in a possible design, after receiving the real attribute information about the target user sent by the client, the method further includes:

[0048] Converting the real attribute information into text prompt content that can be recognized by the image generation tool through the large model;

[0049] The performing image generation processing on the real attribute information through the image generation tool to obtain a confused image, and transmitting the confused image to the client includes:

[0050] The image generation tool performs image generation processing based on the text prompt content to obtain a confused image;

[0051] The image generation tool encrypts the confused image to obtain an encrypted confused image; after the encrypted confused image is sent by the image generation tool to the large model, it is sent by the large model to the client.

[0052] Combined with the second aspect, in a possible design, the real image is a user-uploaded photo;

[0053] The large model analyzes the image set to generate a second Q&A result, including:

[0054] The large model analyzes the uploaded photo and the confused image in the image set respectively to generate a second Q&A result; wherein, the first Q&A result is included in the second Q&A result, and the first Q&A result is the response text obtained by the large model analyzing the user's uploaded photo.

[0055] In a third aspect, an embodiment of the present application provides a computer device, including a processor and a memory, the processor is configured to run a computer program in the memory to execute the method of the first aspect and its possible design manners.

[0056] In a fourth aspect, an embodiment of the present application provides a server, including a memory and a processor, a computer program is stored in the memory, and the processor is set to run the computer program to execute the method of the second aspect and its possible design manners.

[0057] In a fifth aspect, an embodiment of the present application provides a system for image privacy protection during the use of a large model, including a client and a server. The client, which depends on a computer device to implement functions, is configured to execute the method of the first aspect and its possible design manners, and the server is configured to execute the method of the second aspect and its possible design manners.

[0058] In a sixth aspect, an embodiment of the present application provides a storage medium, in which a computer program is stored, and the computer program is set to execute the method of the first aspect and its possible design manners, or execute the method of the second aspect and its possible design manners when running.

[0059] Compared with the prior art, a method and system for image privacy protection during the use of large models provided by an embodiment of the present application. In the method, the client receives a first question-and-answer request from the user and obtains a real image from the first question-and-answer request; the first question-and-answer request is used to request the generation of a first question-and-answer result for the real image. The client obtains real attribute information from the real image, and then uses the real attribute information to generate a confused image. The client then randomly rearranges the order of the real image and the confused image to generate an image set; a second question-and-answer request is generated based on the image set; the second question-and-answer request is used to request the generation of a second question-and-answer result for the image set. After that, the client sends the second question-and-answer request to the large model, receives the second question-and-answer result returned by the large model, and returns the first question-and-answer result generated for the real image in the second question-and-answer result to the user. By generating a confused image and mixing it with the real image, it can effectively increase the difficulty for attackers to identify the real image. The large model uniformly processes all images without knowing which image is the real image, which can avoid directly exposing the user's real image while ensuring the realization of the service function. This not only improves the security of the system but also ensures the quality and efficiency of the service.

[0060] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more concise and understandable. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0062] Figure 1 A schematic diagram of a system for user interaction with a large model agent provided by an embodiment of the present application is shown;

[0063] Figure 2 A schematic diagram of a method for image privacy protection during the use of a large model provided by an embodiment of the present application is shown;

[0064] Figure 3 A schematic diagram of a system for image privacy protection during the use of a large model provided by an embodiment of the present application is shown;

[0065] Figure 4 A flowchart of a method for image privacy protection during the use of a large model provided by an embodiment of the present application is shown;

[0066] Figure 5 A flowchart of another method for image privacy protection during the use of a large model provided by an embodiment of the present application is shown;

[0067] Figure 6 A schematic diagram showing a method for mixing obfuscated images provided by an embodiment of the present application;

[0068] Figure 7 A hardware structure block diagram of an electronic device provided by an embodiment of the present application;

[0069] Figure 8 A structure block diagram of a device for image privacy protection during the use of large models provided by an embodiment of the present application. Detailed implementation manners

[0070] To more clearly understand the purpose, technical solution, and advantages of the present application, the present application will be described and explained below with reference to the accompanying drawings and embodiments.

[0071] Unless otherwise defined, the technical terms or scientific terms involved in the present application shall have the general meaning understood by those with ordinary skills in the technical field to which the present application belongs. In the present application, words such as "a", "an", "one kind", "the", "these", etc. do not indicate a limitation in quantity, and they can be singular or plural. The terms "including", "comprising", "having" and any variants thereof involved in the present application are intended to cover non-exclusive inclusion; for example, a process, method, device, product or equipment including a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent in these processes, methods, products or equipment. The terms "connected", "coupled" and other similar words involved in the present application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly connected. The term "plurality" involved in the present application means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" may represent: A exists alone, A and B exist simultaneously, and B exists alone. Usually, the character " / " indicates that the objects associated before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in the present application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0072] First, the terms that may be involved in the embodiments of the present application will be explained:

[0073] DH (Diffie-Hellman key exchange protocol): A security protocol that ensures secure key exchange between two communicating parties over an insecure channel.

[0074] WD (Wasserstein Distance): A distance metric method based on probability distributions, which describes the minimum cost required to transform one probability distribution into another. It is commonly used in fields such as probability distributions, image processing, and machine learning.

[0075] Text-to-image generation model: A machine learning-based model that generates images consistent with the input natural language descriptions. Generally, such models contain a language model and a generation model. The language model transforms the natural language input descriptions into vectors in the latent space, and the generation model uses these text vectors to generate corresponding images. For example, the image generation tool mentioned in this application can generate obfuscated images through this model.

[0076] Key exchange protocol: Often applied in insecure communication environments, allowing multiple parties to securely establish a shared key. The shared key can be used to encrypt and decrypt information, ensuring the confidentiality, integrity, and authenticity of messages during communication. The key exchange protocol allows parties to establish a shared key without prior knowledge of each other's keys.

[0077] t-closeness principle: A privacy protection principle to prevent the leakage of sensitive attributes. According to this principle, the attributes in a data record can be divided into three categories:

[0078] Explicit identifier: An attribute that can directly and uniquely identify an individual.

[0079] Quasi-identifier: An attribute that cannot uniquely identify an individual when used alone, but may narrow down the range of individuals when combined with other quasi-identifiers.

[0080] Sensitive attribute: An attribute that is usually regarded as private or sensitive.

[0081] The t-closeness principle requires that the distribution of sensitive attributes in each equivalence class (a group of records with the same quasi-identifiers) should be as close as possible to the overall distribution of sensitive attributes in the entire dataset. Specifically, the gap between the distribution of sensitive attributes in the equivalence class and the distribution of sensitive attributes in the dataset should not exceed the set threshold t.

[0082] First, describe the process of interaction between the user and the large model agent. Please refer to Figure 1 , Figure 1The figure shows a schematic diagram of a system for user interaction with a large model agent provided by an embodiment of the present application. The system includes a user 101, a large model 102, and a task execution tool 103. In the first step, the user request Q = {q, x}, which includes a public query q and a private image input x, where the sensitive information included in x is denoted as p(x). The user 101 hopes to obtain an answer through the large model 102 and avoid the large model 102 and the task execution tool 103 inferring the relevance between the sensitive information p(x) and the user 101.

[0083] The large model 102 acts as a controller in the system, responsible for task planning and selecting an appropriate task execution tool 103 to complete different tasks. The task execution tool 103 executes the tasks assigned by the large model 102 according to the input parameter param. There are two-way communication channels between the user 101 and the large model 102, and between the large model 102 and the task execution tool 103. The user 101 sends the request Q to the large model 102 and expects to receive a correct response. After receiving the user request, the large model 102 performs the second step, parses the request Q and decomposes it into multiple subtasks to form a task set task i = {key, target, param}. The large model 102 plans the task sequence and dependencies, and then generates a response template t based on all the tasks.

[0084] In the third step, the large model 102 assigns the parsed tasks to the task execution tool 103. The task execution tool 103 executes the assigned tasks according to the input parameter param and returns the prediction result to the large model 102.

[0085] In the fourth step, the large model 102 fills the received results into the template t, generates the final response result T, and returns it to the user 101.

[0086] In the fifth step, the user 101 obtains the response result.

[0087] As can be seen from Figure 1 it, when a user interacts with a large model, it involves the user submitting a public query and a private image (equivalent to the real image in the embodiment of the present application) to the large model, and then the large model orchestrates tasks and invokes relevant enhancement tools to execute the tasks. In this process, the large model obtains the user's private image input and shares this information with relevant tools during the task execution process, leading to the risk of exposure of the user's privacy information. For example, when a user provides a private image during interaction with a large model agent, the large model shares the image with a third-party tool according to business needs, and the large model and the tool can infer the user's privacy information through analysis.

[0088] Although some privacy protection schemes have been proposed for submitting private images during the use of large models, each method has its deficiencies: local deployment requires users to have high technical capabilities and hardware resources, and is limited to using open-source models and tools; anonymous communication cannot provide complete privacy protection; the data masking method will affect the processing effect of images.

[0089] To solve the privacy protection problem when users submit images during the interaction with large models, the embodiments of this application provide a new solution. Please refer to Figure 2 , Figure 2 FIG. shows a schematic diagram of a method for image privacy protection during the use of large models provided by the embodiments of this application. In the method, the client does not send the real image to the large model alone, but mixes the obfuscated image into the real image. The large model processes both the real image and the obfuscated image. The client receives the processing results and only returns the part of the processing results corresponding to the real image to the user. Based on the solution of mixing the obfuscated image into the real image, the image privacy of users during dialogue and Q&A with large models is protected from being leaked. In this embodiment, the client can use an image generation tool to generate the obfuscated image. As another implementation, the client can execute the steps of the image generation tool, that is, the client generates the obfuscated image based on the real attribute information.

[0090] The following takes the method provided by the embodiments of this application applied to skin condition analysis as an example for illustration. It should be understood that the examples given in the embodiments of this application do not limit the scope of patent protection. The method provided by the embodiments of this application can be applied to various scenarios such as medical and health, financial data desensitization, and intelligent security. The purpose is to protect the information in the real image that can reveal the user's real portrait and prevent the large model and its tools from further mining the user's privacy based on this information. Taking the medical and health field as an example, this method can be used to protect patient privacy. For example, when analyzing skin conditions, the user uploads a skin image, and big data analyzes it. During this process, the system generates an obfuscated image and encrypts the transmission to ensure that the large model and its tools can accurately diagnose while being unable to identify the patient's identity. Another example is that in the financial industry, this method can be used to process sensitive image data (the user's ID card image) so that it is not leaked. Another example is that when applied to the field of intelligent security, this method can hide the residential location of pedestrians to avoid the large model associating the residential location of pedestrians with their identity information and inferring the pedestrian's movement trajectory.

[0091] The method provided by the embodiments of this application is applied to a system for image privacy protection during the use of large models, as shown in Figure 3 FIG., the system includes a client 31 and a server 32. A large model and an image generation tool 33 are deployed on the server.

[0092] Among them, the client provides an interface for users, responsible for receiving the real images uploaded by users and the first question-and-answer request. The client refers to a program that provides local services for customers. The client can be software installed in computer devices, including mobile phones, computers, etc. The client can be a browser, a mail client, or a chat application. The large model is responsible for analyzing the images and generating question-and-answer results. The image generation tool is configured to generate obfuscated images based on the feature information.

[0093] In some embodiments, the client communicates bidirectionally with the large model and the image generation tool respectively. In the embodiments, please refer to Figure 4 , Figure 4 FIG. shows a flowchart of a method for image privacy protection during the use of a large model provided by an embodiment of the present application. The method includes steps S401 to step S411.

[0094] Step S401: The client receives a first question-and-answer request from the user.

[0095] In this embodiment, step S401 refers to the client receiving a question-and-answer request initiated by the user, and this request contains the user's question-and-answer intention and related information. Specifically, the first question-and-answer request includes request information, and the first question-and-answer request is used to request the generation of a first question-and-answer result for the real image.

[0096] In practical applications, the user initiates a first question-and-answer request to the client through the interaction interface, and the client receives the first question-and-answer request initiated by the user. This request contains the real image uploaded by the user and the associated semantic instruction. Among them, the real image refers to the original image file provided by the user that contains personal biometric or sensitive information, and the associated semantic instruction can be, for example, "analyze skin condition" correspondingly. What the user wants to get is the question-and-answer result (i.e., the first question-and-answer result) after analyzing the real image.

[0097] The first question-and-answer request is described below in combination with an application scenario. In the scenario of skin condition analysis and processing, the first question-and-answer request may be a facial photo uploaded by the user, with an instruction of "please analyze my skin condition" attached. In this example, the user requests an analysis of their facial skin condition and uploads a facial photo of the user. This facial photo refers to the real image. The client receives the request from the user to analyze their facial skin condition.

[0098] Step S402: The client obtains the real image from the first question-and-answer request.

[0099] After receiving the user's first question-and-answer request, the client needs to extract the real image uploaded by the user from it. As an example, the user can upload the real image to the client so that the client can obtain the data.

[0100] After step S402, the client does not directly send the real image to the large model for analysis. Instead, it extracts the attribute information from the real image and sends the attribute information to the large model, so that the large model cannot restore the user's real image based on the attribute information. For specific reference, see step S403.

[0101] Step S403: The client extracts real attribute information from the real image.

[0102] In this step, the client deeply analyzes the obtained real image and extracts the real attribute information contained therein. Among them, the real attribute information includes information such as the face, limbs, torso, skin, geographical location, etc. that can reveal the user's real portrait.

[0103] Step S403 may further include: extracting features from the user's uploaded photo to obtain picture feature information. Among them, the uploaded photo can be a human photo, a landscape photo, a house photo, etc. Among them, the human photo can be a face photo, a hand photo, a leg photo, etc. In the case where the human photo is a face photo, the real attribute information includes one or more of skin type, facial features, gender, and age, which will be elaborated one by one below.

[0104] Skin type: Extract features such as skin color and texture in the image to determine whether the user's skin is dry, oily or combination; extract skin lesion features in the image to determine the symptom manifestations of the user's skin; extract facial follicle state features in the image to determine whether the user has inflammation or skin lesions.

[0105] Facial features: Extract features such as the user's face shape (such as round, long) and the position and shape of the facial features (such as eye size, nose bridge height).

[0106] Gender: Infer the user's gender based on factors such as facial contour and eyebrow shape as the user's gender feature.

[0107] Age: Estimate the user's age range in combination with features such as wrinkle distribution and skin elasticity as the user's age feature.

[0108] One or more of the above features are used to generate a confused image subsequently. It can be understood that since the real attribute information is extracted from the real image but is not completely equivalent to the real image, these data not only help generate a confused image that is similar to but not exactly the same as the real image, but also ensure that the confused image maintains consistency in some key attributes, thus achieving the effect of protecting user privacy.

[0109] In some of these embodiments, the client performs multi-dimensional analysis on the real image through a pre-trained feature extraction model and outputs real attribute information. For example, the U-Net segmentation model is used to locate the skin area and calculate the pore density and erythema index; the FaceNet model is adopted to generate an embedding vector, and the embedding vector contains geometric features such as the proportion of facial features and the contour curvature, etc.

[0110] Step S404: The client sends the real attribute information to the image generation tool.

[0111] After the feature extraction is completed, the client will send the extracted real attribute information to the image generation tool dedicated to generating the obfuscated image. Among them, the image generation tool can be deployed on the client or on the server side. In this embodiment, the image generation tool and the large model do not communicate with each other, so the client directly interacts with the image generation tool. For example, in the above example, the client may pack a series of feature information such as "dry skin", "round face", "female", "30 years old" into a data packet and then transmit it to the image generation tool through network transmission.

[0112] Step S405: The image generation tool performs image generation processing on the real attribute information to obtain the obfuscated image.

[0113] The real attribute information is used for the image generation tool to generate the obfuscated image; that is, after the image generation tool receives the real attribute information sent by the client, it will use an algorithm to process it and generate the corresponding obfuscated image. The number of obfuscated images can be one or multiple.

[0114] Continuing with the skin condition analysis as an example, assume that the feature extraction obtains that the user's skin type is "dry", the facial feature is "round face", the gender is "female", and the age is about "30 years old". Then, the image generation tool may generate the following types of obfuscated images.

[0115] Obfuscated Image A: A facial photo with dry skin characteristics but slightly different facial features, and there are erythemas on the face.

[0116] Obfuscated Image B: A facial photo with a round face but a different skin type (such as oily or combination), and there is dermatitis on the face.

[0117] Although Obfuscated Image A and Obfuscated Image B have some differences in features from the real image, they maintain a high correlation with the real image in key attributes (such as having lesion features on the face), which can increase the difficulty for the large model or the image generation tool to identify which image is the user's real image, thereby achieving the purpose of protecting user privacy.

[0118] Step S406: The client receives the obfuscated image from the image generation tool.

[0119] In this embodiment, the client can communicate with the image generation tool. Then, the obfuscated images generated by the image generation tool cannot be obtained by the large model, and the large model cannot infer which image is the user's real image and which images are the obfuscated images generated by the image generation tool. Therefore, in this embodiment, the obfuscated images can be unencrypted. Of course, to improve transmission security, the obfuscated images can be encrypted before transmission to prevent attackers from intercepting the obfuscated images and analyzing the user's sensitive information.

[0120] The number of obfuscated images can be one or more. As an example, the image generation tool generates five obfuscated images at once and sends them back to the client in the form of a folder or a compressed package.

[0121] Step S407: The client randomly rearranges the order of the real image and the obfuscated images generated by the image generation tool to generate an image set.

[0122] In this step, the client mixes the real image obtained in step S402 and the newly generated obfuscated images. The mixing method is to rearrange the order of the images according to a certain random rule to form a new image set. In this way, the large model is prevented from inferring the position of the real image in the image set through the orderliness of the image arrangement.

[0123] The following explains the mixing method: If the real image is a user's facial photo and there are four obfuscated images of different types, then the client can number these five images as A, B, C, D, and E, and then use a random number generator to shuffle their order. For example, the new order obtained is C, A, E, B, D.

[0124] This step further increases the difficulty for attackers to identify the real image by randomly rearranging the order of the real image and the obfuscated images, and improves the security of privacy protection.

[0125] Step S408: The client sends a second question-and-answer request generated based on the image set to the large model.

[0126] After completing the construction of the image set, the client generates a second question-and-answer request based on this. The second question-and-answer request contains the user's real image and the generated obfuscated images. The second question-and-answer request is used to request the large model to analyze the real image and the obfuscated images respectively and generate analysis results.

[0127] Continuing with the above skin condition analysis example, the second question-and-answer request can be "Please perform a skin condition analysis on this image set containing multiple facial photos". This request requires the large model to perform corresponding analysis processing on each image.

[0128] In this step, by generating a second Q&A request and passing the image set to the large model, the large model can uniformly process all the images without knowing which image is real. This not only ensures the realization of the large model service function but also avoids directly exposing the user's real images.

[0129] Step S409: The large model analyzes the image set and generates a second Q&A result.

[0130] After receiving the second Q&A request and the attached image set sent by the client, the large model will analyze and process each image one by one and give corresponding analysis results.

[0131] Continuing with the example of skin condition analysis, the large model will separately evaluate the skin condition for each image in the image set and obtain response results such as "The skin condition shown in this image is dry with slight wrinkles" or "You have a dermatitis problem and need to seek medical attention as soon as possible".

[0132] Since neither the real image nor the confused image has been transmitted to the large model separately, the large model cannot analyze which image in the image set is the user's real image, so the large model cannot use the visual clues in the images to infer privacy information such as the user's identity, location, interests, and intentions.

[0133] Step S410: The client obtains the first Q&A result generated from the real image from the second Q&A result.

[0134] After receiving the second Q&A result returned by the large model, the client finds the part corresponding to the original real image from it, which is the first Q&A result.

[0135] In the above example, if there are five records in the second Q&A result, corresponding to the five images in the image set, and only one record is the analysis result of the real facial photo uploaded by the user, then that record is the first Q&A result. If the real image is the user's facial photo and the first Q&A result is the skin condition response text obtained by the large model analyzing the user's facial photo, then the first Q&A result can be "You have a dermatitis problem and need to seek medical attention as soon as possible".

[0136] Step S411: The client returns the first Q&A result to the user.

[0137] Through the above steps S401 to S411, the client receives the first Q&A request from the user, obtains the real image from the first Q&A request; the first Q&A request is used to request the generation of the first Q&A result for the real image. Then the client extracts features from the real image to obtain the real attribute information of the user, and then uses the real attribute information to generate a confused image. The client then randomly rearranges the order of the real image and the confused image generated by the image generation tool to generate an image set; generates a second Q&A request based on the image set; the second Q&A request is used to request the generation of the second Q&A result for the image set. After that, the client sends the second Q&A request to the large model, receives the second Q&A result returned by the large model, and returns the first Q&A result generated for the real image in the second Q&A result to the user. By generating a confused image and mixing it with the real image, it can effectively increase the difficulty for attackers to identify the real image. The large model uniformly processes all images without knowing which image is the real image, which can avoid directly exposing the user's real image while ensuring the realization of the service function. This not only improves the security of the system but also ensures the quality and efficiency of the service.

[0138] The above takes the case where the large model and the image generation tool do not communicate with each other as an example to illustrate this solution. When the large model is an agent large model, that is, the large model integrates an image generation tool, the user directly communicates with the large model. The large model transmits the real attribute information to the image generation tool and transmits the confused image returned by the image generation tool to the user. In this scenario, in order to avoid the leakage of user privacy data, the confused image generated by the image generation tool needs to be encrypted.

[0139] Specifically, please refer to Figure 5 , Figure 5 shows a flowchart of another method for image privacy protection during the use of a large model provided by an embodiment of the present application. This method includes steps S401 to S404, S501 to S507, and steps S407 to S411. Among them, for steps S401 to S404 and steps S407 to S411, reference can be made to the above description. The following mainly describes steps S501 to S507.

[0140] Step S401: The client receives the first Q&A request from the user.

[0141] Step S402: The client obtains the real image from the first Q&A request.

[0142] Step S403: The client extracts features from the real image to obtain the real attribute information of the user.

[0143] Step S404: The client sends the real attribute information to the large model.

[0144] Step S501: The large model converts the real attribute information into text prompt content that can be recognized by the image generation tool.

[0145] In this step, the large model can convert the real attribute information into instructions that conform to the input specifications of the image generation model through a semantic mapping algorithm.

[0146] Step S502: The large model sends the text prompt content to the image generation tool.

[0147] The large model transmits the text prompt content to the image generation tool through API (Application Programming Interface) call and triggers the image generation service, so as to obtain a set of obfuscated images that meet the privacy protection requirements.

[0148] Step S503: The image generation tool performs image generation processing according to the text prompt content to obtain obfuscated images.

[0149] The image generation tool generates multiple obfuscated images that are semantically related to the real image but not real based on the text prompt content. That is, the obfuscated image refers to a synthetic image that is visually similar to the user's real image but cannot be associated with a specific individual. By creating visually similar but identity-unrelated interference data, it increases the difficulty for the large model to obtain privacy.

[0150] Step S504: The image generation tool encrypts the obfuscated images to obtain encrypted obfuscated images.

[0151] Before step S504, such as before and after the execution of step S404, in addition to sending the real attribute information to the large model, the client also sends a first public key; wherein, the first public key is used to combine with the second public key generated by the image generation tool to obtain a shared key. The shared key is generated in the image generation tool, and the client can obtain the shared key based on the key exchange protocol. The shared key is only obtained by the client and the image generation tool. The shared key is used for the image generation tool to encrypt the obfuscated images to obtain encrypted obfuscated images, and is also used for the client to decrypt the encrypted obfuscated images to obtain obfuscated images.

[0152] Step S505: The image generation tool sends the encrypted obfuscated images to the large model.

[0153] Step S506: The large model sends the encrypted obfuscated images to the client.

[0154] In steps S504 to S506, the image generation tool ensures the confidentiality of the obfuscated images during transmission with the large model through encryption.

[0155] In some of these embodiments, after step S506, the method further includes: converting the image set into a data table, determining the sensitive attributes of the user from the data table; calculating the difference value between the distribution of the sensitive attributes in each equivalence class and the distribution of the sensitive attributes in the data table; wherein, one equivalence class corresponds to the data converted from one image, and the images include real images and obfuscated images; in the case where the difference value is greater than a preset threshold, adjusting the generation quantity and attribute difference when the image generation tool generates obfuscated images next time.

[0156] Specifically, in the case where the difference value is greater than a preset threshold, increasing the generation quantity when the image generation tool generates obfuscated images next time, and / or increasing the attribute difference when the image generation tool generates obfuscated images next time, so that the privacy always conforms to the t-closeness principle.

[0157] This embodiment dynamically adjusts the generation quantity and attribute difference of the obfuscated pictures based on the difference value, optimizing the effect of privacy protection. Compared with the fixed privacy protection strategy, this adaptive mechanism allows users to flexibly adjust the intensity of privacy protection according to actual tasks and requirements.

[0158] Step S507: The client decrypts the encrypted obfuscated image to obtain the obfuscated image.

[0159] That is, the client uses the private key to decrypt the encrypted obfuscated image to restore it to a processable plaintext image (i.e., the obfuscated image).

[0160] Step S407: The client randomly rearranges the order of the real image and the obfuscated image generated by the image generation tool to generate an image set.

[0161] Step S408: The client sends a second question-and-answer request generated based on the image set to the large model.

[0162] Step S409: The large model analyzes the image set and generates a second question-and-answer result.

[0163] Step S410: The client obtains the first question-and-answer result generated from the real image from the second question-and-answer result.

[0164] Step S411: The client returns the first question-and-answer result to the user.

[0165] In this example, the image generation tool is combined with the large model. So the user only needs to upload a facial photo, and the large model can use the image generation tool to generate a blurred image for the user. The large model will analyze all the received images and output the results, while the client only returns the results corresponding to the user's facial photo to the user. In this way, for the user, the process of protecting the privacy of the photo is imperceptible. Therefore, this method greatly improves the user experience. And in this method, the blurred image is encrypted and transmitted, so the user's privacy and security are still guaranteed. And this method can be applied to the user private image privacy protection framework for various scenarios and tasks, and is compatible with open-source and closed-source large models and third-party tools.

[0166] The following uses a specific example to further illustrate the privacy protection of the user by the method provided in this embodiment. The method based on mixing blurred images can be divided into two stages as shown in Figure 6 follows: The first stage: generating forged images based on attributes; the second stage: privacy protection requests based on blurred images.

[0167] In the first stage, the user hopes to obtain a blurred image to cover the privacy attributes in the real image. The specific operations are as follows:

[0168] 1. The user sends a request.

[0169] The user sends a request Q data to the LLM, which contains the requirement for generating encrypted forged images and the user's public key A. The goal of the request is to generate d encrypted input images to ensure that the user's privacy will not be leaked.

[0170] 2. The LLM parses the request Q data .

[0171] The LLM parses the request Q data and generates Task data . Among them, Task data includes generating blurred images similar to the user's privacy attributes. The LLM transmits Taskdata to the image generation tool.

[0172] 3. Generate d blurred images.

[0173] After receiving the task, the image generation tool will generate d images x j , where j ∈ d. These images are generated based on the user's privacy attributes to ensure that the generated blurred images are similar to the user's real images.

[0174] 4. Generate public key B and obtain the shared key s.

[0175] The image generation tool generates its own public key B and uses the DH key exchange protocol to calculate and generate a shared key s together with the user's public key A. This shared key s will be used to encrypt and decrypt the obfuscated image.

[0176] 5. Encrypt the generated obfuscated image.

[0177] The image generation tool uses the shared key s to encrypt the generated forged image x j to generate the encrypted obfuscated image , where j ∈ d. The encrypted obfuscated image can prevent the image data from being directly accessed or understood by the outside world. After encryption is completed, the image generation tool will send the public key B and the encrypted obfuscated image to the customer.

[0178] 6. The user obtains the shared key s.

[0179] The user obtains the shared key s through the DH protocol for secure communication with the image generation tool, enabling the user to decrypt the received image in the next step.

[0180] 7. The user decrypts the encrypted obfuscated image.

[0181] The user uses the shared key s to decrypt the encrypted image to recover the original image , where j ∈ d. At this point, the user obtains d obfuscated images similar to their private image.

[0182] In the second stage, the user's goal is to obtain the response of the private image without revealing the privacy attributes. The user submits the set of obfuscated images to the LLM, and the LLM and the task execution tool cannot identify which one is the user's private image, thus satisfying the t-closeness principle.

[0183] 8. Mix d + 1 images.

[0184] The user mixes all d decrypted obfuscated images and one real image of the user together to form d + 1 images x j .

[0185] 9. The user sends an obfuscated request Q', which includes the query q and d + 1 images x j .

[0186] The user randomly obfuscates the d + 1 images to generate a new request , where q represents the public query and x j represents the d + 1 mixed images. After completing the random obfuscation, the user sends the request Q' to the LLM.

[0187] 10. Parse the user request q into Task i , where i ∈ m.

[0188] The LLM parses the user's public query q and converts it into m specific tasks Task i , where i ∈ m. These tasks are related to the goals of the user request, such as involving image processing, analysis, etc.

[0189] 11. Generate the response template t.

[0190] The LLM generates a general response template t for each task to structure the task execution results. After generating the response template, the LLM sends the task Task i and d + 1 images X j .

[0191] 12. Execute task Taski to generate the encrypted response result , where i ∈ m, j ∈ d + 1. After completing the task execution, the task execution tool sends the result set to the LLM.

[0192] 13. Combine the response result with the response template.

[0193] The LLM combines the task response result and the response template t, and generates a complete response j for each image x .

[0194] 14. Select the response result of the real image.

[0195] The user selects the response result corresponding to their real image from the d + 1 response results. This can ensure that the user's real image is confused throughout the process, achieving the purpose of effectively protecting the privacy of the user's real image.

[0196] The security of this embodiment is analyzed below.

[0197] First is the isolation between the first stage and the second stage:

[0198] In the first stage, the image generation tool generates d confused images according to the tasks of the LLM, and encrypts these images and returns them to the user through the LLM. In the second stage, the user mixes one real image and d confused images, and sends these d + 1 images to the LLM. Since the images transmitted in the first stage are encrypted, and the user's real image and confused images are mixed in the second stage, neither the LLM nor the task execution tool can infer the user's private image through cross - analysis of the image sets in the two stages.

[0199] Then, the t-closeness principle is introduced for verification: The core of t-closeness is that given a sensitive attribute, in the released aggregated data, the distribution of this sensitive attribute must be close enough to the distribution of the sensitive attribute in the entire dataset. Specifically, t-closeness requires that the distance between the distribution of the sensitive attribute in each equivalence class and the distribution of the sensitive attribute in the entire dataset does not exceed the preset threshold t.

[0200] Here, WD is used to quantify the difference between the distribution of the sensitive attribute in each equivalence class and the distribution in the entire image set. In a set containing n images, the distribution of the sensitive features in the equivalence class and the distribution in the entire set are respectively:

[0201] (1);

[0202] (2);

[0203] Among them, P v is the weight associated with class v in the distribution, q w is the weight associated with class w in the Q distribution, and the WD between P and Q is expressed as:

[0204] (3);

[0205] The WD expression satisfies the following constraint conditions:

[0206] ① ;

[0207] ② ;

[0208] ③ ;

[0209] Among them, d vw is the distance between class v and class w, f vw is the flow variable, and f vw represents the probability mass transferred from class v in distribution P to class w in distribution Q; According to the definition of t-closeness, if WD(P, Q) does not exceed the preset threshold t, it can be considered that the distribution of the sensitive features in this equivalence class is close enough to the distribution in the entire image set, thus preventing accidental leakage of sensitive features.

[0210] According to the definition of privacy attributes in the image, these attributes are all categorical attributes, and each category value is independent of each other. Therefore, an equidistant metric is used here to measure the distance between two distributions. In this metric, the distance between any two values in the categorical attribute is defined as 1, so the WD expression is converted to:

[0211] (4).

[0212] By controlling the distribution of sensitive features in the image set, it is ensured that even if some images are in special positions in the set, their sensitive information will not deviate significantly from the overall distribution, thus effectively protecting user privacy.

[0213] In the scheme based on mixing obfuscated images, even if the LLM and the specific task execution tool know that the user's private images are in the image set they cannot directly infer the user's real images due to the mixing obfuscated image mechanism and the encryption scheme. By measuring the difference between the WD measurement equivalence class and the overall distribution, it is ensured that each equivalence class satisfies t-closeness. Even if the image contains user-sensitive features, the distribution of this feature in the equivalence class will not deviate significantly from the overall feature distribution, ensuring user privacy security.

[0214] The following introduces the specific scenarios to which the above method based on mixing obfuscated images can be applied. When the large model is a medical and health large model, the specific implementation steps are as follows:

[0215] (1) Extract key attributes.

[0216] The user first uploads their facial photo to the local system. The system extracts the key attributes in the photo, such as skin type, facial features, gender, and age group, etc., and sends the extracted image key attribute information to the large model.

[0217] (2) Generate text prompts.

[0218] The large model generates text prompt content based on the image key attributes and sends the prompt content to the image generation tool.

[0219] (3) Generate obfuscated images.

[0220] The image generation tool generates several facial obfuscated images according to the above text prompt information and encrypts the generated images. The generated images have similar facial attributes to the user's real photo, but have sufficient differences in appearance and features to avoid direct association with the user himself.

[0221] (4) Image set construction.

[0222] The local system receives and decrypts the encrypted obfuscated image set returned by the image generation tool. Then, it combines the user's real photo and the generated obfuscated images into an image set and randomizes their order to prevent the user's real photo from being recognized by the large model.

[0223] (5) Privacy protection evaluation.

[0224] During each interaction, the local system calculates the privacy metric through t-closeness, which can evaluate to what extent the generated set of obfuscated images enhances the security of user privacy. If the enhancement degree is lower than the pre-set threshold T, the operation of generating obfuscated images needs to be performed again.

[0225] (6) Submit an obfuscated image request.

[0226] The user submits the image set mixed with obfuscated images and their health query request to the medical and health large model. The request content includes the information of "conduct skin condition analysis on these facial photos".

[0227] (7) Dataset analysis task.

[0228] After receiving the image set, the medical and health large model will analyze each image and return the relevant results. Since the received image set is randomly obfuscated, the model cannot identify which one is the user's real photo, preventing the large model from further inferring the user's health privacy information.

[0229] (8) Obtain the analysis results.

[0230] The local system will screen the analysis results of the image set returned by the large model, select the analysis results of the user's real photos, and filter out the analysis results of the obfuscated images. In this way, the user only obtains the analysis information related to their real photos.

[0231] Through the privacy protection scheme based on obfuscated images, when the user interacts with the medical and health large model for images, the user's real photos can be effectively hidden, preventing the large model from inferring the user's health privacy information, and without affecting the accuracy of the system to provide diagnosis.

[0232] The embodiment of the present application also provides a method for image privacy protection during the use of a large model, and the method includes the following steps:

[0233] Receive a first question-and-answer request from the user, and obtain a real image from the first question-and-answer request; the first question-and-answer request is used to request to generate a first question-and-answer result for the real image;

[0234] Extract features from the real image to obtain the real attribute information of the user, and the real attribute information is used by the image generation tool to generate obfuscated images;

[0235] Randomly rearrange the order of the real image and the obfuscated images generated by the image generation tool to generate an image set; generate a second question-and-answer request based on the image set; the second question-and-answer request is used to request to generate a second question-and-answer result for the image set;

[0236] Send the second question-and-answer request to the large model and receive the second question-and-answer result returned by the large model;

[0237] Return the first Q&A result generated for the real image in the second Q&A result to the user.

[0238] This method can be executed in an electronic device, a computer, or a similar computing system. Taking running on an electronic device as an example, Figure 7 The following shows a hardware structure block diagram of an electronic device provided by an embodiment of the present application. As Figure 7 shown, the electronic device may include one or more ( Figure 7 only one is shown in the figure) processors 702, and a memory 704 for storing data. The processor 702 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above-mentioned electronic device may further include a transmission device 706 for communication functions and an input / output device 708. Those of ordinary skill in the art can understand that Figure 2 the structure shown is only schematic, and it does not limit the structure of the above-mentioned electronic device. For example, the electronic device may further include more or fewer components than Figure 2 shown in the figure, or have a different configuration from Figure 2 shown in the figure.

[0239] The memory 704 can be used to store computer programs, for example, software programs and modules of application software. The processor 702 executes various functional applications and data processing by running the computer programs stored in the memory 704, that is, the above-mentioned method is implemented. The memory 704 can be used to store data, for example, real images, real attribute information, etc. The memory 704 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 704 may further include a memory remotely set relative to the processor 702, and these remote memories can be connected to the electronic device through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0240] The transmission device 706 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of the electronic device. In one instance, the transmission device 706 includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 706 may be a radio frequency (abbreviated as RF) module, which is used to communicate with the Internet wirelessly.

[0241] The embodiments of the present application further provide a method for image privacy protection during the use of large models, which is applied to a server. The server is deployed with a large model and an image generation tool. In this method, the server performs the following steps:

[0242] Receive the real attribute information of the target user sent by the client. The real attribute information is obtained by the client extracting features from the real image of the target user;

[0243] Through the image generation tool, perform image generation processing on the real attribute information to obtain a confused image, and transmit the confused image to the client;

[0244] Receive the second question-and-answer request sent by the client, and obtain an image set from the second question-and-answer request. The image set includes the real image and the confused image whose order is randomly rearranged;

[0245] Analyze the image set through the large model to generate a second question-and-answer result.

[0246] Figure 8 The structural block diagram of a device for image privacy protection during the use of large models provided by the embodiments of the present application is shown, as Figure 8 shown, the device includes:

[0247] A data receiving module 801, configured to receive a first question-and-answer request from a user, and obtain a real image from the first question-and-answer request; the first question-and-answer request is used to request to generate a first question-and-answer result for the real image;

[0248] A feature extraction module 802, configured to extract features from the real image to obtain the real attribute information of the user, and the real attribute information is used by the image generation tool to generate a confused image;

[0249] An image confusion module 803, configured to randomly rearrange the order of the real image and the confused image generated by the image generation tool to generate an image set; generate a second question-and-answer request based on the image set; the second question-and-answer request is used to request to generate a second question-and-answer result for the image set;

[0250] A request sending module 804, configured to send the second question-and-answer request to the large model and receive the second question-and-answer result returned by the large model;

[0251] A question-and-answer result screening module 805, configured to return the first question-and-answer result generated for the real image in the second question-and-answer result to the user.

[0252] It should be noted that the above-mentioned modules can be functional modules or program modules, and can be implemented either by software or by hardware. For the modules implemented by hardware, the above-mentioned modules can be located in the same processor; or the above-mentioned modules can also be located in different processors in any combined form.

[0253] It should be noted that the specific examples in this embodiment can refer to the examples described in the above-mentioned embodiment and optional implementation manners, and will not be elaborated in this embodiment.

[0254] In addition, in combination with the method provided in the above-mentioned embodiment, a storage medium can also be provided in this embodiment to implement it. A computer program is stored on the storage medium; when the computer program is executed by a processor, any one of the methods for image privacy protection during the use of the large model in the above-mentioned embodiment is implemented.

[0255] The embodiment of the present application also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute each function or step executed by the processor in the above-mentioned method embodiment.

[0256] It should be understood that the specific embodiments described here are only used to explain this application, rather than to limit it. According to the embodiments provided by the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the protection scope of the present application.

[0257] Obviously, the drawings are only some examples or embodiments of the present application. For those of ordinary skill in the art, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it can be understood that although the work done during the development process here may be complex and time-consuming, for those of ordinary skill in the art, some design, manufacturing or production changes based on the technical content disclosed in the present application are only conventional technical means and should not be regarded as insufficient disclosure of the present application.

[0258] The term "embodiment" in the present application means that the specific features, structures or characteristics described in combination with the embodiment may be included in at least one embodiment of the present application. The phrase appears in various positions in the specification does not necessarily mean the same embodiment, nor does it mean independence or alternative to other embodiments. Those of ordinary skill in the art can clearly or implicitly understand that the embodiments described in the present application can be combined with other embodiments without conflict.

[0259] The above embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of patent protection. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A method for protecting image privacy during the use of a large model, characterized in that: Applied to a client, the method comprises: receiving a first question-and-answer request from a user, and obtaining a real image from the first question-and-answer request; the first question-and-answer request is used to request generation of a first question-and-answer result for the real image; extracting real attribute information from the real image; Using the real attribute information, generating a confused image includes the following steps: Sending the real attribute information to the large model; wherein the real attribute information is converted by the large model into text prompt content that can be recognized by the image generation tool and sent to the image generation tool; Receiving the encrypted obfuscated image generated by the image generation tool based on the text prompt content, and performing decryption processing to obtain the decrypted obfuscated image; wherein the encrypted obfuscated image is sent to the large model by the image generation tool, and then sent to the client through the large model; Randomly rearrange the order of the real image and the obfuscated image to generate an image set; generate a second question-answering request based on the image set; the second question-answering request is used to request generation of a second question-answering result for the image set; Sending the second question-and-answer request to the large model, and receiving a second question-and-answer result returned by the large model; The first question-and-answer result generated for the real image in the second question-and-answer result is returned to the user.

2. The method for protecting image privacy during the use of large models according to claim 1, characterized in that: Before receiving the encrypted obfuscated image generated by the image generation tool based on the text prompt content, the method further includes: Sending an image generation request to the large model, the image generation request including request information and a first public key; wherein the first public key is used by the image generation tool to combine with a second public key generated by the image generation tool to obtain a shared key; The shared key is obtained based on a key exchange protocol, and the shared key is used by the client to decrypt the encrypted obfuscated image to obtain the decrypted obfuscated image.

3. The method for protecting image privacy during the use of large models according to claim 1, characterized in that: The extracting real attribute information from the real image comprises: Feature extraction is performed on the user's uploaded photos to obtain picture feature information.

4. The method for protecting image privacy during the use of large models according to claim 1, characterized in that: After randomly rearranging the order of the real image and the confused image to generate an image set, the method further includes: converting the image collection into a data table, and determining sensitive attributes of the user from the data table; Calculate the difference between the distribution of sensitive attributes in each equivalence class and the distribution of sensitive attributes in the data table; wherein one equivalence class corresponds to data converted from an image, and the image includes the real image and the obfuscated image; When the difference value is greater than a preset threshold, the generated quantity and attribute difference when the image generation tool generates a confused image next time are adjusted.

5. The method for protecting image privacy during use of a large model according to claim 4, characterized in that: The calculating the difference between the distribution of sensitive attributes in each equivalence class and the distribution of sensitive attributes in the data table includes: WD is used to represent the difference value. In a set of n images, the distribution of sensitive features in the equivalence class and the distribution in the entire set are expressed as: ； ； Among them, P v is the weight associated with class v in the distribution, q w is the weight associated with category w in the Q distribution, and the WD between P and Q is expressed as: ； The WD expression satisfies the following constraints: ； ； ； Among them, d vw is the distance between category v and category w, f vw is the flow variable, f vw represents the probability mass of transferring from category v in distribution P to category w in distribution Q; Convert the WD expression to: 。 6. A method for protecting image privacy during the use of a large model, characterized in that: Applied to a server, the server is deployed with a large model and an image generation tool, the method comprises: receiving real attribute information for a target user sent by a client, the real attribute information being extracted by the client from a real image of the target user; and converting the real attribute information into text prompt content that can be recognized by the image generation tool through the large model; The method further comprises: performing image generation processing on the real attribute information through the image generation tool to obtain an obfuscated image, and transmitting the obfuscated image to the client, including: Performing image generation processing based on the text prompt content by the image generation tool to obtain a confused image; The obfuscated image is encrypted by the image generation tool to obtain an encrypted obfuscated image; after the encrypted obfuscated image is sent to the large model by the image generation tool, it is sent to the client through the large model; receiving a second question-and-answer request sent by the client, and obtaining an image set from the second question-and-answer request, wherein the image set includes the real image and the obfuscated image after random rearrangement of their order; The image set is analyzed by the large model to generate a second question-answering result.

7. The method for protecting image privacy during use of a large model according to claim 6, characterized in that: The real image is a photo uploaded by the user; The step of analyzing the image set by using the large model to generate a second question-answering result includes: The uploaded photos and the confused images in the image collection are analyzed respectively by the large model to generate a second question and answer result; wherein the second question and answer result includes the first question and answer result, and the first question and answer result is the reply text obtained by the large model through analyzing the uploaded photos of the user.

8. A system for protecting image privacy during the use of large models, characterized in that: The system includes a client and a server, wherein the client is configured to execute the method for protecting image privacy during the use of a large model as described in any one of claims 1 to 5, and the server is configured to execute the method for protecting image privacy during the use of a large model as described in any one of claims 6 to 7.

Images