Method and system for image privacy protection in use process of large model

Abstract

The invention discloses an image privacy protection method and system in a large model use process, and relates to the technical field of large models, in the method, a client does not independently send a real image to a large model, but mixes a confused image in the real image, and the large model processes the real image and the confused image, so that the image privacy protection efficiency is improved. And the client receives the processing result and only returns the part, aiming at the real image, in the processing result to the user. Based on a solution that a confused image is mixed in a real image, image privacy is protected from being leaked when a user answers and answers by means of a large model dialogue.

Description

Technical Field

[0001] The present application relates to the field of large model technology, and in particular to a method and system for protecting image privacy during the use of large models. Background Art

[0002] In recent years, breakthroughs in deep learning technology have promoted the significant development of large language models (LLMs) in the field of natural language processing. Large language models (or large models) not only perform well in tasks such as text generation and question-answering systems, but are also gradually combined with various auxiliary tools to form large model agents that integrate enhanced tools. When using large model agents with integrated tools, there is a scenario where users upload private images. In this scenario, the large model and the third-party tools it calls can use visual clues in the image to infer the user's identity, location, interests, intentions and other private information. And advanced analysis techniques can be used to dig deeper into the user's privacy. The privacy risks caused by the use of users' private images are attracting much attention, and effective solutions are urgently needed to protect users' privacy information.

[0003] At present, the privacy protection schemes for submitting private images in the process of using large models mainly include local deployment, anonymous communication, and data masking. However, these methods have limitations in actually protecting user privacy.

[0004] Local deployment method: Deploying large open source models and tools locally can ensure that data does not leave the local execution environment, thereby achieving the purpose of privacy protection. However, this method requires users to have a high level of expertise and hardware resources, and can only use large open source models and tools, resulting in the inability to use many powerful closed source models and tools.

[0005] Anonymous communication method: Anonymous communication can hide the identities of both parties and prevent third parties from tracking or identifying the participants. However, anonymous communication does not provide complete privacy protection, and Internet Protocol (IP) addresses may still be tracked.

[0006] Data masking method: Data masking partially covers or blurs private images submitted by users to ensure that private information is not exposed during image transmission and processing. However, data masking will affect the processing effect of images, especially in tasks that require high-precision image analysis.

[0007] In summary, there is an urgent need to find a new solution to the privacy protection issue when users submit images during interaction with large models. Summary of the invention

[0008] In an embodiment of the present application, a method and system for protecting image privacy during the use of a large model are provided, which is based on a solution of mixing obfuscated images into real images to protect the image privacy of users from being leaked when they are having conversations and asking questions with the help of a large model.

[0009] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions: In a first aspect, an embodiment of the present application provides a method for protecting image privacy during the use of a large model, which is applied to a client, and the method includes: receiving a first question-and-answer request from a user, and obtaining a real image from the first question-and-answer request; the first question-and-answer request is used to request generation of a first question-and-answer result for the real image; extracting real attribute information from the real image; Using the real attribute information, generating a confused image; Randomly rearrange the order of the real image and the obfuscated image to generate an image set; generate a second question-answering request based on the image set; the second question-answering request is used to request generation of a second question-answering result for the image set; Sending the second question-and-answer request to the large model, and receiving a second question-and-answer result returned by the large model; The first question-and-answer result generated for the real image in the second question-and-answer result is returned to the user.

[0010] In combination with the first aspect, in a possible design manner, the generating of the obfuscated image by using the real attribute information includes the following steps: Sending the real attribute information to the large model; wherein the real attribute information is converted by the large model into text prompt content that can be recognized by the image generation tool and sent to the image generation tool; Receive the encrypted obfuscated image generated by the image generation tool based on the text prompt content, and perform decryption processing to obtain the decrypted obfuscated image; wherein the encrypted obfuscated image is sent to the large model by the image generation tool and then sent to the client through the large model.

[0011] In combination with the first aspect, in a possible design manner, before receiving the encrypted obfuscated image generated by the image generation tool based on the text prompt content, the method further includes: Sending an image generation request to the large model, the image generation request including request information and a first public key; wherein the first public key is used by the image generation tool to combine with a second public key generated by the image generation tool to obtain a shared key; The shared key is obtained based on a key exchange protocol, and the shared key is used by the client to decrypt the encrypted obfuscated image to obtain the decrypted obfuscated image.

[0012] In combination with the first aspect, in a possible design manner, extracting real attribute information from a real image includes: Feature extraction is performed on the user's uploaded photos to obtain picture feature information.

[0013] In combination with the first aspect, in a possible design manner, after randomly rearranging the order of the real image and the confused image to generate an image set, the method further includes: converting the image collection into a data table, and determining sensitive attributes of the user from the data table; Calculate the difference between the distribution of sensitive attributes in each equivalence class and the distribution of sensitive attributes in the data table; wherein one equivalence class corresponds to data converted from an image, and the image includes the real image and the obfuscated image; When the difference value is greater than a preset threshold, the generated quantity and attribute difference when the image generation tool generates a confused image next time are adjusted.

[0014] In combination with the first aspect, in a possible design manner, the calculating the difference between the distribution of the sensitive attributes in each equivalence class and the distribution of the sensitive attributes in the data table includes: WD is used to represent the difference value. In a set of n images, the distribution of sensitive features in the equivalence class and the distribution in the entire set are expressed as: ; ; Among them, P v is the weight associated with class v in the distribution, q w is the weight associated with category w in the Q distribution, and the WD between P and Q is expressed as: ; The WD expression satisfies the following constraints: ; ; ; Among them, d vw is the distance between category v and category w, f vw is the flow variable, f vw represents the probability mass of transferring from category v in distribution P to category w in distribution Q; Convert the WD expression to: .

[0015] In a second aspect, an embodiment of the present application provides a method for protecting image privacy during the use of a large model, which is applied to a server, wherein the server is deployed with a large model and an image generation tool, and the method includes: Receiving real attribute information for a target user sent by a client, where the real attribute information is extracted by the client from a real image of the target user; By using the image generation tool, performing image generation processing on the real attribute information to obtain an obfuscated image, and transmitting the obfuscated image to the client; receiving a second question-and-answer request sent by the client, and obtaining an image set from the second question-and-answer request, wherein the image set includes the real image and the obfuscated image after random rearrangement of their order; The image set is analyzed by the large model to generate a second question-answering result.

[0016] In conjunction with the second aspect, in a possible design manner, after receiving the real attribute information for the target user sent by the client, the method further includes: By means of the large model, the real attribute information is converted into text prompt content that can be recognized by the image generation tool; The step of performing image generation processing based on the real attribute information by the image generation tool to obtain a confused image, and transmitting the confused image to the client includes: Performing image generation processing based on the text prompt content by the image generation tool to obtain a confused image; The obfuscated image is encrypted by the image generation tool to obtain an encrypted obfuscated image; after the encrypted obfuscated image is sent to the large model by the image generation tool, it is sent to the client through the large model.

[0017] In combination with the second aspect, in a possible design, the real image is a photo uploaded by the user; The step of analyzing the image set by using the large model to generate a second question-answering result includes: The uploaded photos and the confused images in the image collection are analyzed respectively by the large model to generate a second question and answer result; wherein the second question and answer result includes the first question and answer result, and the first question and answer result is the reply text obtained by the large model through analyzing the uploaded photos of the user.

[0018] In a third aspect, an embodiment of the present application provides a computer device, comprising a processor and a memory, wherein the processor is configured to run a computer program in the memory to execute the method of the first aspect and possible design methods thereof.

[0019] In a fourth aspect, an embodiment of the present application provides a server, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method of the second aspect and possible design methods thereof.

[0020] In the fifth aspect, an embodiment of the present application provides a system for protecting image privacy during the use of large models, including a client and a server. The client relies on a computer device to implement functions and is configured as the method of the first aspect and its possible design methods. The server is configured to execute the method of the second aspect and its possible design methods.

[0021] In a sixth aspect, an embodiment of the present application provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the method of the first aspect and its possible design method, or execute the method of the second aspect and its possible design method when running.

[0022] Compared with the prior art, the embodiment of the present application provides a method and system for image privacy protection during the use of a large model. In the method, the client receives a first question-and-answer request from the user, obtains a real image from the first question-and-answer request; the first question-and-answer request is used to request the generation of a first question-and-answer result for the real image. The client obtains real attribute information from the real image, and then generates a confused image using the real attribute information. The client then randomly rearranges the order of the real image and the confused image to generate an image set; generates a second question-and-answer request based on the image set; the second question-and-answer request is used to request the generation of a second question-and-answer result for the image set. Afterwards, the client sends the second question-and-answer request to the large model, receives the second question-and-answer result returned by the large model, and returns the first question-and-answer result generated for the real image in the second question-and-answer result to the user. By generating a confused image and mixing it with the real image, it can effectively increase the difficulty for an attacker to identify the real image. The large model uniformly processes all images without knowing which image is the real image, which can avoid directly exposing the user's real image and ensure the realization of the service function. This not only improves the security of the system, but also ensures the quality and efficiency of the service.

[0023] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A schematic diagram of a system for user interaction with a large model agent provided by an embodiment of the present application is shown; Figure 2 A schematic diagram of a method for protecting image privacy during use of a large model provided by an embodiment of the present application is shown; Figure 3 A schematic diagram of a system for protecting image privacy during use of a large model provided by an embodiment of the present application is shown; Figure 4 A flowchart of a method for protecting image privacy during the use of a large model provided by an embodiment of the present application is shown; Figure 5 A flowchart of another method for protecting image privacy during the use of a large model provided by an embodiment of the present application is shown; Figure 6 A schematic diagram of a method for mixing an obfuscated image provided by an embodiment of the present application is shown; Figure 7 A hardware structure block diagram of an electronic device provided in an embodiment of the present application is shown; Figure 8 A structural block diagram of a device for protecting image privacy during the use of a large model provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0026] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and apparatus, product or equipment comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or equipment. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0027] First, the terms that may be involved in the embodiments of the present application are explained: DH (Diffie-Hellman key exchange protocol): is a security protocol that ensures that both communicating parties can exchange keys securely in an insecure channel.

[0028] WD (Wasserstein Distance): A distance measurement method based on probability distribution, which describes the minimum cost required to transform one probability distribution into another probability distribution. It is often used in probability distribution, image processing, machine learning and other fields.

[0029] Text-to-image generative model: a machine learning-based model that generates images that match the input natural language descriptions. Usually, this type of model includes a language model and a generative model. The language model converts the natural language input description into vectors in the latent space, and the generative model uses these text vectors to generate the corresponding images. For example, the image generation tool mentioned in this application can generate obfuscated images through this model.

[0030] Key exchange protocol: often used in insecure communication environments, allowing multiple parties to securely establish a shared key. The shared key can be used to encrypt and decrypt information, ensuring the confidentiality, integrity and authenticity of the message during the communication process. The key exchange protocol allows each party to establish a shared key without knowing the other party's key in advance.

[0031] t-closeness principle: It is a privacy protection principle to prevent the leakage of sensitive attributes. According to this principle, the attributes in the data record can be divided into three categories: Explicit identifiers: Attributes that can directly and uniquely identify an individual.

[0032] Quasi-identifiers: cannot uniquely identify an individual when used alone, but may narrow the scope of the individual when combined with other quasi-identifiers.

[0033] Sensitive attributes: Attributes that are generally considered private or sensitive.

[0034] The t-closeness principle requires that the distribution of sensitive attributes in each equivalence class (a set of records with the same quasi-identifier) ​​should be as close as possible to the overall distribution of sensitive attributes in the entire dataset. Specifically, the gap between the distribution of sensitive attributes in an equivalence class and the distribution of sensitive attributes in the dataset cannot exceed a set threshold t.

[0035] First, the process of interaction between the user and the large model agent is described. Figure 1 , Figure 1 A schematic diagram of a system for user interaction with a large model agent provided by an embodiment of the present application is shown, and the system includes a user 101, a large model 102, and a task execution tool 103. In the first step, the user request Q={q, x} includes a public query q and a private image input x, where the sensitive information contained in x is recorded as p(x). User 101 hopes to obtain an answer through the large model 102, and avoid the large model 102 and the task execution tool 103 from inferring the association between the sensitive information p(x) and the user 101.

[0036] The big model 102 acts as a controller in the system, responsible for planning tasks and selecting appropriate task execution tools 103 to complete different tasks. The task execution tool 103 executes the tasks assigned by the big model 102 according to the input parameter param. There are two-way communication channels between the user 101 and the big model 102, and between the big model 102 and the task execution tool 103. The user 101 sends a request Q to the big model 102 and expects to get a correct response. After receiving the user request, the big model 102 executes the second step, parsing the request Q and decomposing it into multiple subtasks to form a task set task. i={key, target, param}. The large model 102 plans the task sequence and dependencies, and then generates a response template t based on all tasks.

[0037] In the third step, the large model 102 assigns the parsed task to the task execution tool 103. The task execution tool 103 executes the assigned task according to the input parameter param and sends the prediction result to Return to the large model 102.

[0038] In the fourth step, the large model 102 fills the received results into the template t, generates the final response result T, and returns it to the user 101.

[0039] Step 5: User 101 obtains the response result.

[0040] from Figure 1 It can be seen that when a user interacts with the big model, the user needs to submit public queries and private images (equivalent to the real image in the embodiment of the present application) to the big model. The big model then arranges tasks and calls related enhancement tools to perform tasks. In this process, the big model obtains the user's private image input and shares this information with related tools during task execution, which causes the risk of user privacy information exposure. For example, when a user provides a private image when interacting with the big model agent, the big model shares the image with a third-party tool according to business needs, and the big model and the tool can infer the user's privacy information through analysis.

[0041] Although some privacy protection schemes have been proposed for submitting private images when using large models, each method has its shortcomings: local deployment requires users to have high technical capabilities and hardware resources, and is limited to using open source models and tools; anonymous communication cannot provide complete privacy protection; data masking methods will affect the processing effect of images.

[0042] In order to solve the privacy protection problem when submitting images during the interaction between users and large models, this application embodiment provides a new solution, please refer to Figure 2 , Figure 2A schematic diagram of a method for protecting image privacy during the use of a large model provided by an embodiment of the present application is shown. In the method, the client does not send the real image alone to the large model, but mixes the obfuscated image into the real image. The large model processes both the real image and the obfuscated image. The client receives the processing result and only returns the part of the processing result for the real image to the user. Based on the solution of mixing the obfuscated image into the real image, the image privacy of the user is protected from being leaked when he or she is having a conversation with the large model. In this embodiment, the client can generate the obfuscated image with the help of an image generation tool. As another implementation method, the client can execute the steps of the image generation tool, that is, the client generates the obfuscated image based on the real attribute information.

[0043] The following is an example of the method provided in the embodiment of the present application being applied to skin condition analysis. It should be understood that the examples given in the embodiment of the present application are not limitations on the scope of patent protection. The method provided in the embodiment of the present application can be applied to various scenarios such as medical and health care, financial data desensitization, and smart security. The purpose is to protect the information in the real image that can reveal the real portrait of the user, and prevent the big model and its tools from digging deeper into the user's privacy based on this information. Taking the medical and health field as an example, this method can be used to protect patient privacy. For example, when analyzing the skin state, the user uploads the skin image, and the big data analyzes it. During this process, the system generates an obfuscated image and encrypts it for transmission, ensuring that the big model and its tools can accurately diagnose and cannot identify the patient's identity. For another example, in the financial industry, this method can be used to process sensitive image data (user's ID card image) so that it is not leaked. For another example, applied to the field of smart security, this method can hide the residential location of pedestrians to prevent the big model from associating the residential location of pedestrians with the pedestrian's identity information and inferring the pedestrian's movement trajectory.

[0044] The method provided in the embodiment of the present application is applied to a system for protecting image privacy during the use of a large model, such as Figure 3 As shown, the system includes a client 31 and a server 32, and a large model and image generation tool 33 are deployed on the server.

[0045] The client provides a user-oriented interface and is responsible for receiving real images uploaded by users and the first question-and-answer request. The client refers to a program that provides local services to customers. The client can be software installed in a computer device, such as a mobile phone or a computer. The client can be a browser, an email client, or a chat application. The large model is responsible for analyzing the image and generating the question-and-answer result. The image generation tool is configured to generate a confused image based on the feature information.

[0046] In some embodiments, the client communicates bidirectionally with the large model and the image generation tool. In the embodiments, please refer to Figure 4 , Figure 4 A flowchart of a method for protecting image privacy during use of a large model provided by an embodiment of the present application is shown. The method includes steps S401 to S411.

[0047] Step S401: The client receives a first question-and-answer request from a user.

[0048] In this embodiment, step S401 refers to the client receiving a question-and-answer request initiated by a user, the request including the user's question-and-answer intention and related information. Specifically, the first question-and-answer request includes request information, and the first question-and-answer request is used to request the generation of a first question-and-answer result for a real image.

[0049] In actual applications, the user initiates the first question-and-answer request to the client through the interactive interface, and the client receives the first question-and-answer request initiated by the user, which includes the real image uploaded by the user and the associated semantic instructions. The real image refers to the original image file provided by the user containing personal biometrics or sensitive information, and the associated semantic instruction may correspond to "analyze skin condition". The response that the user wants to get is the question-and-answer result after analyzing the real image (i.e., the first question-and-answer result).

[0050] The first question-and-answer request is explained below in conjunction with the application scenario. In the scenario of skin condition analysis and processing, the first question-and-answer request may be a facial photo uploaded by the user, with the instruction "Please analyze my skin condition". In this example, the user requests an analysis of his or her facial skin condition and uploads a facial photo of the user. The facial photo refers to a real image. The client receives a request from the user to analyze his or her facial skin condition.

[0051] Step S402: The client obtains a real image from the first question-and-answer request.

[0052] After receiving the first question-and-answer request from the user, the client needs to extract the real image uploaded by the user. As an example, the user can upload the real image to the client so that the client can obtain the data.

[0053] After step S402, the client does not directly send the real image to the large model for analysis, but extracts the attribute information in the real image and sends the attribute information to the large model, so that the large model cannot restore the user's real image based on the attribute information. See step S403 for details.

[0054] Step S403: The client extracts real attribute information from the real image.

[0055] In this step, the client conducts in-depth analysis on the acquired real image and extracts the real attribute information contained therein, including information on the face, limbs, torso, skin, geographic location, etc. that can reveal the real image of the user.

[0056] Step S403 may further include: extracting features from the user's uploaded photos to obtain picture feature information. The uploaded photos may be body photos, landscape photos, house photos, etc. The body photos may be facial photos, hand photos, leg photos, etc. When the body photos are facial photos, the real attribute information includes one or more of skin type, facial features, gender, and age, which are described one by one below.

[0057] Skin type: Extract skin color, texture and other features from the image to determine whether the user's skin is dry, oily or mixed; extract skin lesion features from the image to determine the symptoms of the user's skin; extract facial hair follicle status features from the image to determine whether the user has inflammation or skin damage.

[0058] Facial features: Extract features such as the user's face shape (such as round, long), the position and shape of facial features (such as eye size, nose bridge height), etc.

[0059] Gender: Infer the user's gender based on factors such as facial contour and eyebrow shape as the user's gender characteristics.

[0060] Age: The user’s age group is estimated based on wrinkle distribution, skin elasticity and other features as the user’s age feature.

[0061] One or more of the above features are used to subsequently generate obfuscated images. It can be understood that since the real attribute information is extracted from the real image but is not completely identical to the real image, this data not only helps to generate obfuscated images that are similar to but not completely the same as the real image, but also ensures that the obfuscated image maintains consistency in some key attributes, thereby protecting user privacy.

[0062] In some of the embodiments, the client performs multi-dimensional analysis on the real image through a pre-trained feature extraction model and outputs real attribute information. For example, the U-Net segmentation model is used to locate the skin area and calculate the pore density and erythema index; the FaceNet model is used to generate an embedded vector, which contains geometric features such as facial features proportions and contour curvature.

[0063] Step S404: The client sends real attribute information to the image generation tool.

[0064] After completing the feature extraction, the client will send the extracted real attribute information to the image generation tool that is specifically responsible for generating the obfuscated image. The image generation tool can be deployed on the client side or on the server side. In this embodiment, the image generation tool and the large model do not communicate with each other, so the client interacts directly with the image generation tool. For example, in the above example, the client may package a series of feature information such as "dry skin", "round face", "female", "30 years old" into a data packet, and then pass it to the image generation tool via network transmission.

[0065] Step S405: The image generation tool performs image generation processing on the real attribute information to obtain a confused image.

[0066] The real attribute information is used by the image generation tool to generate an obfuscated image; that is, after the image generation tool receives the real attribute information sent by the client, it will use an algorithm to process it and generate a corresponding obfuscated image. The number of obfuscated images can be one or more.

[0067] Continuing with the skin condition analysis as an example, assuming that the feature extraction obtains the user's skin type as "dry", facial features as "round face", gender as "female", and age as "30 years old", then the image generation tool may generate the following types of confusing images.

[0068] Confused Image A: A photo of a face with dry skin features but slightly different facial features, with red spots on the face.

[0069] Confusion image B: A facial photo of a person with a round face but a different skin type (e.g., oily or combination) and dermatitis on the face.

[0070] Although obfuscated image A and obfuscated image B are different from the real image in some features, they maintain a high correlation with the real image in key attributes (such as facial pathological features). This can increase the difficulty for large models or image generation tools to identify which image is the user's real image, thereby achieving the purpose of protecting user privacy.

[0071] Step S406: The client receives the obfuscated image from the image generation tool.

[0072] In this embodiment, the client can communicate with the image generation tool, so the obfuscated image generated by the image generation tool will not be obtained by the large model, and the large model cannot infer which image is the user's real image and which image is the obfuscated image generated by the image generation tool. Therefore, in this embodiment, the obfuscated image can be unencrypted. Of course, in order to improve transmission security, the obfuscated image can be encrypted before transmission to prevent attackers from intercepting the obfuscated image and analyzing the user's sensitive information.

[0073] The number of obfuscated images may be one or more. As an example, the image generation tool generates five obfuscated images at one time and packages them into a folder or a compressed package and transmits them back to the client.

[0074] Step S407: The client randomly rearranges the order of the real image and the obfuscated image generated by the image generation tool to generate an image set.

[0075] In this step, the client mixes the real image obtained in step S402 and the newly generated obfuscated image by rearranging the order of the images according to a random rule to form a new image set. In this way, the large model is prevented from inferring the position of the real image in the image set through the orderliness of the image arrangement.

[0076] The mixing method is explained below: If the real image is a photo of a user's face, and the obfuscated images contain four photos of different types, the client can number these five photos as A, B, C, D, and E, and then use a random number generator to shuffle their order, for example, the new order is C, A, E, B, D.

[0077] This step further increases the difficulty for attackers to identify real images by randomly rearranging the order of real images and obfuscated images, thereby improving the security of privacy protection.

[0078] Step S408: The client sends a second question-answering request generated based on the image set to the large model.

[0079] After completing the construction of the image collection, the client will generate a second question-answering request based on it. The second question-answering request contains the user's real image and the generated obfuscated image. The second question-answering request is used to request the large model to analyze the real image and the obfuscated image respectively and generate analysis results.

[0080] Continuing with the skin condition analysis example, the second question-answering request may be “Please perform skin condition analysis on this image set containing multiple facial photos.” This request requires the large model to perform corresponding analysis on each image.

[0081] This step generates a second question-answering request and passes the image set to the big model, so that the big model can process all images uniformly without knowing which image is real. This ensures the realization of the big model service function and avoids directly exposing the user's real image.

[0082] Step S409: The large model analyzes the image set and generates a second question-and-answer result.

[0083] After the large model receives the second question-and-answer request and the accompanying image set from the client, it will analyze and process each image one by one and give the corresponding analysis results.

[0084] Continuing with the example of skin condition analysis, the large model will evaluate the skin condition of each image in the image collection and come up with answers such as "The skin condition shown in this image is dry with slight wrinkles" or "You have dermatitis and need to see a doctor as soon as possible."

[0085] Since neither the real image nor the obfuscated image has been transmitted to the big model separately, the big model cannot analyze which image is the user's real image from the image collection. Therefore, the big model cannot use the visual clues in the image to infer the user's identity, location, interests, intentions and other private information.

[0086] Step S410: The client obtains the first question-and-answer result generated by the real image from the second question-and-answer result.

[0087] After receiving the second question-and-answer result returned by the large model, the client finds the part corresponding to the original real image, that is, the first question-and-answer result.

[0088] In the above example, if there are five records in the second question-and-answer result, corresponding to the five images in the image collection, and only one of them is the analysis result of the real facial photo uploaded by the user, then this record is the first question-and-answer result. If the real image is the user's facial photo, and the first question-and-answer result is the skin condition response text obtained by the large model based on the user's facial photo, then the first question-and-answer result can be "You have dermatitis and need to see a doctor as soon as possible."

[0089] Step S411: The client returns the first question and answer result to the user.

[0090] Through the above steps S401 to S411, the client receives a first question-and-answer request from the user, obtains a real image from the first question-and-answer request; the first question-and-answer request is used to request the generation of a first question-and-answer result for the real image. Then the client extracts features from the real image to obtain the real attribute information of the user, and then generates a confused image using the real attribute information. The client then randomly rearranges the order of the real image and the confused image generated by the image generation tool to generate an image set; generates a second question-and-answer request based on the image set; the second question-and-answer request is used to request the generation of a second question-and-answer result for the image set. Afterwards, the client sends the second question-and-answer request to the large model, receives the second question-and-answer result returned by the large model, and returns the first question-and-answer result generated for the real image in the second question-and-answer result to the user. By generating a confused image and mixing it with the real image, it is possible to effectively increase the difficulty for an attacker to identify the real image. The large model processes all images uniformly without knowing which image is the real image, which can avoid directly exposing the real image of the user, while ensuring the realization of the service function. This not only improves the security of the system, but also ensures the quality and efficiency of the service.

[0091] The above example illustrates this solution by taking the case where the big model and the image generation tool do not communicate with each other. When the big model is an intelligent big model, that is, the big model is integrated with the image generation tool, the user directly communicates with the big model, and the big model transmits the real attribute information to the image generation tool, and transmits the obfuscated image returned by the image generation tool to the user. In this scenario, in order to avoid leakage of user privacy data, the obfuscated image generated by the image generation tool needs to be encrypted.

[0092] For details, please refer to Figure 5 , Figure 5 A flowchart of another method for protecting image privacy during the use of a large model provided by an embodiment of the present application is shown, the method comprising steps S401 to S404, S501 to S507, and steps S407 to S411. For steps S401 to S404, and steps S407 to S411, please refer to the above description. The following mainly describes steps S501 to S507.

[0093] Step S401: The client receives a first question-and-answer request from a user.

[0094] Step S402: The client obtains a real image from the first question-and-answer request.

[0095] Step S403: The client extracts features from the real image to obtain the user's real attribute information.

[0096] Step S404: The client sends real attribute information to the large model.

[0097] Step S501: The large model converts the real attribute information into text prompt content that can be recognized by the image generation tool.

[0098] In this step, the large model can convert the real attribute information into instructions that meet the input specifications of the image generation model through a semantic mapping algorithm.

[0099] Step S502: The large model sends text prompt content to the image generation tool.

[0100] The large model is called through the API (Application Programming Interface) to transmit the text prompt content to the image generation tool and trigger the image generation service to obtain a set of obfuscated images that meet privacy protection requirements.

[0101] Step S503: The image generation tool performs image generation processing according to the text prompt content to obtain a confused image.

[0102] The image generation tool generates multiple obfuscated images based on the text prompt content, which are semantically related to the real image but not real. In other words, obfuscated images refer to synthetic images that are similar to the user's real image in visual features but cannot be associated with a specific individual. By creating visually similar but identity-irrelevant interference data, it increases the difficulty for large models to obtain privacy.

[0103] Step S504: The image generation tool encrypts the obfuscated image to obtain an encrypted obfuscated image.

[0104] Before step S504, such as before and after step S404 is executed, the client sends the first public key to the large model in addition to the real attribute information; wherein the first public key is used in the image generation tool to combine with the second public key generated by the image generation tool to obtain a shared key. The shared key is generated in the image generation tool, and the client can obtain the shared key based on the key exchange protocol. The shared key is only obtained by the client and the image generation tool. The shared key is used by the image generation tool to encrypt the obfuscated image to obtain the encrypted obfuscated image, and is used by the client to decrypt the encrypted obfuscated image to obtain the obfuscated image.

[0105] Step S505: The image generation tool sends the encrypted obfuscated image to the large model.

[0106] Step S506: The large model sends the encrypted obfuscated image to the client.

[0107] In step S504 to step S506, the image generation tool ensures the confidentiality of the obfuscated image when it is transmitted with the large model through encryption.

[0108] In some of the embodiments, after step S506, the method further includes: converting the image set into a data table, and determining the sensitive attributes of the user from the data table; calculating the difference between the distribution of sensitive attributes in each equivalence class and the distribution of sensitive attributes in the data table; wherein one equivalence class corresponds to data converted from an image, and the image includes a real image and a confused image; when the difference value is greater than a preset threshold, adjusting the number of generated images and the attribute difference when the image generation tool generates a confused image next time.

[0109] Specifically, when the difference value is greater than a preset threshold, the number of obfuscated images generated by the image generation tool next time is increased, and / or the attribute difference of the obfuscated image generated by the image generation tool next time is increased, so that privacy always complies with the t-closeness principle.

[0110] This embodiment dynamically adjusts the number of generated obfuscated images and attribute differences based on the difference value to optimize the effect of privacy protection. Compared with a fixed privacy protection strategy, this adaptive mechanism allows users to flexibly adjust the intensity of privacy protection according to actual tasks and needs.

[0111] Step S507: The client decrypts the encrypted obfuscated image to obtain an obfuscated image.

[0112] That is, the client uses the private key to decrypt the encrypted obfuscated image to restore it to a processable plaintext image (i.e., the obfuscated image).

[0113] Step S407: The client randomly rearranges the order of the real image and the obfuscated image generated by the image generation tool to generate an image set.

[0114] Step S408: The client sends a second question-answering request generated based on the image set to the large model.

[0115] Step S409: The large model analyzes the image set and generates a second question-answering result.

[0116] Step S410: The client obtains the first question-and-answer result generated by the real image from the second question-and-answer result.

[0117] Step S411: The client returns the first question and answer result to the user.

[0118] In this example, the image generation tool is combined with the big model, so the user only needs to upload a facial photo, and the big model can use the image generation tool to generate an obfuscated image for the user. The big model will analyze all received images and output the results, while the client only returns the results corresponding to the user's facial photo to the user. In this way, the privacy protection process of the photo is imperceptible to the user, so this method greatly improves the user experience. And in this way, the obfuscated image is encrypted and transmitted, so the user's privacy and security are still guaranteed. And this method can be applied to the user's private image privacy protection framework for a variety of scenarios and tasks, and is compatible with open source and closed source big models and third-party tools.

[0119] The following is a specific example to further illustrate the privacy protection of users by the method provided by this embodiment. The method based on mixing obfuscated images can be divided into the following types: Figure 6 The two stages shown are: stage one: generating fake images based on attributes; stage two: privacy protection request based on obfuscated images.

[0120] In the first stage, the user hopes to obtain a confused image to cover up the privacy attributes in the real image. The specific operations are as follows: 1. The user sends a request.

[0121] The user sends a request Q to the LLM data , which contains the requirement to generate an encrypted forged image and the user's public key A. The goal of the request is to generate d encrypted input images to ensure that user privacy is not leaked.

[0122] 2. LLM analysis request Q data .

[0123] LLM analysis request Q data , generate Task data Among them, Task data This includes generating obfuscated images that are similar to the user's privacy attributes. LLM transfers Taskdata to the image generation tool.

[0124] 3. Generate d obfuscated images.

[0125] After receiving the task, the image generation tool will generate d images x based on the stored privacy attribute library. j , where j∈d. These images are generated based on the user's privacy attributes to ensure that the generated obfuscated images are similar to the user's real images.

[0126] 4. Generate public key B and obtain shared key s.

[0127] The image generation tool generates its own public key B, and uses the DH key exchange protocol to calculate a shared key s together with the user's public key A. The shared key s will be used to encrypt and decrypt obfuscated images.

[0128] 5. Encrypt the generated obfuscated image.

[0129] The image generation tool uses the shared key s to generate a fake image x j Encrypt and generate an encrypted obfuscated image , where j∈d. The encrypted obfuscated image can prevent the image data from being directly accessed or understood by the outside world. After encryption is completed, the image generation tool will send the public key B and the encrypted obfuscated image Send to customer.

[0130] 6. The user obtains the shared key s.

[0131] The user obtains the shared key s through the DH protocol and communicates securely with the image generation tool, allowing the user to decrypt the received image in the next step.

[0132] 7. The user decrypts the encrypted obfuscated image.

[0133] The user uses the shared key s to encrypt the image Decrypt and restore the original image , where j∈d. So far, the user has obtained d obfuscated images similar to his private images.

[0134] In the second stage, the user's goal is to obtain a response to the private image without leaking the privacy attributes. The user submits the obfuscated image set to the LLM, and the LLM and the task execution tool cannot identify which one is the user's private image, thus satisfying the t-closeness principle.

[0135] 8. Mix d+1 images.

[0136] The user mixes all d decrypted obfuscated images and one of the user’s real images together to form d+1 images x j .

[0137] 9. The user sends a confusion request Q', which contains the query q and d+1 images x j .

[0138] The user randomly confuses d+1 images and generates new requests , where q represents a public query, x j Denotes d+1 mixed images. After completing the random confusion, the user sends the request Q' to the LLM.

[0139] 10. Parse user request q into Task i , where i∈m.

[0140] LLM parses the user’s public query q and converts it into m specific tasks. i , where i∈m. These tasks are related to the goals requested by the user, such as image processing and analysis.

[0141] 11. Generate response template t.

[0142] LLM generates a common response template t for each task, which is used to structure the task execution results. After completing the generation of the response template, LLM sends the task Task to the task execution tool. i and d+1 images X j .

[0143] 12. Execute the task Taski and generate an encrypted response result , where i∈m, j∈d+1. After the task is completed, the task execution tool sends the result set to the LLM.

[0144] 13. Combine the response result and the response template.

[0145] LLM will respond to the task results and response template t, for each image x j Generate a complete response .

[0146] 14. Select the response result of the real image.

[0147] The user selects the response result corresponding to his / her real image from the d+1 response results. This ensures that the user's real image is obfuscated throughout the process, achieving the purpose of effectively protecting the privacy of the user's real image.

[0148] The security of this embodiment is analyzed below.

[0149] The first is the isolation between the first and second stages: In the first stage, the image generation tool generates d obfuscated images according to the LLM task, and encrypts these images and returns them to the user through the LLM. In the second stage, the user mixes a real image with d obfuscated images and sends these d+1 images to the LLM. Since the images transmitted in the first stage are encrypted, and the user's real image and obfuscated image are mixed in the second stage, neither the LLM nor the task execution tool can infer the user's private image by cross-analyzing the image collections of the two stages.

[0150] Then the t-closeness principle is introduced for verification: the core of t-closeness is that given a sensitive attribute, the distribution of the sensitive attribute in the published aggregated data must be close enough to the distribution of the sensitive attribute in the entire dataset. Specifically, t-closeness requires that the distance between the distribution of the sensitive attribute in each equivalence class and the distribution of the sensitive attribute in the entire dataset does not exceed the preset threshold t.

[0151] Here, WD is used to quantify the difference between the distribution of sensitive attributes in each equivalence class and the distribution in the entire image set. In a set containing n images, the distribution of sensitive features in the equivalence class and the distribution in the entire set are: (1); (2); Among them, P v is the weight associated with class v in the distribution, q w is the weight associated with category w in the Q distribution, and the WD between P and Q is expressed as: (3); The WD expression satisfies the following constraints: ① ; ② ; ③ ; Among them, d vw is the distance between category v and category w, f vw is the flow variable, f vw It represents the probability mass of transferring from category v in distribution P to category w in distribution Q. According to the definition of t-closeness, if WD(P, Q) does not exceed the preset threshold t, it can be considered that the distribution of sensitive features in the equivalence class is close enough to the distribution in the entire image set, so as to avoid accidental leakage of sensitive features.

[0152] According to the definition of privacy attributes in images, these attributes are categorical attributes, and the values ​​of each category are independent of each other. Therefore, the equal distance metric is used here to measure the distance between two distributions. In this metric, the distance between any two values ​​in the categorical attribute is defined as 1, so the WD expression is converted to: (4).

[0153] By controlling the distribution of sensitive features in the image collection, it is ensured that even if some images are in special positions in the collection, their sensitive information will not deviate significantly from the overall distribution, thereby effectively protecting user privacy.

[0154] In the scheme based on mixing obfuscated images, LLM and specific task execution tools can detect the user’s private images in the image collection even if they know that the user’s private images are in the image collection. However, due to the mixed obfuscation image mechanism and encryption scheme, the user's real image cannot be directly inferred. WD measures the difference between the equivalence class and the overall distribution to ensure that each equivalence class meets t-closeness. Even if the image contains user sensitive features, the distribution of the feature in the equivalence class will not significantly deviate from the overall feature distribution, ensuring user privacy and security.

[0155] The following introduces specific scenarios in which the above method based on mixing confusion images can be applied. When the large model is a medical health model, the specific implementation steps are as follows: (1) Extract key attributes.

[0156] The user first uploads his or her facial photo to the local system. The system extracts key attributes from the photo, such as skin type, facial features, gender, and age group, and sends the extracted image key attribute information to the big model.

[0157] (2) Generate text prompts.

[0158] The large model generates text prompts based on key image attributes and sends the prompts to the image generation tool.

[0159] (3) Generate a confused image.

[0160] The image generation tool generates several obfuscated facial images based on the above text prompt information and encrypts the generated images. The generated images have similar facial attributes to the user's real photos, but have sufficient differences in appearance and features to avoid direct association with the user.

[0161] (4) Image set construction.

[0162] The local system receives and decrypts the encrypted obfuscated image set returned by the image generation tool. Then, the user's real photo and the generated obfuscated image are combined into an image set, and their order is randomized to prevent the user's real photo from being recognized by the large model.

[0163] (5) Privacy protection assessment.

[0164] During each interaction, the local system calculates the privacy metric through t-closeness, which can evaluate the extent to which the generated obfuscated image set enhances the security of user privacy. If the enhancement level is lower than the pre-set threshold T, the operation of generating obfuscated images needs to be performed again.

[0165] (6) Submit an obfuscated image request.

[0166] The user submits a set of images mixed with obfuscated images and a health query request to the medical health model. The request content contains the information of "analyzing the skin condition of these facial photos".

[0167] (7) Dataset analysis tasks.

[0168] After receiving the image set, the medical health model will analyze each image and return relevant results. Since the received image set has been randomly obfuscated, the model cannot identify which one is the user's real photo, preventing the model from further inferring the user's health privacy information.

[0169] (8) Obtain analysis results.

[0170] The local system will filter the analysis results of the image set returned by the large model, select the analysis results of the user's real photos, and filter out the analysis results of the obfuscated images. In this way, the user only obtains the analysis information related to his or her real photos.

[0171] Through the privacy protection scheme based on obfuscated images, users can effectively hide their real photos when interacting with the medical and health big model through images, preventing the big model from inferring the user's health privacy information without affecting the accuracy of the diagnosis provided by the system.

[0172] The present application also provides a method for protecting image privacy during the use of a large model, the method comprising the following steps: Receiving a first question-and-answer request from a user, and obtaining a real image from the first question-and-answer request; the first question-and-answer request is used to request generation of a first question-and-answer result for the real image; Extract features from real images to obtain real attribute information of users, which is then used by image generation tools to generate obfuscated images. Randomly rearrange the order of the real image and the obfuscated image generated by the image generation tool to generate an image set; generate a second question-answering request based on the image set; the second question-answering request is used to request generation of a second question-answering result for the image set; Sending a second question-and-answer request to the large model, and receiving a second question-and-answer result returned by the large model; The first question-and-answer result generated for the real image in the second question-and-answer result is returned to the user.

[0173] The method can be executed in an electronic device, a computer or a similar computing system. Taking running on an electronic device as an example, Figure 7 FIG. 1 shows a hardware structure block diagram of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device may include one or more ( Figure 7The electronic device may further include a transmission device 706 and an input / output device 708 for communication functions. It will be appreciated by those skilled in the art that Figure 2 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 2 More or fewer components as shown, or with Figure 2 Different configurations are shown.

[0174] The memory 704 can be used to store computer programs, for example, software programs and modules of application software. The processor 702 executes various functional applications and data processing by running the computer programs stored in the memory 704, that is, to implement the above method. The memory 704 can be used to store data, for example, real images, real attribute information, etc. The memory 704 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 704 may further include a memory remotely arranged relative to the processor 702, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0175] The transmission device 706 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the electronic device. In one example, the transmission device 706 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 706 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0176] The embodiment of the present application also provides a method for protecting image privacy during the use of a large model, which is applied to a server, where a large model and an image generation tool are deployed. In the method, the server performs the following steps: receiving real attribute information of a target user sent by a client, where the real attribute information is obtained by extracting features of a real image of the target user by the client; Using an image generation tool, the real attribute information is processed to generate an image to obtain an obfuscated image, and the obfuscated image is transmitted to the client; receiving a second question-answering request sent by the client, and obtaining an image set from the second question-answering request, wherein the image set includes real images and obfuscated images whose order is randomly rearranged; The image collection is analyzed through a large model to generate a second question-answering result.

[0177] Figure 8 A structural block diagram of a device for protecting image privacy during the use of a large model provided by an embodiment of the present application is shown. Figure 8 As shown, the device comprises: The data receiving module 801 is used to receive a first question-and-answer request from a user, and obtain a real image from the first question-and-answer request; the first question-and-answer request is used to request generation of a first question-and-answer result for the real image; A feature extraction module 802 is used to extract features from a real image to obtain real attribute information of the user, and the real attribute information is used by an image generation tool to generate a confusing image; The image obfuscation module 803 is used to randomly rearrange the order of the real image and the obfuscated image generated by the image generation tool to generate an image set; generate a second question-answering request based on the image set; the second question-answering request is used to request the generation of a second question-answering result for the image set; A request sending module 804 is used to send the second question-answering request to the large model and receive the second question-answering result returned by the large model; The question and answer result screening module 805 is used to return the first question and answer result generated for the real image in the second question and answer result to the user.

[0178] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0179] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0180] In addition, in combination with the method provided in the above embodiment, a storage medium can also be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by the processor, any one of the methods for protecting image privacy during the use of large models in the above embodiment is implemented.

[0181] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes each function or step executed by the processor in the above method embodiment.

[0182] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0183] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0184] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0185] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A method for protecting image privacy during the use of a large model, characterized in that: Applied to a client, the method comprises: receiving a first question-and-answer request from a user, and obtaining a real image from the first question-and-answer request; the first question-and-answer request is used to request generation of a first question-and-answer result for the real image; extracting real attribute information from the real image; Using the real attribute information, generating a confused image; Randomly rearrange the order of the real image and the obfuscated image to generate an image set; generate a second question-answering request based on the image set; the second question-answering request is used to request generation of a second question-answering result for the image set; Sending the second question-and-answer request to the large model, and receiving a second question-and-answer result returned by the large model; The first question-and-answer result generated for the real image in the second question-and-answer result is returned to the user.

2. The method for protecting image privacy during the use of large models according to claim 1, characterized in that: The method of generating a confused image by using the real attribute information comprises the following steps: Sending the real attribute information to the large model; wherein the real attribute information is converted by the large model into text prompt content that can be recognized by the image generation tool and sent to the image generation tool; Receive the encrypted obfuscated image generated by the image generation tool based on the text prompt content, and perform decryption processing to obtain the decrypted obfuscated image; wherein the encrypted obfuscated image is sent to the large model by the image generation tool and then sent to the client through the large model.

3. The method for protecting image privacy during use of a large model according to claim 2, characterized in that: Before receiving the encrypted obfuscated image generated by the image generation tool based on the text prompt content, the method further includes: Sending an image generation request to the large model, the image generation request including request information and a first public key; wherein the first public key is used by the image generation tool to combine with a second public key generated by the image generation tool to obtain a shared key; The shared key is obtained based on a key exchange protocol, and the shared key is used by the client to decrypt the encrypted obfuscated image to obtain the decrypted obfuscated image.

4. The method for protecting image privacy during the use of large models according to claim 1, characterized in that: The extracting real attribute information from the real image comprises: Feature extraction is performed on the user's uploaded photos to obtain picture feature information.

5. The method for protecting image privacy during use of a large model according to claim 2, characterized in that: After randomly rearranging the order of the real image and the confused image to generate an image set, the method further includes: Converting the image collection into a data table, and determining sensitive attributes of the user from the data table; Calculate the difference between the distribution of sensitive attributes in each equivalence class and the distribution of sensitive attributes in the data table; wherein one equivalence class corresponds to data converted from an image, and the image includes the real image and the obfuscated image; When the difference value is greater than a preset threshold, the generated quantity and attribute difference when the image generation tool generates a confused image next time are adjusted.

6. The method for protecting image privacy during use of a large model according to claim 5, characterized in that: The calculating the difference between the distribution of sensitive attributes in each equivalence class and the distribution of sensitive attributes in the data table includes: WD is used to represent the difference value. In a set of n images, the distribution of sensitive features in the equivalence class and the distribution in the entire set are expressed as: ； ； Among them, P v is the weight associated with class v in the distribution, q w is the weight associated with category w in the Q distribution, and the WD between P and Q is expressed as: ； The WD expression satisfies the following constraints: ； ； ； Among them, d vw is the distance between category v and category w, f vw is the flow variable, f vw represents the probability mass of transferring from category v in distribution P to category w in distribution Q; Convert the WD expression to: 。 7. A method for protecting image privacy during the use of a large model, characterized in that: Applied to a server, the server is deployed with a large model and an image generation tool, the method comprises: Receiving real attribute information for a target user sent by a client, where the real attribute information is extracted by the client from a real image of the target user; By using the image generation tool, performing image generation processing on the real attribute information to obtain an obfuscated image, and transmitting the obfuscated image to the client; receiving a second question-and-answer request sent by the client, and obtaining an image set from the second question-and-answer request, wherein the image set includes the real image and the obfuscated image after random rearrangement of their order; The image set is analyzed by the large model to generate a second question-answering result.

8. The method for protecting image privacy during use of a large model according to claim 7, characterized in that: After receiving the real attribute information of the target user sent by the client, the method further includes: By means of the large model, the real attribute information is converted into text prompt content that can be recognized by the image generation tool; The step of performing image generation processing based on the real attribute information by the image generation tool to obtain a confused image, and transmitting the confused image to the client includes: Performing image generation processing based on the text prompt content by the image generation tool to obtain a confused image; The obfuscated image is encrypted by the image generation tool to obtain an encrypted obfuscated image; after the encrypted obfuscated image is sent to the large model by the image generation tool, it is sent to the client through the large model.

9. The method for protecting image privacy during use of a large model according to claim 7, characterized in that: The real image is a photo uploaded by the user; The step of analyzing the image set by using the large model to generate a second question-answering result includes: The uploaded photos and the confused images in the image collection are analyzed respectively by the large model to generate a second question and answer result; wherein the second question and answer result includes the first question and answer result, and the first question and answer result is the reply text obtained by the large model through analyzing the uploaded photos of the user.

10. A system for protecting image privacy during the use of large models, characterized in that: The system includes a client and a server, wherein the client is configured to execute the method for protecting image privacy during the use of large models described in any one of claims 1 to 6, and the server is configured to execute the method for protecting image privacy during the use of large models described in any one of claims 7 to 9.

Images