Method and system oriented to image privacy protection during use of large model

Abstract

The present invention relates to the technical field of large models. Disclosed are a method and system oriented to image privacy protection during the use of a large model. In the method, a client mixes an obfuscated image into a real image, rather than sending the real image alone to a large model; the large model processes both the real image and the obfuscated image; and the client receives a processing result, and returns to a user only the part of the processing result with respect to the real image. On the basis of the solution in which an obfuscated image is mixed into a real image, the image privacy of a user during large-model-based conversational question answering is protected from leakage.

Description

A method and system for image privacy protection during large-scale model usage. Technical Field

[0001] This application relates to the field of large model technology, and in particular to a method and system for image privacy protection during the use of large models. Background Technology

[0002] In recent years, breakthroughs in deep learning technology have driven the significant development of Large Language Models (LLMs) in the field of natural language processing. Large language models (or large models) not only excel in tasks such as text generation and question-answering systems, but are also increasingly combined with various auxiliary tools to form large model agents with integrated enhancement tools. In scenarios where large model agents using integrated tools encounter users uploading private images, the large model and its invoked third-party tools can utilize visual cues in the images to infer users' identity, location, interests, intentions, and other private information. Furthermore, advanced analytics techniques can be used to delve deeper into users' privacy. The privacy risks arising from the use of users' private images are receiving considerable attention, and effective solutions to protect users' privacy are urgently needed.

[0003] Currently, privacy protection solutions for submitting private images during the use of large models mainly include local deployment, anonymous communication, and data masking. However, these methods have limitations in practically protecting user privacy.

[0004] Local deployment method: Deploying large open-source models and tools locally ensures that data does not leave the local execution environment, thereby protecting privacy. However, this method requires users to have a high level of expertise and hardware resources, and can only use large open-source models and tools, making it impossible to utilize many powerful closed-source models and tools.

[0005] Anonymous communication methods: Anonymous communication can hide the identities of both parties, preventing third parties from tracking or identifying the participants. However, anonymous communication does not provide complete privacy protection; Internet Protocol (IP) addresses can still be tracked.

[0006] Data masking methods: Data masking partially obscures or blurs user-submitted private images to ensure that private information is not exposed during image transmission and processing. However, data masking can affect image processing performance, especially in tasks requiring high-precision image analysis.

[0007] In summary, a new solution is urgently needed to address the privacy protection issues related to image submission during user interaction with large models. Summary of the Invention

[0008] This application provides a method and system for protecting image privacy during the use of large models. Based on a solution of mixing obfuscated images into real images, it protects the image privacy of users when engaging in dialogue and question-and-answer sessions using large models from being leaked.

[0009] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0010] In a first aspect, embodiments of this application provide a method for image privacy protection during the use of large models, applied to a client, the method comprising:

[0011] Receive a first question-and-answer request from a user, and obtain a real image from the first question-and-answer request; the first question-and-answer request is used to request the generation of a first question-and-answer result for the real image;

[0012] Extract real attribute information from the real image;

[0013] Using the real attribute information, a distorted image is generated;

[0014] The order of the real images and the obfuscated images is randomly rearranged to generate an image set; a second question-and-answer request is generated based on the image set; the second question-and-answer request is used to request the generation of a second question-and-answer result for the image set;

[0015] Send the second question-and-answer request to the large model and receive the second question-and-answer result returned by the large model;

[0016] The first question-and-answer result generated for the real image from the second question-and-answer result is returned to the user.

[0017] In conjunction with the first aspect, in one possible design approach, generating the obfuscated image using the real attribute information includes the following steps:

[0018] The real attribute information is sent to the large model; wherein the real attribute information is converted by the large model into text prompts that can be recognized by the image generation tool, and then sent to the image generation tool.

[0019] The image generation tool receives an encrypted obfuscated image generated based on the text prompt content, and decrypts it to obtain a decrypted obfuscated image; wherein, the encrypted obfuscated image is sent by the image generation tool to a large model, and then sent to the client through the large model.

[0020] In conjunction with the first aspect, in one possible design, before receiving the encrypted obfuscated image generated by the image generation tool based on the text prompt content, the method further includes:

[0021] An image generation request is sent to the large model. The image generation request includes request information and a first public key. The first public key is used to combine the image generation tool with a second public key generated by the image generation tool to obtain a shared key.

[0022] The shared key is obtained based on the key exchange protocol. The shared key is used by the client to decrypt the encrypted obfuscated image to obtain the decrypted obfuscated image.

[0023] In conjunction with the first aspect, in one possible design approach, the extraction of real attribute information from real images includes:

[0024] Feature extraction is performed on the user's uploaded photos to obtain image feature information.

[0025] In conjunction with the first aspect, in one possible design approach, after randomly rearranging the order of the real images and the obfuscated images to generate an image set, the method further includes:

[0026] The image set is converted into a data table, and the user's sensitive attributes are determined from the data table;

[0027] Calculate the difference between the distribution of sensitive attributes in each equivalence class and the distribution of sensitive attributes in the data table; wherein, one equivalence class corresponds to one image transformation data, and the image includes the real image and the obfuscated image;

[0028] If the difference value is greater than a preset threshold, the number of images generated and the attribute differences will be adjusted when the image generation tool generates the next scrambled image.

[0029] In conjunction with the first aspect, in one possible design approach, calculating the difference between the distribution of sensitive attributes in each equivalence class and the distribution of sensitive attributes in the data table includes:

[0030] Let WD represent the difference value. In a set containing n images, the distribution of sensitive features in the equivalence class and the distribution in the entire set are represented as follows:

[0031] Among them, P v q is the weight associated with class v in the distribution. w The weights associated with class w in the Q-distribution are represented by WD between P and Q as follows:

[0032] The WD expression satisfies the following constraints:

[0033] f vw ≥0, where 1≤v≤n, 1≤w≤n;

[0034] Where 1≤v≤n

[0035] Where, d vw f is the distance between category v and category w. vw It is a flow variable, f vw This represents the probability mass of a transition from class v in distribution P to class w in distribution Q;

[0036] Convert the WD expression to:

[0037] Secondly, embodiments of this application provide a method for image privacy protection during the use of large models, applied to a server, wherein the server is deployed with large models and image generation tools, and the method includes:

[0038] The system receives real attribute information about a target user sent by a client, the real attribute information being extracted by the client from a real image of the target user;

[0039] The image generation tool is used to process the real attribute information to generate an obfuscated image, which is then transmitted to the client.

[0040] Receive a second question-and-answer request sent by the client, and obtain an image set from the second question-and-answer request. The image set includes the real image after random rearrangement of its order and the obfuscated image.

[0041] The large model is used to analyze the image set to generate a second question-and-answer result.

[0042] In conjunction with the second aspect, in one possible design approach, after receiving the real attribute information of the target user sent by the client, the method further includes:

[0043] The large model is used to convert the real attribute information into text prompts that can be recognized by the image generation tool.

[0044] The step of generating an image based on the real attribute information using the image generation tool to obtain a scrambled image and transmitting the scrambled image to the client includes:

[0045] The image generation tool performs image generation processing based on the text prompt content to obtain a distorted image;

[0046] The image generation tool is used to encrypt the obfuscated image to obtain an encrypted obfuscated image; the encrypted obfuscated image is then sent to the large model by the image generation tool, and then sent to the client through the large model.

[0047] In conjunction with the second aspect, in one possible design approach, the real image is a photo uploaded by the user;

[0048] The step of analyzing the image set using the large model to generate a second question-and-answer result includes:

[0049] The large model analyzes the uploaded photos and the obfuscated images in the image set to generate a second question-and-answer result; wherein the second question-and-answer result contains a first question-and-answer result, which is the reply text obtained by the large model based on the user's uploaded photos.

[0050] Thirdly, embodiments of this application provide a computer device including a processor and a memory, the processor being configured to run a computer program in the memory to perform the method of the first aspect and its possible design.

[0051] Fourthly, embodiments of this application provide a server including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the method of the second aspect and its possible design.

[0052] Fifthly, embodiments of this application provide a system for image privacy protection during the use of large models, including a client and a server. The client, which relies on computer devices to implement functions, is configured as a method of the first aspect and its possible design. The server is configured to perform a method of the second aspect and its possible design.

[0053] In a sixth aspect, embodiments of this application provide a storage medium storing a computer program, wherein the computer program is configured to execute a method for the first aspect and its possible design methods at runtime, or to execute a method for the second aspect and its possible design methods.

[0054] Compared with existing technologies, this application provides a method and system for image privacy protection during the use of a large model. In the method, a client receives a first question-and-answer request from a user and obtains a real image from the request. The first question-and-answer request is used to request the generation of a first question-and-answer result for the real image. The client obtains real attribute information from the real image and then uses this information to generate a scrambled image. The client then randomly rearranges the order of the real and scrambled images to generate an image set. Based on the image set, a second question-and-answer request is generated; this request is used to request the generation of a second question-and-answer result for the image set. Afterwards, the client sends the second question-and-answer request to the large model, receives the second question-and-answer result returned by the large model, and returns the first question-and-answer result generated for the real image from the second question-and-answer result to the user. By generating a scrambled image and mixing it with the real image, the difficulty for attackers to identify the real image is effectively increased. The large model processes all images uniformly without knowing which image is the real image, avoiding direct exposure of the user's real image while ensuring the implementation of service functions. This improves system security and ensures service quality and efficiency.

[0055] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0056] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0057] Figure 1 shows a schematic diagram of a system for user interaction with a large model intelligent agent according to an embodiment of this application;

[0058] Figure 2 shows a schematic diagram of an image privacy protection method for large model usage provided in an embodiment of this application;

[0059] Figure 3 shows a schematic diagram of a system for image privacy protection during large-scale model usage provided in an embodiment of this application;

[0060] Figure 4 shows a flowchart of a method for image privacy protection during large-scale model usage provided in an embodiment of this application;

[0061] Figure 5 shows a flowchart of another method for image privacy protection during large-scale model usage provided in an embodiment of this application;

[0062] Figure 6 shows a schematic diagram of a method for mixing in obfuscated images according to an embodiment of this application;

[0063] Figure 7 shows a hardware structure block diagram of an electronic device provided in an embodiment of this application;

[0064] Figure 8 shows a structural block diagram of an image privacy protection device for use in large models, according to an embodiment of this application. Detailed Implementation

[0065] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0066] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0067] First, the terms that may be involved in the embodiments of this application will be explained:

[0068] DH (Diffie-Hellman key exchange protocol) is a security protocol that ensures that communicating parties can securely exchange keys over an insecure channel.

[0069] WD (Wasserstein Distance): A distance metric based on probability distributions, describing the minimum cost required to transform one probability distribution into another. It is commonly used in fields such as probability distributions, image processing, and machine learning.

[0070] Text-to-image generative models are machine learning-based models that generate images that match natural language descriptions given as input. Typically, these models consist of a language model and a generative model. The language model transforms the natural language input descriptions into vectors in a latent space, and the generative model uses these text vectors to generate the corresponding images. Image generation tools, as mentioned in this application, can use this model to generate obfuscated images.

[0071] Key exchange protocols are commonly used in insecure communication environments to allow multiple parties to securely establish a shared key. The shared key can be used to encrypt and decrypt information, ensuring the confidentiality, integrity, and authenticity of messages during communication. Key exchange protocols allow parties to establish a shared key without knowing each other's keys beforehand.

[0072] The t-closeness principle is a privacy protection principle that prevents the disclosure of sensitive attributes. According to this principle, attributes in data records can be divided into three categories:

[0073] Explicit identifiers: attributes that can directly and uniquely identify an individual.

[0074] Quasi-identifiers: When used alone, they cannot uniquely identify an individual, but when combined with other quasi-identifiers, they may narrow down the range of individuals.

[0075] Sensitive attributes: These are attributes that are typically considered to be private or sensitive.

[0076] The t-closeness principle requires that the distribution of sensitive attributes in each equivalence class (a set of records with the same quasi-identifier) ​​should be as close as possible to the overall distribution of sensitive attributes in the entire dataset. Specifically, the difference between the distribution of sensitive attributes in an equivalence class and the distribution of sensitive attributes in the dataset should not exceed a set threshold t.

[0077] First, the process of user interaction with the large model agent will be explained. Please refer to Figure 1, which shows a schematic diagram of a system for user interaction with a large model agent according to an embodiment of this application. The system includes user 101, large model 102, and task execution tool 103. In the first step, the user requests Q = {q, x}, which includes a public query q and a private image input x, where the sensitive information contained in x is denoted as p(x). User 101 wants to obtain the answer through the large model 102 and avoid the large model 102 and task execution tool 103 inferring the correlation between the sensitive information p(x) and user 101.

[0078] The large model 102 acts as the controller in the system, responsible for planning tasks and selecting appropriate task execution tools 103 to complete different tasks. The task execution tool 103 executes the tasks assigned by the large model 102 according to the input parameter `param`. There are bidirectional communication channels between the user 101 and the large model 102, and between the large model 102 and the task execution tool 103. The user 101 sends a request Q to the large model 102, expecting a correct response. After receiving the user request, the large model 102 performs the second step, parsing the request Q and decomposing it into multiple sub-tasks, forming a task set `task`. i ={key, target, param}. The large model 102 plans the task order and dependencies, and then generates a response template t based on all tasks.

[0079] Third, the large model 102 assigns the parsed tasks to the task execution tool 103. The task execution tool 103 executes the assigned tasks according to the input parameters param and outputs the prediction results C. i∈m Return to large model 102.

[0080] In the fourth step, the large model 102 fills the received results into the template t, generates the final response result T, and returns it to the user 101.

[0081] Fifth step, user 101 obtains the response result.

[0082] As shown in Figure 1, when a user interacts with the large model, the user needs to submit public queries and private images (equivalent to real images in the embodiments of this application) to the large model. The large model then orchestrates tasks and calls relevant enhancement tools to execute them. During this process, the large model obtains the user's private image input and shares this information with relevant tools during task execution, raising the risk of exposing the user's privacy information. For example, when a user provides a private image when interacting with the large model agent, the large model shares the image with third-party tools according to business needs. The large model and the tools can then analyze and infer the user's private information.

[0083] While some privacy protection schemes have been proposed for the submission of private images during the use of large models, each method has its shortcomings: local deployment requires users to have high technical skills and hardware resources, and is limited to the use of open source models and tools; anonymous communication cannot provide complete privacy protection; and data masking methods can affect the image processing results.

[0084] To address the privacy protection issue related to image submission during user interaction with large models, this application provides a novel solution. Referring to Figure 2, which illustrates a method for image privacy protection during large model usage, this application's embodiment demonstrates a solution where the client does not send the real image separately to the large model. Instead, a scrambled image is mixed into the real image. The large model processes both the real and scrambled images. The client receives the processing results and only returns the portion related to the real image to the user. This solution of mixing a scrambled image into the real image protects the user's image privacy during Q&A sessions with the large model. In this embodiment, the client can use an image generation tool to generate the scrambled image. Alternatively, the client can execute the steps of the image generation tool, generating the scrambled image based on the real attribute information.

[0085] The following description uses the method provided in this application embodiment for skin condition analysis as an example. It should be understood that the examples given in this application embodiment are not intended to limit the scope of patent protection. The method provided in this application embodiment can be applied to various scenarios such as medical and health care, financial data anonymization, and smart security. Its purpose is to protect information in real images that could reveal a user's true profile, preventing large models and their tools from further mining user privacy based on this information. For example, in the medical and health care field, this method can be used to protect patient privacy. For instance, during skin condition analysis, when a user uploads a skin image, big data analyzes it. During this process, the system generates an obfuscated image and encrypts its transmission, ensuring that large models and their tools can accurately diagnose while failing to identify the patient. Similarly, in the financial industry, this method can be used to process sensitive image data (user ID card images) to prevent its leakage. Furthermore, in the field of smart security, this method can hide the residential location of pedestrians to prevent large models from associating the pedestrian's residential location with their identity information and inferring their movement trajectory.

[0086] The method provided in this application embodiment is applied to a system for image privacy protection during the use of large models. As shown in Figure 3, the system includes a client 31 and a server 32, and a large model and image generation tool 33 are deployed on the server.

[0087] The client provides a user-facing interface, responsible for receiving real images uploaded by users and the initial question-and-answer request. The client refers to a program that provides local services to the client. The client can be software installed on a computer device, such as a mobile phone or computer, and can be a browser, email client, or chat application. The large model is responsible for analyzing the images and generating question-and-answer results. The image generation tool is configured to generate obfuscated images based on feature information.

[0088] In some embodiments, the client communicates bidirectionally with the large model and the image generation tool respectively. In an embodiment, please refer to FIG4, which shows a flowchart of a method for image privacy protection during the use of a large model provided by an embodiment of this application. The method includes steps S401 to S411.

[0089] Step S401: The client receives the first question and answer request from the user.

[0090] In this embodiment, step S401 refers to the client receiving a question-and-answer request initiated by the user. This request includes the user's question-and-answer intent and related information. Specifically, the first question-and-answer request includes request information, which is used to request the generation of a first question-and-answer result for the real image.

[0091] In practical applications, users initiate an initial Q&A request to the client through an interactive interface. The client receives this request, which includes a real image uploaded by the user and associated semantic instructions. The real image refers to the original image file provided by the user that contains personal biometric features or sensitive information, and the associated semantic instructions could be something like "analyze skin condition." The response the user wants is the Q&A result based on the analysis of the real image (i.e., the initial Q&A result).

[0092] The first question-and-answer request will be explained below in the context of an application scenario. In the scenario of skin condition analysis and processing, the first question-and-answer request might be a facial photo uploaded by the user, along with the instruction "Please analyze my skin condition." In this example, the user requests analysis of their facial skin condition and has uploaded a facial photo. This facial photo refers to a real image. The client receives the user's request to analyze their facial skin condition.

[0093] Step S402: The client obtains the real image from the first question and answer request.

[0094] After receiving the user's initial question-and-answer request, the client needs to extract the actual image uploaded by the user. As an example, the user can upload a real image to the client so that the client can retrieve the data.

[0095] After step S402, the client does not directly send the real image to the large model for analysis. Instead, it extracts the attribute information from the real image and sends the attribute information to the large model, preventing the large model from reconstructing the user's real image based on the attribute information. See step S403 for details.

[0096] Step S403: The client extracts real attribute information from the real image.

[0097] In this step, the client performs in-depth analysis of the acquired real images to extract the real attribute information contained within. This real attribute information includes details such as face, limbs, torso, skin, and geographical location, which can reveal the user's true profile.

[0098] Step S403 may further include: extracting features from the user's uploaded photos to obtain image feature information. The uploaded photos can be photos of people, landscapes, houses, etc. Among these, photos of people can be photos of faces, hands, legs, etc. When the photo is a face, the actual attribute information includes one or more of skin type, facial features, gender, and age, which will be explained in detail below.

[0099] Skin type: Extract features such as skin color and texture from the image to determine whether the user's skin is dry, oily, or combination; extract skin lesion features from the image to determine the symptoms of the user's skin; extract facial hair follicle status features from the image to determine whether the user has inflammation or skin damage.

[0100] Facial features: Extract features such as the user's face shape (e.g., round, oblong), the position and shape of facial features (e.g., eye size, nose bridge height).

[0101] Gender: The user's gender is inferred based on factors such as facial contours and eyebrow shape, serving as the user's gender characteristic.

[0102] Age: The user's age range is estimated by combining features such as wrinkle distribution and skin elasticity, and used as the user's age feature.

[0103] One or more of the above features are used to generate obfuscated images. It is understood that since the real attribute information is extracted from the real image but is not completely identical to the real image, this data not only helps to generate obfuscated images that are similar to but not exactly the same as the real image, but also ensures that the obfuscated image maintains consistency in some key attributes, thereby protecting user privacy.

[0104] In some embodiments, the client performs multi-dimensional analysis of real images using a pre-trained feature extraction model, outputting real attribute information. For example, the U-Net segmentation model is used to locate skin regions and calculate pore density and erythema index; the FaceNet model is used to generate embedding vectors, which contain geometric features such as facial proportions and contour curvature.

[0105] Step S404: The client sends the actual attribute information to the image generation tool.

[0106] After feature extraction is complete, the client sends the extracted real attribute information to an image generation tool specifically responsible for generating the obfuscated image. This image generation tool can be deployed on the client side or on the server side. In this embodiment, the image generation tool and the large model do not communicate with each other, so the client interacts directly with the image generation tool. For example, in the above example, the client might package a series of feature information such as "dry skin," "round face," "female," and "30 years old" into a data packet and then transmit it to the image generation tool over the network.

[0107] Step S405: The image generation tool performs image generation processing on the real attribute information to obtain a distorted image.

[0108] The real attribute information is used by the image generation tool to generate obfuscated images; that is, after receiving the real attribute information from the client, the image generation tool uses an algorithm to process it and generate the corresponding obfuscated image. There can be one or more obfuscated images.

[0109] Continuing with skin condition analysis as an example, suppose feature extraction reveals the user's skin type to be "dry," facial features to be "round face," gender to be "female," and age to be approximately "30 years old." Then, the image generation tool might generate the following types of obfuscated images.

[0110] Confusing Image A: A facial photograph with dry skin characteristics but slightly different facial features, including erythema on the face.

[0111] Confusing Image B: A photo of a round face but with a different skin type (e.g., oily or combination), showing signs of dermatitis.

[0112] Although spoofed images A and B differ from the real image in some features, they maintain a high correlation with the real image in key attributes (such as facial lesion features). This increases the difficulty for large models or image generation tools to identify which image is the user's real image, thereby achieving the goal of protecting user privacy.

[0113] Step S406: The client receives the obfuscated image from the image generation tool.

[0114] In this embodiment, the client can communicate with the image generation tool. Therefore, the obfuscated images generated by the tool will not be obtained by the large model, making it impossible for the large model to determine which images are the user's real images and which are obfuscated images generated by the tool. Thus, in this embodiment, the obfuscated images can be unencrypted. Of course, to improve transmission security, the obfuscated images can be encrypted before transmission to prevent attackers from intercepting them and analyzing the user's sensitive information.

[0115] The number of obfuscated images can be one or more. As an example, the image generation tool generates five obfuscated images at once and packages them into a folder or compressed file to send back to the client.

[0116] Step S407: The client randomly rearranges the order of the real image and the scrambled image generated by the image generation tool to generate an image set.

[0117] In this step, the client mixes the real image obtained in step S402 with the newly generated obfuscated image. The mixing method involves rearranging the image order according to a certain random rule to form a new image set. This method prevents large models from inferring the position of the real image in the image set based on the order of the image arrangement.

[0118] The following explains the mixing method: If the real image is a user's face photo, and the mixed image consists of four photos of different types, the client can number these five images A, B, C, D, and E, and then use a random number generator to shuffle their order, for example, the new order is C, A, E, B, and D.

[0119] This step further increases the difficulty for attackers to identify the real images by randomly rearranging the order of the real and obfuscated images, thereby improving the security of privacy protection.

[0120] Step S408: The client sends a second question-and-answer request based on the image set to the large model.

[0121] After constructing the image set, the client generates a second question-and-answer request based on it. This second request contains both the user's real image and the generated obfuscated image. The second request is used to request the larger model to analyze both the real and obfuscated images separately and generate analysis results.

[0122] Continuing with the skin condition analysis example above, the second question could be, "Please perform a skin condition analysis on this image set containing multiple facial photographs." This request requires the large model to perform corresponding analysis and processing on each image.

[0123] This step generates a second question-and-answer request and passes the image set to the large model, enabling the large model to process all images uniformly without knowing which image is real. This ensures the large model's service functionality is implemented while avoiding directly exposing the user's real image.

[0124] Step S409: The large model analyzes the image set and generates the second question-and-answer result.

[0125] After receiving the second question-and-answer request and its accompanying image set from the client, the large model will analyze and process each image one by one and provide the corresponding analysis results.

[0126] Continuing with the example of skin condition analysis, the large model will evaluate the skin condition of each image in the image set and give results such as "This image shows that the skin condition is dry and has slight wrinkles" or "Your skin has dermatitis and you need to see a doctor as soon as possible".

[0127] Since neither the real image nor the obfuscated image was transmitted separately to the large model, the large model could not analyze the image set to determine which image was the user's real image. Therefore, the large model could not use visual cues in the images to infer the user's identity, location, interests, intentions, and other private information.

[0128] Step S410: The client obtains the first question and answer result generated from the real image from the second question and answer result.

[0129] After receiving the second question-and-answer result returned by the large model, the client finds the part that corresponds to the original real image, which is the first question-and-answer result.

[0130] In the example above, if the second question-and-answer result contains five records, each corresponding to one of the five images in the image set, and only one of these records is an analysis result about the user's uploaded real facial photo, then that record is the first question-and-answer result. If the real image is a photo of the user's face, and the first question-and-answer result is the skin condition response text obtained by the large model analyzing the user's facial photo, then the first question-and-answer result could be "You have dermatitis and need to see a doctor as soon as possible."

[0131] Step S411: The client returns the first question and answer result to the user.

[0132] Through steps S401 to S411, the client receives a first question-and-answer request from the user and retrieves the real image from it. This first request is used to request the generation of a first question-and-answer result for the real image. The client then extracts features from the real image to obtain the user's real attribute information and uses this information to generate a scrambled image. The client then randomly rearranges the order of the real image and the scrambled image generated by the image generation tool to generate an image set. Based on this image set, a second question-and-answer request is generated to request the generation of a second question-and-answer result for the image set. Afterward, the client sends the second question-and-answer request to the large model, receives the second question-and-answer result returned by the large model, and returns the first question-and-answer result generated for the real image from the second question-and-answer result to the user. By generating a scrambled image and mixing it with the real image, the difficulty for attackers to identify the real image is effectively increased. The large model processes all images uniformly without knowing which image is the real image, avoiding direct exposure of the user's real image while ensuring the service functionality. This improves system security and ensures service quality and efficiency.

[0133] The above example illustrates this solution using the case where the large model and the image generation tool do not communicate with each other. When the large model is an intelligent agent model, i.e., the large model integrates the image generation tool, the user communicates directly with the large model. The large model transmits the real attribute information to the image generation tool and transmits the obfuscated image returned by the image generation tool to the user. In this scenario, to avoid leakage of user privacy data, the obfuscated image generated by the image generation tool needs to be encrypted.

[0134] Specifically, please refer to Figure 5, which shows a flowchart of another method for image privacy protection during large-scale model usage provided by an embodiment of this application. This method includes steps S401 to S404, S501 to S507, and S407 to S411. For steps S401 to S404 and S407 to S411, please refer to the description above. The following mainly describes steps S501 to S507.

[0135] Step S401: The client receives the first question and answer request from the user.

[0136] Step S402: The client obtains the real image from the first question and answer request.

[0137] Step S403: The client extracts features from the real image to obtain the user's real attribute information.

[0138] Step S404: The client sends the actual attribute information to the large model.

[0139] Step S501: The large model converts the real attribute information into text prompts that can be recognized by the image generation tool.

[0140] In this step, the large model can use a semantic mapping algorithm to convert real attribute information into instructions that conform to the input specifications of the image generation model.

[0141] Step S502: The large model sends a text prompt to the image generation tool.

[0142] The large model transmits the text prompt content to the image generation tool through API (Application Programming Interface) calls, and triggers the image generation service to obtain a set of obfuscated images that meet privacy protection requirements.

[0143] Step S503: The image generation tool performs image generation processing based on the text prompt to obtain a garbled image.

[0144] Image generation tools generate multiple obfuscated images that are semantically related to real images but are not actually real, based on text prompts. In other words, obfuscated images are synthetic images that are visually similar to a user's real image but cannot be associated with a specific individual. By creating visually similar but identity-irrelevant interference data, these tools increase the difficulty for large models to obtain privacy information.

[0145] Step S504: The image generation tool encrypts the obfuscated image to obtain the encrypted obfuscated image.

[0146] Before step S504, as before or after step S404, in addition to sending the real attribute information to the large model, the client also sends a first public key. This first public key is used by the image generation tool to combine with a second public key generated by the image generation tool to obtain a shared key. The shared key is generated in the image generation tool, and the client can obtain the shared key based on a key exchange protocol. The shared key is only available to the client and the image generation tool. The shared key is used by the image generation tool to encrypt the obfuscated image, obtaining an encrypted obfuscated image, and by the client to decrypt the encrypted obfuscated image, obtaining the obfuscated image.

[0147] Step S505: The image generation tool sends the encrypted obfuscated image to the large model.

[0148] Step S506: The large model sends the encrypted obfuscated image to the client.

[0149] In steps S504 to S506, the image generation tool ensures the confidentiality of the obfuscated image when it is transmitted with the large model through encryption.

[0150] In some embodiments, after step S506, the method further includes: converting the image set into a data table, determining the user's sensitive attributes from the data table; calculating the difference between the distribution of sensitive attributes in each equivalence class and the distribution of sensitive attributes in the data table; wherein, an equivalence class corresponds to the data of one image conversion, and the images include real images and obfuscated images; if the difference value is greater than a preset threshold, adjusting the number of generated images and the attribute difference when the image generation tool generates obfuscated images next time.

[0151] Specifically, when the difference value is greater than a preset threshold, the number of images generated by the image generation tool in the next generation of obfuscated images is increased, and / or the attribute differences in the next generation of obfuscated images by the image generation tool are increased, so that privacy always conforms to the t-closeness principle.

[0152] This embodiment dynamically adjusts the number of generated obfuscated images and attribute differences based on the difference values, optimizing the privacy protection effect. Compared to a fixed privacy protection strategy, this adaptive mechanism allows users to flexibly adjust the strength of privacy protection according to actual tasks and needs.

[0153] Step S507: The client decrypts the encrypted obfuscated image to obtain the obfuscated image.

[0154] That is, the client uses its private key to decrypt the encrypted obfuscated image, restoring it to a processable plaintext image (i.e., the obfuscated image).

[0155] Step S407: The client randomly rearranges the order of the real image and the scrambled image generated by the image generation tool to generate an image set.

[0156] Step S408: The client sends a second question-and-answer request based on the image set to the large model.

[0157] Step S409: The large model analyzes the image set and generates the second question-and-answer result.

[0158] Step S410: The client obtains the first question and answer result generated from the real image from the second question and answer result.

[0159] Step S411: The client returns the first question and answer result to the user.

[0160] In this example, the image generation tool is integrated with a large model. Users only need to upload a facial photo, and the large model can generate an obfuscated image using the image generation tool. The large model analyzes all received images and outputs the results, while the client only returns the result corresponding to the user's facial photo. Thus, the privacy protection process is imperceptible to the user, significantly improving the user experience. Furthermore, the obfuscated image is transmitted with encryption, ensuring user privacy. This method is applicable to various scenarios and tasks, providing a user privacy protection framework that is compatible with both open-source and closed-source large models and third-party tools.

[0161] The following specific example further illustrates the user privacy protection provided by the method in this embodiment. The method based on the mixing of obfuscated images can be divided into two stages as shown in Figure 6: Stage 1: generating a fake image based on attributes; Stage 2: requesting privacy protection based on the obfuscated image.

[0162] In the first phase, users want to obtain obfuscated images to mask the privacy attributes of the real images. The specific steps are as follows:

[0163] 1. The user sends a request.

[0164] The user sends a request Q to the LLM. data The request includes the requirement to generate encrypted forged images and the user's public key A. The goal is to generate d encrypted input images to ensure user privacy is not compromised.

[0165] 2. LLM parsing request Q data .

[0166] LLM parsing request Q data Generate Task data Among them, Task data This includes generating obfuscated images that resemble user privacy attributes. LLM transmits Taskdata to the image generation tool.

[0167] 3. Generate d scrambled images.

[0168] After receiving the task, the image generation tool will generate d images x based on the stored privacy attribute library. j , where j∈d. These images are generated based on the user's privacy attributes to ensure that the generated obfuscated images are similar to the user's real images.

[0169] 4. Generate public key B and obtain shared key s.

[0170] The image generation tool generates its own public key B and uses the DH key exchange protocol to calculate a shared key s together with the user's public key A. This shared key s will be used to encrypt and decrypt obfuscated images.

[0171] 5. Encrypt the generated obfuscated image.

[0172] The image generation tool uses a shared key s to generate a fake image x. j Encryption is performed to generate an encrypted obfuscated image. Where j∈d. The encrypted obfuscated image prevents the image data from being directly accessed or understood by outsiders. After encryption, the image generation tool will use the public key B and the encrypted obfuscated image. Send to the customer.

[0173] 6. The user retrieves the shared key s.

[0174] Users obtain a shared key 's' via the DH protocol to securely communicate with the image generation tool. This allows users to decrypt the received image in subsequent steps.

[0175] 7. Users decrypt encrypted obfuscated images.

[0176] Users use a shared key to encrypt images. Decrypt and recover the original image. Where j∈d. Thus, the user has obtained d obfuscated images similar to their own private image.

[0177] In the second phase, the user's goal is to obtain a response containing private images without revealing their privacy attributes. The user submits a set of obfuscated images to the LLM (Local Management Module), and the LLM and the task execution tool cannot identify which image is the user's private, thus satisfying the t-closeness principle.

[0178] 8. Mix d+1 images.

[0179] The user combines all d decrypted obfuscated images with one of their own real images to form d+1 images x. j .

[0180] 9. The user sends an obfuscation request Q', containing a query for q and d+1 images x. j .

[0181] The user randomly obfuscates d+1 images and generates a new request.

[0182] Q′={q,x j∈d+1}, where q represents a public query, x j This represents d+1 mixed images. After random obfuscation is complete, the user sends request Q' to the LLM.

[0183] 10. Parse user request q as Task i , where i∈m.

[0184] LLM parses a user's public query q and transforms it into m specific tasks. i , where i∈m. These tasks are related to the user's requested objectives, such as image processing and analysis.

[0185] 11. Generate response template t.

[0186] LLM generates a generic response template t for each task, used to structure the task execution results. After generating the response template, LLM sends the task (Task) to the task execution tool. i and d+1 images X j .

[0187] 12. Execute TaskSki to generate an encrypted response. Where i∈m, j∈d+1. After the task is completed, the task execution tool sends the result set to the LLM.

[0188] 13. Combination of response results and response templates.

[0189] LLM will provide the task response results. Combined with the response template t, for each image x j Generate a complete response

[0190] 14. Select the response result of the real image.

[0191] The user selects the response that corresponds to their real image from d+1 response results. This ensures that the user's real image is obfuscated throughout the process, effectively protecting the privacy of the user's real image.

[0192] The security of this embodiment will be analyzed below.

[0193] First, there is the isolation between phase one and phase two:

[0194] In the first stage, the image generation tool generates d obfuscated images according to the LLM's task, encrypts these images, and returns them to the user via the LLM. In the second stage, the user mixes one real image with the d obfuscated images and sends these d+1 images to the LLM. Because the images transmitted in the first stage are encrypted, and the second stage mixes the user's real image with obfuscated images, neither the LLM nor the task execution tool can infer the user's private image by cross-analyzing the image sets from both stages.

[0195] Then, the t-closeness principle is introduced for verification: the core of t-closeness is that, given a sensitive attribute, the distribution of that sensitive attribute in the published aggregated data must be sufficiently close to the distribution of sensitive attributes in the entire dataset. Specifically, t-closeness requires that the distance between the distribution of sensitive attributes in each equivalence class and the distribution of sensitive attributes in the entire dataset does not exceed a preset threshold t.

[0196] Here, WD is used to quantify the difference between the distribution of sensitive attributes in each equivalence class and the distribution in the entire image set. In a set containing n images, the distributions of sensitive features in the equivalence classes and the distribution in the entire set are as follows:

[0197] Among them, P v q is the weight associated with class v in the distribution. w The weights associated with class w in the Q-distribution are represented by WD between P and Q as follows:

[0198] The WD expression satisfies the following constraints:

[0199] ①f vw ≥0, where 1≤v≤n, 1≤w≤n;

[0200] ② Where 1≤v≤n

[0201] ③

[0202] Where, d vw f is the distance between category v and category w. vw It is a flow variable, f vw d represents the probability quality of transition from category v in distribution P to category w in distribution Q; according to the definition of t-closeness, if WD(P,Q) does not exceed the preset threshold t, it can be considered that the distribution of sensitive features in the equivalence class is close enough to the distribution in the entire image set, so as not to cause accidental leakage of sensitive features.

[0203] Based on the definition of privacy attributes in images, these attributes are all categorical attributes, and the values ​​of each category are independent of each other. Therefore, an equal interval metric is used here to measure the distance between the two distributions. In this metric, the distance between any two values ​​in the categorical attribute is defined as 1, so the WD expression is transformed into:

[0204] By controlling the distribution of sensitive features in the image set, it is ensured that even if some images are in special positions in the set, their sensitive information will not deviate significantly from the overall distribution, thus effectively protecting user privacy.

[0205] In schemes based on obfuscated images, LLMs and specific task execution tools can prevent the spread of even private images from the user's image set χ. j∈d+ However, due to the inclusion of image obfuscation mechanisms and encryption schemes, it is impossible to directly infer the user's real image. By measuring the difference between equivalence classes and the overall distribution using WD, it is ensured that each equivalence class satisfies t-closeness. Even if the image contains user-sensitive features, the distribution of these features in the equivalence class will not significantly deviate from the overall feature distribution, thus guaranteeing user privacy and security.

[0206] The following describes specific scenarios where the method based on mixed-in obfuscated images can be applied. When the large model is a medical and health model, the specific implementation steps are as follows:

[0207] (1) Extract key attributes.

[0208] Users first upload their facial photos to the local system. The system extracts key attributes from the photos, such as skin type, facial features, gender, and age group, and then sends the extracted key image attribute information to the large model.

[0209] (2) Generate text prompts.

[0210] The large model generates text prompts based on key image attributes and sends the prompts to the image generation tool.

[0211] (3) Generate a distorted image.

[0212] The image generation tool generates several facial obfuscation images based on the aforementioned text prompts and encrypts the generated images. The generated images have similar facial attributes to the user's real photos, but differ sufficiently in appearance and features to avoid direct association with the user.

[0213] (4) Image set construction.

[0214] The local system receives and decrypts the encrypted and obfuscated image set returned by the image generation tool. Then, it combines the user's real photo and the generated obfuscated images into a single image set, randomizing their order to prevent the large model from recognizing the user's real photo.

[0215] (5) Privacy protection assessment.

[0216] During each interaction, the local system calculates a privacy metric using t-closeness, which can assess the extent to which the generated obfuscated image set enhances the security of user privacy. If the enhancement is lower than a pre-set threshold T, the obfuscated image generation operation needs to be repeated.

[0217] (6) Submit an obfuscated image request.

[0218] Users submit sets of images mixed with obfuscated images and their health query requests to a large-scale healthcare model. The request content includes information such as "skin condition analysis of these facial photos".

[0219] (7) Dataset analysis task.

[0220] After receiving the image set, the large-scale healthcare model analyzes each image and returns relevant results. Because the received image set is randomly obfuscated, the model cannot identify which image is the user's real photo, preventing the large-scale model from further speculating on the user's health and privacy information.

[0221] (8) Obtain the analysis results.

[0222] The local system filters the image set analysis results returned by the large model, selecting the analysis results for the user's real photos and filtering out the analysis results for obfuscated images. In this way, the user only receives analysis information relevant to their real photos.

[0223] By employing a privacy protection scheme based on obfuscated images, users can effectively hide their real photos when interacting with large medical and health models, preventing the models from speculating on users' health and privacy information, without affecting the accuracy of the diagnoses provided by the system.

[0224] This application also provides a method for image privacy protection during the use of large models, the method comprising the following steps:

[0225] Receive the first question-and-answer request from the user, and obtain the real image from the first question-and-answer request; the first question-and-answer request is used to request the generation of the first question-and-answer result for the real image;

[0226] Feature extraction is performed on real images to obtain the user's real attribute information, which is then used by image generation tools to generate obfuscated images.

[0227] The order of real images and obfuscated images generated by image generation tools is randomly rearranged to generate an image set; a second question-and-answer request is generated based on the image set; the second question-and-answer request is used to request the generation of a second question-and-answer result for the image set;

[0228] Send the second question-and-answer request to the large model and receive the second question-and-answer result returned by the large model;

[0229] The first question and answer result generated from the real image in the second question and answer result is returned to the user.

[0230] This method can be executed in electronic devices, computers, or similar computing systems. Taking an electronic device as an example, Figure 7 shows a hardware structure block diagram of an electronic device according to an embodiment of this application. As shown in Figure 7, the electronic device may include one or more processors 702 (only one is shown in Figure 7), and a memory 704 for storing data. The processor 702 may include, but is not limited to, a processing device such as a microprocessor (MCU) or a programmable logic device (FPGA). The electronic device may also include a transmission device 706 for communication functions and an input / output device 708. Those skilled in the art will understand that the structure shown in Figure 2 is merely illustrative and does not limit the structure of the electronic device. For example, the electronic device may include more or fewer components than shown in Figure 2, or have a different configuration than that shown in Figure 2.

[0231] Memory 704 can be used to store computer programs, such as application software programs and modules. Processor 702 executes various functional applications and data processing by running the computer programs stored in memory 704, thereby implementing the methods described above. Memory 704 can be used to store data, such as real images, real attribute information, etc. Memory 704 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, memory 704 may further include memory remotely located relative to processor 702, and these remote memories can be connected to electronic devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0232] The transmission device 706 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the electronic device. In one example, the transmission device 706 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 706 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0233] This application also provides a method for image privacy protection during the use of large models, applied to a server, which deploys large models and image generation tools. In this method, the server performs the following steps:

[0234] Receive real attribute information for the target user sent by the client. The real attribute information is obtained by the client extracting features from the real image of the target user.

[0235] An image generation tool is used to process the real attribute information to generate an obfuscated image, which is then transmitted to the client.

[0236] Receive the second question-and-answer request sent by the client, and obtain the image set from the second question-and-answer request. The image set includes real images and spoofed images after random rearrangement of their order.

[0237] A second question-and-answer result is generated by analyzing the image set using a large model.

[0238] Figure 8 shows a structural block diagram of an image privacy protection device for large model usage provided in an embodiment of this application. As shown in Figure 8, the device includes:

[0239] Data receiving module 801 is used to receive a first question-and-answer request from a user and obtain a real image from the first question-and-answer request; the first question-and-answer request is used to request the generation of a first question-and-answer result for the real image;

[0240] The feature extraction module 802 is used to extract features from real images to obtain the user's real attribute information. The real attribute information is used by the image generation tool to generate obfuscated images.

[0241] The image obfuscation module 803 is used to randomly rearrange the order of real images and obfuscated images generated by the image generation tool to generate an image set; and to generate a second question-and-answer request based on the image set; the second question-and-answer request is used to request the generation of a second question-and-answer result for the image set.

[0242] The request sending module 804 is used to send the second question-and-answer request to the large model and receive the second question-and-answer result returned by the large model;

[0243] The question and answer result filtering module 805 is used to return the first question and answer result generated for the real image from the second question and answer result to the user.

[0244] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0245] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0246] Furthermore, in conjunction with the methods provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the image privacy protection methods described in the above embodiments for use with large models.

[0247] This application also provides a computer program product that, when run on a computer, causes the computer to perform various functions or steps executed by the processor in the above method embodiments.

[0248] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0249] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0250] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0251] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for protecting image privacy during the use of large models, characterized in that, Applied to a client, the method includes: Receive a first question-and-answer request from a user, and obtain a real image from the first question-and-answer request; the first question-and-answer request is used to request the generation of a first question-and-answer result for the real image; Extract real attribute information from the real image; Using the real attribute information, a distorted image is generated; The order of the real images and the obfuscated images is randomly rearranged to generate an image set; a second question-and-answer request is generated based on the image set; the second question-and-answer request is used to request the generation of a second question-and-answer result for the image set; Send the second question-and-answer request to the large model and receive the second question-and-answer result returned by the large model; The first question-and-answer result generated for the real image from the second question-and-answer result is returned to the user.

2. The method for image privacy protection during the use of large models according to claim 1, characterized in that, The process of generating a distorted image using the real attribute information includes the following steps: The real attribute information is sent to the large model; wherein the real attribute information is converted by the large model into text prompts that can be recognized by the image generation tool, and then sent to the image generation tool. The image generation tool receives an encrypted obfuscated image generated based on the text prompt content, and decrypts it to obtain a decrypted obfuscated image; wherein, the encrypted obfuscated image is sent by the image generation tool to a large model, and then sent to the client through the large model.

3. The method for image privacy protection during the use of large models according to claim 2, characterized in that, Before receiving the encrypted and obfuscated image generated by the image generation tool based on the text prompt content, the method further includes: An image generation request is sent to the large model. The image generation request includes request information and a first public key. The first public key is used to combine the image generation tool with a second public key generated by the image generation tool to obtain a shared key. The shared key is obtained based on the key exchange protocol. The shared key is used by the client to decrypt the encrypted obfuscated image to obtain the decrypted obfuscated image.

4. The method for image privacy protection during the use of large models according to claim 1, characterized in that, The step of extracting real attribute information from the real image includes: Feature extraction is performed on the user's uploaded photos to obtain image feature information.

5. The method for image privacy protection during the use of large models according to claim 2, characterized in that, After randomly rearranging the order of the real images and the obfuscated images to generate an image set, the method further includes: The image set is converted into a data table, and the user's sensitive attributes are determined from the data table; Calculate the difference between the distribution of sensitive attributes in each equivalence class and the distribution of sensitive attributes in the data table; wherein, one equivalence class corresponds to one image transformation data, and the image includes the real image and the obfuscated image; If the difference value is greater than a preset threshold, the number of images generated and the attribute differences will be adjusted when the image generation tool generates the next scrambled image.

6. The method for image privacy protection during the use of large models according to claim 5, characterized in that, The calculation of the difference between the distribution of sensitive attributes in each equivalence class and the distribution of sensitive attributes in the data table includes: Let WD represent the difference value. In a set containing n images, the distribution of sensitive features in the equivalence class and the distribution in the entire set are represented as follows: Among them, P v q is the weight associated with class v in the distribution. w The weights associated with class w in the Q-distribution are represented by WD between P and Q as follows: The WD expression satisfies the following constraints: f vw ≥0, where 1≤v≤n, 1≤w≤n; Where 1≤v≤n; Where, d vw f is the distance between category v and category w. vw It is a flow variable, f vw This represents the probability mass of a transition from class v in distribution P to class w in distribution Q; Convert the WD expression to:

7. A method for protecting image privacy during the use of large models, characterized in that, Applied to a server, the server being deployed with large models and image generation tools, the method includes: The system receives real attribute information about a target user sent by a client, the real attribute information being extracted by the client from a real image of the target user; The image generation tool is used to process the real attribute information to generate an obfuscated image, which is then transmitted to the client. Receive a second question-and-answer request sent by the client, and obtain an image set from the second question-and-answer request. The image set includes the real image after random rearrangement of its order and the obfuscated image. The large model is used to analyze the image set to generate a second question-and-answer result.

8. The method for image privacy protection during the use of large models according to claim 7, characterized in that, After receiving the real attribute information of the target user sent by the client, the method further includes: The large model is used to convert the real attribute information into text prompts that can be recognized by the image generation tool. The step of generating an image based on the real attribute information using the image generation tool to obtain a scrambled image and transmitting the scrambled image to the client includes: The image generation tool performs image generation processing based on the text prompt content to obtain a distorted image; The image generation tool is used to encrypt the obfuscated image to obtain an encrypted obfuscated image; the encrypted obfuscated image is then sent to the large model by the image generation tool, and then sent to the client through the large model.

9. The method for image privacy protection during the use of large models according to claim 7, characterized in that, The real image is a photo uploaded by the user; The step of analyzing the image set using the large model to generate a second question-and-answer result includes: The large model analyzes the uploaded photos and the obfuscated images in the image set to generate a second question-and-answer result; wherein the second question-and-answer result contains a first question-and-answer result, which is the reply text obtained by the large model based on the user's uploaded photos.

10. A system for image privacy protection during the use of large models, characterized in that, The system includes a client and a server, the client being configured to perform the method for image privacy protection during large-scale model usage as described in any one of claims 1 to 6, and the server being configured to perform the method for image privacy protection during large-scale model usage as described in any one of claims 7 to 9.

Images