Image desensitization method, device, equipment, medium and product

Abstract

The invention discloses an image desensitization method, device and equipment, a medium and a product. The method comprises the following steps: acquiring a to-be-desensitized image uploaded by a user and the fidelity required by the user, and determining target desensitized content in the to-be-desensitized image; generating a mask corresponding to the to-be-desensitized image according to a corresponding sensitive area of the target desensitized content in the to-be-desensitized image; inputting the to-be-desensitized image and the mask into a preset sensitive content elimination model to obtain a sensitive content missing image; and repairing the sensitive content missing image according to the user demand fidelity to obtain a target desensitized image. According to the technical scheme, after the sensitive information of the to-be-desensitized image is eliminated, the sensitive content missing image is repaired according to the user demand fidelity specified by the user to obtain the corresponding target desensitized image, so that high-fidelity desensitization processing of the sensitive information in the image is realized, and the image desensitization effect and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image desensitization method, device, equipment, medium and product. Background Art

[0002] With the development of new power systems, images related to power systems are increasing day by day, and inevitably involve some sensitive information, such as the internal structure of power equipment, high-voltage line paths, safety warning areas, and employee ID information. In order to protect this sensitive information from being leaked, images containing sensitive information need to be desensitized. In existing image desensitization methods, most of them directly block sensitive information in the original image to be desensitized, and do not optimize the desensitized image. This may cause problems such as loss of image details and poor image quality, resulting in poor image desensitization effect. Summary of the invention

[0003] The present invention provides an image desensitization method, device, equipment, medium and product to achieve high-fidelity desensitization processing of sensitive information in an image and improve the image desensitization effect and accuracy.

[0004] According to one aspect of the present invention, there is provided an image desensitization method, the method comprising:

[0005] Obtain the desensitized images uploaded by the user and the fidelity required by the user, and determine the target desensitized content in the desensitized images;

[0006] Generate a mask corresponding to the image to be desensitized according to the corresponding sensitive area of ​​the target desensitized content in the image to be desensitized;

[0007] Input the image to be desensitized and the mask into a preset sensitive content elimination model to obtain an image with sensitive content missing;

[0008] The image with missing sensitive content is repaired according to the fidelity required by the user to obtain the target desensitized image.

[0009] According to another aspect of the present invention, there is provided an image desensitization device, the device comprising:

[0010] The desensitized content determination module is used to obtain the desensitized images uploaded by the user and the fidelity required by the user, and determine the target desensitized content in the desensitized images;

[0011] A mask generation module, used to generate a mask corresponding to the image to be desensitized according to the corresponding sensitive area of ​​the target desensitization content in the image to be desensitized;

[0012] The sensitive content removal module is used to input the image to be desensitized and the mask into a preset sensitive content removal model to obtain an image with sensitive content missing;

[0013] The image restoration module is used to restore the image with missing sensitive content according to the fidelity required by the user to obtain the target desensitized image.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image desensitization method described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the image desensitization method described in any embodiment of the present invention when executed by a processor.

[0019] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program, and the computer program, when executed by a processor, implements the image desensitization method described in any embodiment of the present invention.

[0020] The technical solution of the embodiment of the present invention obtains the image to be desensitized uploaded by the user and the user's required fidelity, and determines the target desensitized content in the image to be desensitized; generates a mask corresponding to the image to be desensitized according to the corresponding sensitive area of ​​the target desensitized content in the image to be desensitized; inputs the image to be desensitized and the mask into a preset sensitive content elimination model to obtain an image with missing sensitive content; and repairs the image with missing sensitive content according to the user's required fidelity to obtain the target desensitized image. This technical solution eliminates sensitive information from the image to be desensitized, and then repairs the image with missing sensitive content according to the user's required fidelity specified by the user to obtain the corresponding target desensitized image, thereby solving the problem of loss of image details and poor image quality after image desensitization, achieving high-fidelity desensitization processing of sensitive information in the image, and improving the image desensitization effect and accuracy.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flow chart of an image desensitization method provided according to Embodiment 1 of the present invention;

[0024] Figure 2 is a flow chart of an image desensitization method provided according to Embodiment 2 of the present invention;

[0025] Figure 3 is a flow chart of an image desensitization method provided according to Embodiment 3 of the present invention;

[0026] Figure 4 is a structural schematic diagram of an image desensitization device provided according to a fourth embodiment of the present invention;

[0027] Figure 5 It is a schematic diagram of the structure of an electronic device for implementing the image desensitization method of an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Embodiment 1

[0031] Figure 1This is a flow chart of an image desensitization method provided in the first embodiment of the present invention. This embodiment is applicable to the case of high-fidelity desensitization processing of sensitive information in an image. The method can be executed by an image desensitization device, which can be implemented in the form of hardware and / or software, and can be configured in an electronic device. Figure 1 As shown, the image desensitization method provided in the first embodiment specifically includes the following steps:

[0032] S110, obtaining the to-be-desensitized image uploaded by the user and the fidelity required by the user, and determining the target desensitized content in the to-be-desensitized image.

[0033] The images to be desensitized may be images related to the power system, which may contain sensitive information such as the internal structure of power equipment, high-voltage line paths, safety warning areas, and user identity information.

[0034] User required fidelity may refer to the image fidelity specified by the user for the image to be desensitized. Exemplarily, the value range of the user required fidelity may be 0 to 3, with a value of 0 representing the lowest fidelity and a value of 3 representing the highest fidelity.

[0035] The target desensitized content may refer to the content to be desensitized in the image to be desensitized. The target desensitized content may include textual forms (such as ID number, contact information, etc.) and non-textual forms (such as internal structure diagrams of power facilities, user faces, etc.).

[0036] In an embodiment of the present invention, it is possible to obtain the desensitized image to be uploaded by the user and need to be desensitized, and at the same time obtain the user demand fidelity specified by the user for the image to be desensitized, and then use image recognition software, deep neural network and other means to identify the corresponding target sensitive content from the image to be desensitized, such as power equipment operation information, user identity information, etc. In a specific embodiment, the target detection algorithm and OCR (Optical Character Recognition) recognition algorithm can be used to recognize the image to be desensitized, and the target sensitive content in the image to be desensitized can be identified. Furthermore, the method for determining the target desensitized content in the image to be desensitized can also be: first use the target detection algorithm and the OCR recognition algorithm to identify the first sensitive content in the image to be desensitized, and then feed back the first sensitive content to the user for self-selection, and then extract the target desensitized content specified by the user from the desensitization instruction fed back by the user.

[0037] S120: Generate a mask corresponding to the image to be desensitized according to the corresponding sensitive area of ​​the target desensitization content in the image to be desensitized.

[0038] Among them, the sensitive area can be understood as the pixel area where the target desensitized content is located in the image to be desensitized. The shape of the sensitive area can be a rectangle, circle or irregular shape matching the sensitive area. For example, taking the target desensitized content as sensitive text, the sensitive area can be the smallest rectangular area containing the sensitive text.

[0039] In an embodiment of the present invention, the corresponding sensitive area can be located in the image to be desensitized according to the obtained target desensitization content, such as the rectangular area corresponding to the user's ID number in the image to be desensitized, the irregular pixel area corresponding to the internal structure image of the power equipment, etc., and then each sensitive area is processed as a target object, for example, the pixel value corresponding to the sensitive area is set to 1, etc., and then the corresponding mask is obtained. In a specific embodiment, a pre-trained mask generation model can be obtained, and the model may include an encoder and a decoder, wherein the encoder is used to extract the features of the image to be desensitized and capture the sensitive areas in the image; the decoder is used to receive the output of the decoder and generate a mask for the sensitive area to indicate which image areas need to be desensitized.

[0040] S130: Input the image to be desensitized and the mask into a preset sensitive content elimination model to obtain an image lacking sensitive content.

[0041] The preset sensitive content elimination model may refer to a pre-configured model for eliminating target sensitive content from the image to be desensitized. For example, the preset sensitive content elimination model may be a LaMa image elimination model. The sensitive content missing image may refer to an image obtained after eliminating the target sensitive content in the image to be desensitized.

[0042] In an embodiment of the present invention, the acquired image to be desensitized and the corresponding mask can be input into a pre-configured preset sensitive content elimination model to eliminate the target sensitive content in the image to be desensitized, thereby obtaining a corresponding sensitive content missing image.

[0043] S140: Repair the image with missing sensitive content according to the fidelity required by the user to obtain a target desensitized image.

[0044] In an embodiment of the present invention, since the desensitized image obtained directly after sensitive content is eliminated may have problems such as loss of image details and poor image quality, in order to solve the above problems, this embodiment will further perform image restoration processing on the sensitive content missing image obtained in the aforementioned step S130 according to the user demand fidelity specified by the user, retain the image details after desensitization as much as possible, obtain the restored target desensitized image, and ensure the high quality and high fidelity output of the desensitized image. In a specific embodiment, the restoration process of the sensitive content missing image can be: input the sensitive content missing image and the mask together into a pre-trained image restoration model, wherein the maximum model iteration number of the image restoration model is positively correlated with the user demand fidelity, that is, the higher the user demand fidelity, the greater the corresponding maximum model iteration number, and the image restoration model may include an encoder, a decoder and an output layer, the encoder is used to extract high-level features of the sensitive content missing image, the decoder is used to gradually restore the high-level features to the spatial resolution of the original image, and obtain the corresponding feature map, and the output layer is used to perform image fusion on the feature map and the mask, thereby obtaining the restored target desensitized image.

[0045] The technical solution of the embodiment of the present invention obtains the image to be desensitized uploaded by the user and the user's required fidelity, and determines the target desensitized content in the image to be desensitized; generates a mask corresponding to the image to be desensitized according to the corresponding sensitive area of ​​the target desensitized content in the image to be desensitized; inputs the image to be desensitized and the mask into a preset sensitive content elimination model to obtain an image with missing sensitive content; and repairs the image with missing sensitive content according to the user's required fidelity to obtain the target desensitized image. This technical solution eliminates sensitive information from the image to be desensitized, and then repairs the image with missing sensitive content according to the user's required fidelity specified by the user to obtain the corresponding target desensitized image, thereby solving the problem of loss of image details and poor image quality after image desensitization, achieving high-fidelity desensitization processing of sensitive information in the image, and improving the image desensitization effect and accuracy.

[0046] Embodiment 2

[0047] Figure 2 This is a flowchart of an image desensitization method provided in Embodiment 2 of the present invention, which is further optimized and expanded based on the above-mentioned implementation, and can be combined with various optional technical solutions in the above-mentioned implementation. Figure 2 As shown, the image desensitization method provided in the second embodiment specifically includes the following steps:

[0048] S210, receiving an image to be desensitized uploaded by a user, and calling a preset information extraction model to extract text information from the image to be desensitized; wherein the preset information extraction model at least includes an optical character recognition model.

[0049] In an embodiment of the present invention, an image to be desensitized that needs to be desensitized and uploaded by a user can be obtained, and then an optical character recognition (OCR) model can be called to recognize the image to be desensitized, and text information in the image can be extracted, such as power equipment operation information, user identity information, etc.; wherein the extracted text information may include sensitive text information and non-sensitive text information.

[0050] It should be understood that the above embodiment is described by taking text content as an example. In actual applications, a preset information extraction model based on target detection can also be used to extract non-text content in the image to be desensitized. This embodiment does not impose specific restrictions on this.

[0051] S220: Input the text information into a preset business language model to obtain candidate desensitized content.

[0052] The preset business big language model can be understood as a big language model configured for the power system business. The preset business big language model can directly adopt a commercial big language model, or can be obtained by fine-tuning and training based on a basic big language model. This embodiment does not limit this. The candidate desensitized content can refer to sensitive content identified from text information using a preset business big language model.

[0053] In an embodiment of the present invention, text information extracted from the image to be desensitized can be input into a pre-configured preset business large language model, and the preset business large language model can be used to identify sensitive information, thereby obtaining candidate desensitized content.

[0054] S230: Display the candidate desensitized content and the desensitization suggestions corresponding to the candidate desensitized content to the user, and receive the desensitization instructions fed back by the user.

[0055] In an embodiment of the present invention, since the (candidate) desensitized content output by the preset business large language model may deviate from the actual needs of the user, in order to improve the accuracy of subsequent image desensitization, after the preset business large language model identifies the candidate desensitized content in the image to be desensitized, the candidate desensitized content and the corresponding desensitization suggestions can be fed back to the user together. For example, they can be displayed to the user through a visual interface, allowing the user to select the content to be desensitized by themselves, and feedback the corresponding desensitization instructions to the system.

[0056] S240: extracting the user requirement fidelity and the target desensitized content from the desensitization instruction.

[0057] The target desensitized content may refer to the desensitized content specified by the user from the candidate desensitized content.

[0058] In an embodiment of the present invention, the desensitization instruction fed back by the user can be parsed to extract the user demand fidelity and target desensitization content specified by the user. In a specific embodiment, a preset business language model can be called to extract the user demand fidelity and target desensitization content from the desensitization instruction.

[0059] S250, inputting the image to be desensitized into an image segmentation module of a preset dynamic mask generation model to perform semantic segmentation on the image to be desensitized to obtain an image segmentation region.

[0060] The preset dynamic mask generation model can be understood as a corresponding model for generating a mask that can locate the desensitized content of the target. Exemplarily, the preset dynamic mask generation model can be constructed based on a convolutional neural network. The image segmentation module can include but is not limited to: FCN (Fully Convolutional Networks), U-Net and other image semantic segmentation models.

[0061] In an embodiment of the present invention, an image segmentation module of a preset dynamic mask generation model may be called to perform semantic segmentation on the image to be desensitized, so as to identify the image segmentation area corresponding to the target sensitive content in the image to be desensitized.

[0062] S260: According to the label of the target desensitization content, a sensitive area corresponding to the label of the target desensitization content is identified in each image segmentation area.

[0063] Among them, the labels of the target desensitized content may include but are not limited to: personal information (such as user ID number, contact information, etc.), business information (such as power equipment operation information, research and development information, etc.), communication information (such as user communication records, etc.), etc.

[0064] In an embodiment of the present invention, the pre-set labels for each target desensitized content can be used to identify and match the image segmentation areas obtained by semantic segmentation, so as to locate the sensitive area corresponding to each target desensitized content in the image to be desensitized, such as the rectangular area corresponding to the user's ID number.

[0065] S270, input each sensitive area into a mask generation module of a preset dynamic mask generation model to generate a mask corresponding to the image to be desensitized.

[0066] Among them, the mask generation module may refer to a model for generating a mask corresponding to the image to be desensitized, and the mask generation module may be constructed based on a convolutional neural network.

[0067] In an embodiment of the present invention, each identified sensitive area may be input into a pre-trained mask generation module to generate a mask capable of locating each sensitive area, wherein the mask has the same size as the image to be desensitized.

[0068] Further, based on the above-mentioned embodiment of the invention, S270 further includes:

[0069] Step 1: Using the image to be desensitized and the mask as inputs of a preset coverage effect evaluation function to obtain a coverage effect evaluation value;

[0070] Step 2: Based on the preset coverage effect influencing parameters and with the goal of minimizing the coverage effect evaluation value, the mask is optimized and adjusted to obtain a new mask; wherein the preset coverage effect influencing parameters include: mask transparency, image segmentation accuracy, mask shape, and coverage intensity of sensitive areas.

[0071] In the embodiment of the present invention, after the corresponding mask is generated in S270, its coverage effect can be evaluated, and the mask can be dynamically adjusted according to the corresponding coverage effect evaluation result. Specifically, the following preset coverage effect evaluation function can be called to determine the coverage effect evaluation value E(I,M) corresponding to the original image to be desensitized I and the generated mask M, that is:

[0072] E(I,M)=γSSIM(I,I⊙M)+δPSNR(I,I⊙M)

[0073] Wherein, ⊙ represents the Hadamard product (point-to-point multiplication), which is used to multiply the desensitized image I and the mask M bit by bit; SSIM represents the structural similarity index measure, which is used to measure the quality and structural fidelity of the image; PSNR represents the peak signal-to-noise ratio, which is used to measure the clarity and noise level of the image; γ and δ represent the coefficients used to control the influence of SSIM and PSNR on the overall evaluation, respectively.

[0074] During the mask generation process, the selection of preset coverage effect influencing parameters will have different degrees of influence on the mask coverage effect evaluation value. Among them, the preset coverage effect influencing parameters may include: mask transparency, image segmentation accuracy, mask shape, and coverage intensity of sensitive areas. Therefore, the preset coverage effect evaluation function includes a comprehensive consideration of the above-mentioned multiple factors.

[0075] Based on the above preset coverage effect influencing parameters, the coverage effect of the mask can be dynamically adjusted with the goal of minimizing the coverage effect evaluation value E(I,M) to regenerate the optimal mask M. * The multivariable optimization formula for dynamic mask adjustment is as follows:

[0076] S280: Input the image to be desensitized and the mask into a preset sensitive content elimination model to obtain an image lacking sensitive content.

[0077] In an embodiment of the present invention, the acquired image to be desensitized and the corresponding mask can be input together into a pre-configured preset sensitive content elimination model to eliminate the target sensitive content in the image to be desensitized, thereby obtaining a corresponding sensitive content missing image; wherein, the preset sensitive content elimination model includes: LaMa image elimination model.

[0078] S290. The product of the user demand fidelity and the preset model initial iteration number is taken as the maximum model iteration number.

[0079] The preset model initial iteration number may refer to the model iteration number initially set for the preset image loop restoration model.

[0080] In the embodiment of the present invention, the product of the user demand fidelity F and the preset model initial iteration number T can be used as the maximum model iteration number T max , that is, T max =T·F. It can be understood that the higher the user demand fidelity F, the corresponding maximum number of model iterations T max The bigger it is.

[0081] S2100: Input the image with missing sensitive content into the encoder of the preset image loop repair model to extract high-level features of the image with missing sensitive content.

[0082] Among them, the preset image loop repair model may refer to a model used to repair images with missing sensitive content, and the preset image loop repair model may include an encoder, a decoder and an output layer.

[0083] In an embodiment of the present invention, the acquired sensitive content missing image can be input into an encoder of a pre-trained preset image loop repair model to extract high-level features corresponding to the sensitive content missing image, such as edge features, texture features, and shape features.

[0084] S2110, inputting the high-level features into the decoder of the preset image loop restoration model to generate a feature map corresponding to the high-level features.

[0085] In an embodiment of the present invention, the extracted high-level features can be input into a decoder of a preset image loop restoration model to gradually restore the high-level features to the spatial resolution of the original image to be desensitized, restore the image details, and obtain the corresponding feature map.

[0086] S2120, input the feature map and the mask to the output layer of the preset image loop restoration model for image fusion, obtain the target restoration image, and update the number of iterations of the preset image loop restoration model.

[0087] In an embodiment of the present invention, the output layer of the preset image loop repair model can be used to fuse the feature map generated by the decoder with the mask to generate a repaired target repair image, and the number of iterations of the preset image loop repair model is updated at the same time; wherein, the pixel values ​​in the sensitive area of ​​the target repair image are replaced with the predicted values ​​of the model, and the remaining areas remain unchanged.

[0088] S2130. Determine whether the updated number of iterations reaches the maximum number of model iterations.

[0089] S2140. If yes, the target repaired image is used as the final target desensitized image; if not, the target repaired image is used as the input of the encoder of the preset image loop repair model, and the above image repair process is repeatedly executed until the number of iterations reaches the maximum number of model iterations.

[0090] In an embodiment of the present invention, a preset image loop repair model can be called to continuously repair images with missing sensitive content. After each round of iteration, it is determined whether the number of iterations of the current model reaches the maximum number of model iterations. If so, the currently generated target repair image is used as the target desensitized image after the final desensitization; if not, the target repair image is used as the input of the encoder of the next round of preset image loop repair model, and the image repair process is re-executed until the model reaches the maximum number of model iterations and generates the final target desensitized image.

[0091] Further, based on the above-mentioned embodiments of the invention, after obtaining the target desensitized image, the method may further include:

[0092] Step 1: Display the target desensitized image to the user, and receive the image desensitization feedback instruction from the user;

[0093] Step 2: When the image desensitization feedback instruction is to select the specified sensitive content as the target desensitization content, return to execute the step of generating a mask corresponding to the image to be desensitized according to the corresponding sensitive area of ​​the target desensitization content in the image to be desensitized;

[0094] Step 3: When the image desensitization feedback instruction is to reselect the user requirement fidelity, return to execute the step of obtaining the user requirement fidelity uploaded by the user, and determine the target desensitization content in the image to be desensitized;

[0095] Step 4: When the image desensitization feedback instruction indicates that no modification is required, the image desensitization process ends.

[0096] In an embodiment of the present invention, the generated target desensitized image can be fed back to the user for inspection, and the next step of operation can be performed according to the image desensitization feedback instruction fed back by the user. Specifically, if the image desensitization feedback instruction is that the user manually selects and specifies new sensitive content as the target desensitized content in the image to be desensitized, then return to execute the step of generating a mask corresponding to the image to be desensitized according to the corresponding sensitive area of ​​the target desensitized content in the image to be desensitized; if the image desensitization feedback instruction is that the user reselects the user required fidelity, that is, re-issues the desensitization instruction, then return to execute the step of obtaining the user required fidelity uploaded by the user, and determine the target desensitized content in the image to be desensitized; if the image desensitization feedback instruction is that no modification is required, then the image desensitization process is ended and the image desensitization is completed.

[0097] The technical solution of the embodiment of the present invention extracts text information in the image to be desensitized by using a preset information extraction model, and then uses a preset business large language model to identify sensitive information to obtain candidate desensitized content, extracts the corresponding user demand fidelity and target desensitized content from the desensitization instruction fed back by the user, and then uses a preset dynamic mask generation model to generate a corresponding mask, and uses a preset sensitive content elimination model to eliminate sensitive information to obtain a sensitive content missing image, and finally repairs the sensitive content missing image according to the user demand fidelity specified by the user to obtain the corresponding target desensitized image, which solves the problems of loss of image details and poor image quality after image desensitization. The generated mask can locate sensitive content more accurately, and can dynamically adjust the number of image repair iterations according to the user demand fidelity. While ensuring that the image fidelity meets the requirements, computing power resources are saved, and high-fidelity desensitization processing of sensitive information in the image is achieved, thereby improving the image desensitization effect and accuracy.

[0098] Embodiment 3

[0099] Figure 3 This is a flowchart of an image desensitization method provided in Example 3 of the present invention. Based on the above embodiments, this embodiment provides an implementation of an image desensitization method, which can realize human-computer dialogue, reduce the difficulty of using the system, and realize high-fidelity image sensitive information desensitization processing. Figure 3 As shown, an image desensitization method provided by Embodiment 3 of the present invention specifically includes the following steps:

[0100] S310: receiving an image to be desensitized uploaded by a user.

[0101] S320: Call the OCR model to extract text information from the image to be desensitized.

[0102] S330: Input the text information into a preset business language model to obtain candidate desensitized content, and display the candidate desensitized content and corresponding desensitization suggestions to the user.

[0103] S340: Receive the desensitization instruction fed back by the user, and call the preset business large language model to extract the user demand fidelity and the target desensitization content from the desensitization instruction.

[0104] In an embodiment of the present invention, when the preset business big language model extracts the user demand fidelity and the target desensitized content, a result guiding sentence needs to be used to guide the preset business big language model to complete the content extraction. The result guiding sentence generation formula is expressed as follows:

[0105]

[0106] Where ξ is the fixed part of the identity-guided sentence; C i To give the big model an identity; Q j is the question number; ε is a fixed sentence in the sample; X i Desensitize the target content; Y i Fidelity for user needs; A j is the answer number; θ is the extracted answer of the instruction; is the fixed part of the instruction; Z k is the task description given to the large language model; μ is the output format of the answer, usually a JSON string.

[0107] S350: Input the image to be desensitized and the target desensitized content into a preset dynamic mask generation model to generate a corresponding mask.

[0108] In an embodiment of the present invention, the preset dynamic mask generation model may adopt a convolutional neural network based on masking loss, and the training process thereof is as follows:

[0109] Step 1: Collect sensitive data images and divide them into training set, validation set and test set;

[0110] Step 2: Use tools to perform geometric correction on the data in the dataset, and then perform normalization;

[0111] Step 3: Initialize the network weights of the model, usually using a random initialization method such as Glorot initialization or He initialization to avoid gradient disappearance or explosion problems;

[0112] Step 4: Forward propagation: the input data passes through the network, passing through the convolution layer, activation function, pooling layer, and fully connected layer in sequence until the output is generated; in each layer, the input data is convolved with the weight of the layer, and then nonlinearly transformed through the activation function (such as ReLU);

[0113] Step 5: Loss calculation. The loss function calculation of the convolutional neural network based on masking loss is shown as follows:

[0114]

[0115] Where N c Indicates the total number of categories; w i Represents the weight of the i-th category, which is used to adjust the impact of the classification loss of this category on the total loss; Represents the classification loss of the i-th category, which is used to measure the difference between the model prediction category loss and the true category; N m Indicates the total number of shaded areas; m j Represents the weight of the jth shielding area, which is used to adjust the impact of the shielding loss of this area on the total loss; represents the loss of the jth masked area, which is used to measure the difference between the mask generated by the model and the real mask; α represents the coefficient of the smoothness loss, which is used to control its influence on the total loss; L smooth represents the smoothness loss, which is used to penalize the discontinuity or non-smoothness of the masking boundary; β represents the coefficient of the regularization loss, which is used to control its influence on the total loss; L reg Represents the regularization loss, which is used to penalize the complexity of the model and prevent overfitting.

[0116] Step 6: Back propagate and calculate the gradient of the loss function with respect to the network weights. This is done in the back propagation process using the chain rule. The gradient is used to update the weights in the network so that the model's predictions are closer to the actual labels in the next iteration.

[0117] Step 7: Weight update: Use optimization algorithms (such as stochastic gradient descent SGD, Adam, RMSprop, etc.) to update the weights and biases of the network according to the calculated gradients.

[0118] Step 8: Verification and testing: During the training process, the validation set is used to monitor the model performance to prevent overfitting. Finally, the model is evaluated on the test set to measure the generalization ability of the model on unseen data.

[0119] Furthermore, the mask generation process includes the following steps:

[0120] Step 1: Call FCN, U-Net and other models to segment the image to be desensitized into several image segmentation regions, and then identify the corresponding sensitive areas in each image segmentation region according to the label of the target desensitized content.

[0121] Step 2: Generate a corresponding mask based on the shape and size of the sensitive area.

[0122] This embodiment provides the following mask pattern generation methods: ① Rectangular mask: Generate a rectangular mask of the same size as the sensitive area and cover the sensitive area; ② Circular mask: Generate a circular pattern of the same size as the sensitive area and cover the sensitive area; ③ Custom mask: Generate an irregular pattern of the same size as the sensitive area and cover the sensitive area.

[0123] The mask generation formula used is as follows:

[0124]

[0125] Where M represents the final mask, K represents the total number of sensitive areas, and s k Indicates the coverage intensity of the kth sensitive area, which is used to control the coverage effect. k represents the mask pattern corresponding to the kth sensitive area, and shape represents the shape of the mask.

[0126] For each pixel (x, y) in the mask M, M k The value of is determined by the following formula:

[0127]

[0128] Where σ represents an activation function, such as Sigmoid, which is used to convert the model output into probability; f k (x, y) represents the output of the model at the position (x, y) of the kth sensitive area; τ k The threshold value of the k-th sensitive area is used to determine whether the corresponding position in the mask pattern is covered.

[0129] The coverage intensity can be divided into 4 levels according to the sensitivity of the area. If the area does not need to be covered, it is level 1; if the area is a low-sensitivity area, it is level 2; if the area is a medium-sensitivity area, it is level 3; if the area is a high-sensitivity area, it is level 4. At the same time, the coverage effect will also vary with the sensitivity level. It is divided by the completeness of coverage. The coverage completeness increases step by step from level 2 to 3, and level 4 directly covers the sensitive area.

[0130] Step 3: Cover the sensitive area of ​​the image to be desensitized with the mask pattern to mark the sensitive content and generate the final mask.

[0131] Among them, the mask pattern coverage includes the following three methods:

[0132] ① Overlay marking: directly overlay the mask pattern onto the image to be desensitized;

[0133] ② Mixed marking: Mix the mask pattern with the image to be desensitized, for example, using an alpha channel for mixing;

[0134] ③Transparency adjustment: Adjust the transparency of the mask pattern so that it can cover sensitive content while retaining some image details, making it easier for users to view the sensitive information to be removed.

[0135] Step 4: Call a preset coverage effect evaluation function to determine the coverage effect evaluation value corresponding to the image to be desensitized and the mask.

[0136] The preset coverage effect evaluation function is expressed as follows:

[0137] E(I,M)=γSSIM(I,I⊙M)+δPSNR(I,I⊙M)

[0138] Step 5: Dynamically adjust the mask based on the coverage effect evaluation value.

[0139] The coverage effect influencing parameters such as mask transparency, image segmentation accuracy, mask shape, and coverage intensity of sensitive areas can be preset. By minimizing the coverage effect evaluation value E(I,M) as the goal, the coverage effect of the mask can be dynamically adjusted to regenerate the optimal mask M. * .

[0140] Furthermore, the user can also manually adjust the mask through the user interface.

[0141] S360: Input the image to be desensitized and the mask into the LaMa image elimination model to obtain an image with sensitive content missing.

[0142] In the embodiment of the present invention, the LaMa image removal model adopted in the embodiment can achieve accurate removal of objects by using fast Fourier convolution and a larger training mask.

[0143] S370: Based on the fidelity required by the user, call the preset image loop repair model to repair the image with missing sensitive content and generate a target desensitized image.

[0144] In an embodiment of the present invention, the preset image loop repair model may include an input layer, an encoder, a decoder, and an output layer. The input layer is used to receive the sensitive content missing image I Input (H×W×C, H and W are the height and width of the image, and C is the number of channels) and the corresponding binary mask M(H×W), where the pixel value of the missing area is 0 and the pixel value of the valid area is 1.

[0145] The encoder includes multiple convolutional layers and pooling layers, which are used to perform multi-layer downsampling operations on the input sensitive content missing images to extract high-level features of the image layer by layer, including edges, textures, and shapes. Among them, the convolutional layer uses a 3×3 convolution kernel for feature extraction. 3×3 is the most commonly used convolution kernel size because it can capture local features while requiring relatively little computation. The combination of multiple 3×3 convolutional layers can simulate the effect of a larger convolution kernel, making the convolutional layer more flexible. The activation function uses the ReLU activation function. The pooling layer uses a size of 2×2, which can effectively reduce the spatial size of the feature map while maintaining sufficient information.

[0146] The decoder includes multiple deconvolution layers and pooling layers, which are used to reconstruct the low-level features of the image layer by layer, including pixel-level details and restore the spatial dimension of the image through upsampling operations, so that the output image has the same resolution as the input image. Among them, the deconvolution layer uses a 3×3 transposed convolution kernel for feature extraction. 3×3 is the most commonly used transposed convolution kernel size because it can capture local features while requiring relatively little computation. In addition, the combination of multiple 3×3 transposed convolution layers can simulate the effect of a larger transposed convolution kernel, which improves the flexibility of the deconvolution layer.

[0147] In this embodiment, the preset image cyclic restoration model also adopts a dynamic boundary extension mechanism, which is expressed as follows:

[0148]

[0149] Where W(t,F) represents the width of the repair boundary at the tth iteration; t represents the current iteration number; F represents the user demand fidelity, which ranges from 0≤F≤3; W max Indicates the maximum repair boundary width; W min Indicates the minimum repair boundary width; T max is the maximum number of model iterations, and T max =T·F, T is the number of initial iterations of the preset model.

[0150] ω(F) represents a tuning parameter used to control the speed at which the width of the repair boundary changes with the number of iterations, and can be defined as: ω(F) = 1 + (k-1) (1-F), where k is a constant used to control the range of the change speed. Typically, k can be set to a value between 2 and 4 to suit different application scenarios.

[0151] Secondly, the preset image loop restoration model also adopts an iterative control mechanism, which consists of two parts of control logic. The first part is the iteration termination condition, which includes two conditions: one is that the number of model iterations reaches the maximum number of model iterations T. max, the second is that the missing area is completely repaired, that is, all pixel values ​​in the mask M are 0; the second part is the user required fidelity F specified by the user, ranging from 0 to 3, where 0 represents the lowest fidelity and 3 represents the highest fidelity. The user required fidelity F will affect the maximum number of model iterations and the width of the repair boundary of each iteration.

[0152] The output layer receives the feature map output and mask from the last layer of the decoder. During the processing of the output layer, a convolutional layer generates a prediction of the repaired area, and uses the Sigmoid activation function to limit the output between 0 and 1. The prediction result is then combined with the image lacking sensitive content to generate the repaired target repair image. The pixel values ​​in the repaired area are updated to the values ​​predicted by the model, and other areas remain unchanged. Through the output layer, the model can combine the feature map generated by the decoder with the mask of the repaired area to generate the repaired target repair image.

[0153] Furthermore, the definition steps of the preset image loop restoration model are as follows:

[0154] Step 1: Define an encoder to extract high-level features of images with missing sensitive content. Set the convolution layer to use a 3×3 convolution kernel for feature extraction, the activation function uses the ReLU activation function, and the pooling layer uses a 2×2 maximum pooling layer to reduce the spatial resolution.

[0155] Step 2: Define a decoder to gradually restore high-level features to the spatial resolution of the original sensitive content missing image, and set the deconvolution layer to upsample using a 3×3 transposed convolution kernel.

[0156] Step 3: Define the output layer to combine the feature map generated by the decoder with the mask to generate the repaired target image.

[0157] Step 4: Define the iteration control mechanism to control the iteration situation and set the maximum number of model iterations T according to the user's required fidelity F. max .

[0158] Furthermore, the definition steps of the preset image loop restoration model are as follows:

[0159] Step 1: Define the loss function and corresponding optimization.

[0160] Step 2: Start the training loop, which includes: passing the input image and mask to the model, performing forward propagation to obtain the repaired image; using the loss function to calculate the loss between the model output and the target image; performing backpropagation to calculate the gradient and update the model parameters; recording the loss of each training cycle (epoch) to monitor the training process.

[0161] Step 3: Start a validation loop, which includes: At the end of each training cycle, use the validation set to evaluate the performance of the model; record the loss on the validation set to monitor the generalization ability of the model.

[0162] Furthermore, the image restoration steps of the preset image cycle restoration model are as follows:

[0163] Step 1: When the model is iterated for the first time, the model input layer receives the sensitive content missing image after the LaMa image elimination model eliminates sensitive information and the mask generated by the preset dynamic mask generation model. At the same time, the maximum number of model iterations is set according to the fidelity required by the user, so that the model can perform corresponding iterative operations according to the fidelity required by the user. If it is not the first iteration, the image and mask output by the output layer at the completion of the previous iteration are received.

[0164] Step 2: Pass the received sensitive content missing image to the encoder to extract high-level features of the sensitive content missing image.

[0165] Step 3: Pass the high-level features extracted by the encoder to the decoder, and gradually restore the spatial resolution of the original image with missing sensitive content.

[0166] Step 4: The output layer combines the feature map generated by the decoder with the mask of the repaired area to generate the repaired target image.

[0167] Step 5: Repeat steps 1 to 4 until the iteration situation meets the maximum number of model iterations or the image restoration is completed to generate the final target desensitized image.

[0168] S380: Display the target desensitized image to the user, and receive image desensitization feedback instructions from the user.

[0169] S390: Determine whether the user needs to make feedback modifications.

[0170] S3100: The user manually selects sensitive content in the image, and the process returns to S350.

[0171] S3110. The user reselects the user requirement fidelity and returns to execute S340.

[0172] The technical solution of the embodiment of the present invention can realize human-computer dialogue and reduce the difficulty of using the system. The method supports the generation of dynamic masks, locates sensitive data more accurately, and realizes high-fidelity desensitization, retaining the details of the desensitized image as much as possible. In addition, the method can also dynamically adjust the number of image repair iterations according to the user's required fidelity specified by the user, while ensuring that the image fidelity meets the requirements, saving computing power resources, realizing high-fidelity desensitization processing of sensitive information in the image, and improving the image desensitization effect and accuracy.

[0173] Embodiment 4

[0174] Figure 4 This is a schematic diagram of the structure of an image desensitization device provided in Embodiment 4 of the present invention. Figure 4 As shown, the device comprises:

[0175] The desensitized content determination module 41 is used to obtain the to-be-desensitized image uploaded by the user and the fidelity required by the user, and determine the target desensitized content in the to-be-desensitized image;

[0176] The mask generation module 42 is used to generate a mask corresponding to the image to be desensitized according to the corresponding sensitive area of ​​the target desensitization content in the image to be desensitized;

[0177] The sensitive content elimination module 43 is used to input the image to be desensitized and the mask into a preset sensitive content elimination model to obtain an image lacking sensitive content;

[0178] The image restoration module 44 is used to restore the image with missing sensitive content according to the fidelity required by the user to obtain a target desensitized image.

[0179] The technical solution of the embodiment of the present invention obtains the image to be desensitized uploaded by the user and the user required fidelity through the desensitization content determination module, and determines the target desensitized content in the image to be desensitized; the mask generation module generates a mask corresponding to the image to be desensitized according to the corresponding sensitive area of ​​the target desensitized content in the image to be desensitized; the sensitive elimination module inputs the image to be desensitized and the mask into a preset sensitive content elimination model to obtain a sensitive content missing image; the image repair module repairs the sensitive content missing image according to the user required fidelity to obtain the target desensitized image. This technical solution eliminates the sensitive information of the image to be desensitized, and then repairs the sensitive content missing image according to the user required fidelity specified by the user to obtain the corresponding target desensitized image, thereby solving the problem of loss of image details and poor image quality after image desensitization, and realizes high-fidelity desensitization processing of sensitive information in the image, thereby improving the image desensitization effect and accuracy.

[0180] Further, based on the above-mentioned embodiment of the invention, the desensitized content determination module 41 includes:

[0181] A text information extraction unit, used to receive the image to be desensitized uploaded by the user, and call a preset information extraction model to extract text information from the image to be desensitized; wherein the preset information extraction model at least includes an optical character recognition model;

[0182] A candidate desensitized content identification unit is used to input text information into a preset business language model to obtain candidate desensitized content;

[0183] A desensitization instruction receiving unit, used to display the candidate desensitization content and the desensitization suggestions corresponding to the candidate desensitization content to the user, and receive the desensitization instruction fed back by the user;

[0184] The instruction extraction unit is used to extract the user demand fidelity and target desensitized content from the desensitized instructions.

[0185] Further, based on the above-mentioned embodiment of the invention, the mask generation module 42 includes:

[0186] A semantic segmentation unit, used for inputting the image to be desensitized into an image segmentation module of a preset dynamic mask generation model to perform semantic segmentation on the image to be desensitized to obtain an image segmentation region;

[0187] A sensitive area identification unit, used to identify a sensitive area corresponding to the label of the target desensitization content in each image segmentation area according to the label of the target desensitization content;

[0188] The mask generation unit is used to input each sensitive area into the mask generation module of the preset dynamic mask generation model to generate a mask corresponding to the image to be desensitized.

[0189] Further, based on the above-mentioned embodiment of the invention, the mask generating unit is further used for:

[0190] The image to be desensitized and the mask are used as inputs of a preset coverage effect evaluation function to obtain a coverage effect evaluation value;

[0191] Based on the preset coverage effect influencing parameters, the mask is optimized and adjusted to obtain a new mask with the goal of minimizing the coverage effect evaluation value; wherein the preset coverage effect influencing parameters include: mask transparency, image segmentation accuracy, mask shape, and coverage intensity of sensitive areas.

[0192] Further, based on the above-mentioned embodiment of the invention, the image restoration module 44 includes:

[0193] A maximum model iteration number determination unit, used to take the product of the user requirement fidelity and the preset model initial iteration number as the maximum model iteration number;

[0194] A high-level feature extraction unit, used for inputting the sensitive content missing image into an encoder of a preset image loop repair model to extract high-level features of the sensitive content missing image;

[0195] A feature map generation unit, used to input high-level features into a decoder of a preset image cyclic restoration model to generate a feature map corresponding to the high-level features;

[0196] A target inpainting image generating unit is used to input the feature map and the mask into the output layer of the preset image cyclic inpainting model for image fusion to obtain a target inpainting image, and to update the number of iterations of the preset image cyclic inpainting model;

[0197] An iteration number judgment unit, used to judge whether the updated iteration number reaches the maximum model iteration number;

[0198] The processing execution unit is used to use the target repaired image as the final target desensitized image if yes; if not, use the target repaired image as the input of the encoder of the preset image loop repair model, and repeatedly execute the above image repair process until the number of iterations reaches the maximum number of model iterations.

[0199] Furthermore, based on the above-mentioned embodiment of the invention, the image desensitization device further includes:

[0200] An image desensitization feedback instruction receiving module is used to display the target desensitized image to the user after obtaining the target desensitized image, and receive the image desensitization feedback instruction fed back by the user;

[0201] The first feedback execution module is used for returning to the step of generating a mask corresponding to the image to be desensitized according to the corresponding sensitive area of ​​the target desensitized content in the image to be desensitized when the image desensitization feedback instruction is to select the specified sensitive content as the target desensitized content;

[0202] A second feedback execution module is used to return to the step of obtaining the user requirement fidelity uploaded by the user and determining the target desensitized content in the image to be desensitized when the image desensitization feedback instruction is to reselect the user requirement fidelity;

[0203] The third feedback execution module is used to end the image desensitization process when the image desensitization feedback instruction indicates that no modification is required.

[0204] Further, based on the above-mentioned embodiments of the invention, the preset sensitive content elimination model includes: a LaMa image elimination model.

[0205] The image desensitization device provided in the embodiment of the present invention can execute the image desensitization method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0206] Embodiment 5

[0207] Figure 5A schematic diagram of an electronic device 50 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0208] like Figure 5 As shown, the electronic device 50 includes at least one processor 51, and a memory connected to the at least one processor 51 in communication, such as a read-only memory (ROM) 52, a random access memory (RAM) 53, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 51 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 52 or the computer program loaded from the storage unit 58 to the random access memory (RAM) 53. In the RAM 53, various programs and data required for the operation of the electronic device 50 can also be stored. The processor 51, the ROM 52, and the RAM 53 are connected to each other via a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0209] A number of components in the electronic device 50 are connected to the I / O interface 55, including: an input unit 56, such as a keyboard, a mouse, etc.; an output unit 57, such as various types of displays, speakers, etc.; a storage unit 58, such as a disk, an optical disk, etc.; and a communication unit 59, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 59 allows the electronic device 50 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0210] The processor 51 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 51 performs the various methods and processes described above, such as the image desensitization method.

[0211] In some embodiments, the image desensitization method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 58. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 50 via the ROM 52 and / or the communication unit 59. When the computer program is loaded into the RAM 53 and executed by the processor 51, one or more steps of the image desensitization method described above may be performed. Alternatively, in other embodiments, the processor 51 may be configured to perform the image desensitization method in any other appropriate manner (e.g., by means of firmware).

[0212] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0213] In some embodiments, the image desensitization method may be implemented as a computer program, which is invisibly included in a computer program product. When the computer program is executed by a processor, the image desensitization method of the present invention is implemented. The computer program product can be understood as a software product that implements its solution mainly through a computer program. The computer program for implementing the method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flowchart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine as an independent software package and partially on a remote machine, or entirely on a remote machine or server.

[0214] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0215] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0216] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0217] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0218] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0219] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image desensitization method, characterized in that: The method comprises: Obtaining the to-be-desensitized image uploaded by the user and the fidelity required by the user, and determining the target desensitized content in the to-be-desensitized image; Generate a mask corresponding to the image to be desensitized according to a corresponding sensitive area of ​​the target desensitization content in the image to be desensitized; Inputting the image to be desensitized and the mask into a preset sensitive content elimination model to obtain an image lacking sensitive content; The image with missing sensitive content is repaired according to the fidelity required by the user to obtain a target desensitized image.

2. The method according to claim 1, characterized in that The obtaining of the to-be-desensitized image uploaded by the user and the fidelity required by the user, and determining the target desensitized content in the to-be-desensitized image, includes: Receive the image to be desensitized uploaded by the user, and call a preset information extraction model to extract text information in the image to be desensitized; wherein the preset information extraction model at least includes an optical character recognition model; Inputting the text information into a preset business language model to obtain candidate desensitized content; Displaying the candidate desensitized content and the desensitization suggestions corresponding to the candidate desensitized content to the user, and receiving the desensitization instructions fed back by the user; The user requirement fidelity and the target desensitized content are extracted from the desensitization instruction.

3. The method according to claim 1, characterized in that The step of generating a mask corresponding to the image to be desensitized according to a corresponding sensitive area of ​​the target desensitized content in the image to be desensitized includes: Inputting the image to be desensitized into an image segmentation module of a preset dynamic mask generation model to perform semantic segmentation on the image to be desensitized to obtain image segmentation areas; According to the label of the target desensitized content, identifying the sensitive area corresponding to the label of the target desensitized content in each of the image segmentation areas; Each of the sensitive areas is input into the mask generation module of the preset dynamic mask generation model to generate the mask corresponding to the image to be desensitized.

4. The method according to claim 3, characterized in that The step of inputting each of the sensitive areas into the mask generation module of the preset dynamic mask generation model to generate the mask corresponding to the image to be desensitized further includes: Using the image to be desensitized and the mask as inputs of a preset coverage effect evaluation function to obtain a coverage effect evaluation value; Based on preset coverage effect influencing parameters, the mask is optimized and adjusted with the goal of minimizing the coverage effect evaluation value to obtain a new mask; wherein the preset coverage effect influencing parameters include: mask transparency, image segmentation accuracy, mask shape, and coverage intensity of sensitive areas.

5. The method according to claim 1, characterized in that The step of repairing the image lacking sensitive content according to the fidelity required by the user to obtain a target desensitized image includes: The product of the user demand fidelity and the preset model initial iteration number is taken as the maximum model iteration number; Inputting the sensitive content missing image into an encoder of a preset image loop repair model to extract high-level features of the sensitive content missing image; Inputting the high-level features into a decoder of the preset image cyclic restoration model to generate a feature map corresponding to the high-level features; Inputting the feature map and the mask into the output layer of the preset image cyclic restoration model for image fusion to obtain a target restoration image, and updating the number of iterations of the preset image cyclic restoration model; Determine whether the updated number of iterations reaches the maximum number of model iterations; If so, the target repaired image is used as the final target desensitized image; if not, the target repaired image is used as the input of the encoder of the preset image loop repair model, and the above image repair process is repeatedly executed until the number of iterations reaches the maximum number of model iterations.

6. The method according to claim 1, characterized in that After obtaining the target desensitized image, the method further includes: Displaying the target desensitized image to the user, and receiving an image desensitization feedback instruction from the user; When the image desensitization feedback instruction is to select the specified sensitive content as the target desensitization content, returning to the step of generating a mask corresponding to the image to be desensitized according to the sensitive area corresponding to the target desensitization content in the image to be desensitized; When the image desensitization feedback instruction is to reselect the user required fidelity, returning to the step of obtaining the user required fidelity uploaded by the user, and determining the target desensitized content in the image to be desensitized; When the image desensitization feedback instruction indicates that no modification is required, the image desensitization process is terminated.

7. The method according to claim 1, characterized in that The preset sensitive content elimination model includes: a LaMa image elimination model.

8. An image desensitization device, characterized in that: The device comprises: A desensitized content determination module is used to obtain the to-be-desensitized image uploaded by the user and the fidelity required by the user, and determine the target desensitized content in the to-be-desensitized image; A mask generation module, used for generating a mask corresponding to the image to be desensitized according to the corresponding sensitive area of ​​the target desensitization content in the image to be desensitized; A sensitive content elimination module, used for inputting the image to be desensitized and the mask into a preset sensitive content elimination model to obtain an image lacking sensitive content; The image restoration module is used to restore the image with missing sensitive content according to the fidelity required by the user to obtain a target desensitized image.

9. The device according to claim 8, characterized in that The desensitized content determination module includes: A text information extraction unit, used to receive the image to be desensitized uploaded by the user, and call a preset information extraction model to extract text information from the image to be desensitized; wherein the preset information extraction model at least includes an optical character recognition model; A candidate desensitized content identification unit, used for inputting the text information into a preset business large language model to obtain candidate desensitized content; A desensitization instruction receiving unit, used to display the candidate desensitization content and the desensitization suggestion corresponding to the candidate desensitization content to the user, and receive the desensitization instruction fed back by the user; An instruction extraction unit is used to extract the user demand fidelity and the target desensitized content from the desensitization instruction.

10. The device according to claim 8, characterized in that The mask generation module comprises: A semantic segmentation unit, used for inputting the image to be desensitized into an image segmentation module of a preset dynamic mask generation model to perform semantic segmentation on the image to be desensitized to obtain an image segmentation region; A sensitive area identification unit, configured to identify, in each of the image segmentation regions, the sensitive area corresponding to the label of the target desensitization content according to the label of the target desensitization content; The mask generation unit is used to input each of the sensitive areas into the mask generation module of the preset dynamic mask generation model to generate the mask corresponding to the image to be desensitized.

11. The device according to claim 10, characterized in that The mask generating unit is further used for: Using the image to be desensitized and the mask as inputs of a preset coverage effect evaluation function to obtain a coverage effect evaluation value; Based on preset coverage effect influencing parameters, the mask is optimized and adjusted with the goal of minimizing the coverage effect evaluation value to obtain a new mask; wherein the preset coverage effect influencing parameters include: mask transparency, image segmentation accuracy, mask shape, and coverage intensity of sensitive areas.

12. The device according to claim 8, characterized in that The image restoration module comprises: A maximum model iteration number determination unit, used to take the product of the user demand fidelity and the preset model initial iteration number as the maximum model iteration number; A high-level feature extraction unit, used for inputting the sensitive content missing image into an encoder of a preset image loop repair model to extract high-level features of the sensitive content missing image; A feature map generating unit, used for inputting the high-level features into a decoder of the preset image cyclic restoration model to generate a feature map corresponding to the high-level features; A target inpainting image generating unit, used for inputting the feature map and the mask into the output layer of the preset image cyclic inpainting model for image fusion, obtaining a target inpainting image, and updating the number of iterations of the preset image cyclic inpainting model; An iteration number determination unit, used to determine whether the updated iteration number reaches the maximum model iteration number; A processing execution unit is used to use the target repaired image as the final target desensitized image if yes; if not, use the target repaired image as the input of the encoder of the preset image loop repair model, and repeatedly execute the above-mentioned image repair process until the number of iterations reaches the maximum number of model iterations.

13. The device according to claim 8, characterized in that Also includes: An image desensitization feedback instruction receiving module is used to display the target desensitized image to the user after obtaining the target desensitized image, and receive the image desensitization feedback instruction fed back by the user; A first feedback execution module is used for returning to execute the step of generating a mask corresponding to the image to be desensitized according to a corresponding sensitive area of ​​the target desensitized content in the image to be desensitized when the image desensitization feedback instruction is to select a specified sensitive content as the target desensitized content; A second feedback execution module is used for returning to the step of obtaining the user requirement fidelity uploaded by the user and determining the target desensitized content in the image to be desensitized when the image desensitization feedback instruction is to reselect the user requirement fidelity; The third feedback execution module is used to end the image desensitization process when the image desensitization feedback instruction does not need to be modified.

14. The device according to claim 8, characterized in that The preset sensitive content elimination model includes: a LaMa image elimination model.

15. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the image desensitization method described in any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image desensitization method described in any one of claims 1 to 7 when executed.

17. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the image desensitization method according to any one of claims 1 to 7.

Images