Image processing method and device, equipment, storage medium and program product

Abstract

The invention discloses an image processing method and device, equipment, a storage medium and a program product. The method comprises the steps of obtaining a to-be-processed image generated in a business handling process; the to-be-processed image is input into a predetermined target student network model, a privacy prediction result of the to-be-processed image is determined according to output of the target student network model, and the target student network model is generated through knowledge distillation; if it is determined that the to-be-processed image has privacy information according to the privacy prediction result, the to-be-processed image is detected, the to-be-desensitized area is determined, and desensitization processing is performed on the to-be-desensitized area, so that the problem of user privacy disclosure is solved, and a target student network model is generated through knowledge distillation in advance; the to-be-processed image is identified through the target student network model, and privacy detection of the to-be-processed image is realized; according to the method, the to-be-desensitized area is determined, and the to-be-desensitized area is desensitized, so that the to-be-processed image is prevented from leaking privacy data, the user privacy is protected, and the service security is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, device, equipment, storage medium and program product. Background Art

[0002] Financial services usually involve the privacy information of users, and the rapid development of online services poses many risks of privacy leakage, which may cause property losses to users. Therefore, how to protect the privacy information of users and improve the security of services has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present invention provides an image processing method, device, equipment, storage medium and program product to solve the problem of user privacy leakage, protect user privacy and improve business security.

[0004] According to one aspect of the present invention, there is provided an image processing method, comprising:

[0005] Obtaining the images to be processed generated during the business handling process;

[0006] Inputting the image to be processed into a predetermined target student network model, and determining a privacy prediction result of the image to be processed according to an output of the target student network model, wherein the target student network model is generated by knowledge distillation;

[0007] If it is determined according to the privacy prediction result that the image to be processed contains privacy information, the image to be processed is detected, the area to be desensitized is determined, and the desensitization processing is performed on the area to be desensitized.

[0008] According to another aspect of the present invention, there is provided an image processing apparatus, comprising:

[0009] The module for acquiring images to be processed is used to acquire images to be processed generated during the business handling process;

[0010] A privacy prediction module, used for inputting the image to be processed into a predetermined target student network model, and determining a privacy prediction result of the image to be processed according to an output of the target student network model, wherein the target student network model is generated by knowledge distillation;

[0011] The desensitization module is used to detect the image to be processed, determine the area to be desensitized, and perform desensitization processing on the area to be desensitized if it is determined that the image to be processed has privacy information according to the privacy prediction result.

[0012] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0013] at least one processor, and a memory communicatively coupled to the at least one processor;

[0014] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image processing method described in any embodiment of the present invention.

[0015] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image processing method described in any embodiment of the present invention when executed.

[0016] According to another aspect of the present invention, a computer program product is provided. The computer program product comprises a computer program. When the computer program is executed by a processor, the image processing method according to any embodiment of the present invention is implemented.

[0017] The technical solution of the embodiment of the present invention is as follows: by obtaining an image to be processed generated during a business handling process; inputting the image to be processed into a predetermined target student network model, and determining a privacy prediction result of the image to be processed according to the output of the target student network model, wherein the target student network model is generated by knowledge distillation; if it is determined according to the privacy prediction result that the image to be processed has privacy information, the image to be processed is detected, the area to be desensitized is determined, and the area to be desensitized is desensitized, thereby solving the problem of user privacy leakage, and pre-generating a target student network model by knowledge distillation; the image to be processed generated during a business handling process is identified by the target student network model, and the privacy prediction result of the image to be processed is determined according to the output of the target student network model, thereby realizing privacy detection of the image to be processed; if it is determined according to the privacy prediction result that the image to be processed has privacy information, the image to be processed is detected, the area to be desensitized is determined, and the area to be desensitized is desensitized, thereby avoiding leakage of privacy data of the image to be processed, protecting user privacy, and improving business security.

[0018] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 is a flowchart of an image processing method provided according to Embodiment 1 of the present invention;

[0021] Figure 2 is a flowchart of an image processing method provided according to Embodiment 2 of the present invention;

[0022] Figure 3 is a structural schematic diagram of an image processing device provided according to Embodiment 3 of the present invention;

[0023] Figure 4 It is a schematic diagram of the structure of an electronic device for implementing the image processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] Embodiment 1

[0027] Figure 1This is a flowchart of an image processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to the case of processing an image. The method can be executed by an image processing device. The image processing device can be implemented in the form of hardware and / or software. The image processing device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0028] S101: Acquire an image to be processed generated during a business handling process.

[0029] In this embodiment, the image to be processed is an image with privacy processing requirements. The image to be processed can be an image collected by the user in the process of handling business, and can be an image including the user's face, body and other parts, or an image including the user's mobile phone number, ID number, address, signature and other private data. For example, different image acquisition devices are set up in bank outlets, and different image acquisition devices can collect images of different areas, such as collecting images of counters, cash machines, entrances and exits of outlets and other locations. The image acquisition device can be triggered to collect images according to the set cycle or trigger conditions to collect images of the corresponding area; when the user is handling business, the image acquisition device in the area where the user is located can collect corresponding images, and the collected images may include the user's face, body and other private data. Alternatively, when the user is handling business or generates a corresponding business form, the business form needs to be retained as a voucher, and the image acquisition device can collect the corresponding image of the business form so as to save it as an electronic voucher. The business form may include the user's mobile phone number, ID number, address, signature and other private data. Therefore, the collected image may include the user's mobile phone number, ID number, address, signature and other private data.

[0030] The images collected by the image acquisition device can be saved to a corresponding storage space such as a database or cloud. The execution device reads the corresponding image when the data processing conditions are met and uses it as the image to be processed. The data processing conditions can be preset, for example, the number of saved images exceeds the number threshold, or the time difference between the current time and the last processing time meets the time difference threshold (for example, periodic processing), or the user's trigger instruction / operation is received, etc. After the image is collected, it can also be transmitted to a terminal or other device in real time for processing. The execution device can receive the image collected and transmitted by the image acquisition device in real time and use the received image as the image to be processed.

[0031] S102, inputting the image to be processed into a predetermined target student network model, determining the privacy prediction result of the image to be processed according to the output of the target student network model, and the target student network model is generated by knowledge distillation.

[0032] In this embodiment, the target student network model can be understood as a student network model generated in advance through knowledge distillation training. The privacy prediction result can be understood as the result obtained by predicting whether the processed image contains private data, for example, whether the image contains private data or not, and if it contains private data, it can also include the location of the private data, etc.

[0033] Knowledge distillation adopts the Teacher-Student model, with a complex and large model as the Teacher and a relatively simple Student model structure. The Teacher is used to assist the training of the Student model. The Teacher has strong learning ability and can transfer the knowledge it has learned to the Student model with relatively weak learning ability, thereby enhancing the generalization ability of the Student model. Knowledge distillation is the transfer of model capabilities. According to different transfer methods, it can be simply divided into two major directions: target-based distillation (also called soft-target distillation or logits method distillation) and feature-based distillation algorithms.

[0034] Pre-build the teacher network model and the student network model, collect training samples, train the teacher network model through the training samples, and perform knowledge distillation learning on the student network model based on the trained teacher network model to obtain a target student network model that meets the requirements and complete the model training. Input the image to be processed into the predetermined target student network model, and the target student network model detects and identifies the image to be processed based on the learned knowledge, determines whether the image to be processed has privacy and outputs it, and determines the privacy prediction result of the image to be processed based on the output of the target student network model.

[0035] S103: If it is determined according to the privacy prediction result that the image to be processed contains privacy information, the image to be processed is detected, a region to be desensitized is determined, and desensitization is performed on the region to be desensitized.

[0036] In this embodiment, the area to be desensitized can be understood as an area where data desensitization is required.

[0037] Analyze the privacy prediction results to determine whether the image to be processed contains privacy information; for example, if the privacy prediction result is that the image includes privacy data, it can be determined that the image to be processed contains privacy information; if the privacy prediction result is that the image does not include privacy data, it can be determined that the image to be processed does not contain privacy information. If it is determined that the image to be processed contains privacy information, detect the image to be processed, determine the area where privacy exists, and use this area as the area to be desensitized; desensitize the area to be desensitized according to one or more pre-set methods, for example, cover the area to be desensitized with a color block, erase information, etc.

[0038] The embodiment of the present invention provides an image processing method, which obtains an image to be processed generated during a business processing process; inputs the image to be processed into a predetermined target student network model, and determines a privacy prediction result of the image to be processed according to the output of the target student network model, wherein the target student network model is generated by knowledge distillation; if it is determined according to the privacy prediction result that the image to be processed has privacy information, the image to be processed is detected, an area to be desensitized is determined, and the area to be desensitized is desensitized, thereby solving the problem of user privacy leakage, and pre-generating a target student network model by knowledge distillation; the image to be processed generated during a business processing process is identified by the target student network model, and the privacy prediction result of the image to be processed is determined according to the output of the target student network model, thereby realizing privacy detection of the image to be processed; if it is determined according to the privacy prediction result that the image to be processed has privacy information, the image to be processed is detected, an area to be desensitized is determined, and the area to be desensitized is desensitized, thereby avoiding the image to be processed from leaking privacy data, protecting user privacy, and improving business security.

[0039] Embodiment 2

[0040] Figure 2 This is a flowchart of an image processing method provided by Embodiment 2 of the present invention. This embodiment is refined on the basis of the above embodiment. Figure 2 As shown, the method includes:

[0041] S201. Acquire an image to be processed generated during a business handling process.

[0042] S202, inputting the image to be processed into a predetermined target student network model, determining the privacy prediction result of the image to be processed according to the output of the target student network model, and the target student network model is generated by knowledge distillation.

[0043] As an optional embodiment, this optional embodiment further optimizes the step of determining the target student network model, including:

[0044] A1. Obtain a set of target domain training samples, where the target domain training samples include target domain images and target privacy labels, where the target privacy labels are foreground labels and / or background labels.

[0045] In this embodiment, the target domain training sample can be understood as the training sample used to train the student network model; the target domain image can be understood as the image used to train the student network model; the target privacy label can be understood as the information for identifying the privacy included in the image. The foreground label can be understood as the information for identifying the foreground of the image; and the background label can be understood as the information for identifying the background of the image.

[0046] Pre-collect an image, use the image as the target domain image and identify the privacy information in the image, and determine the target privacy label of the target domain image according to the location where the privacy information appears in the image. For example, if the privacy information appears in the background of the image, the target privacy label is the background label; if the privacy information appears in the foreground of the image, the target privacy label is identified as the foreground label; if the privacy information appears in both the foreground and the background of the image, the target privacy label is identified as the foreground label and the background label. According to the target domain image and its corresponding target privacy label, a target domain training sample is formed, and all the generated target domain training samples are saved in a set. When training the model, the set of target domain training samples is obtained according to the storage address of the set.

[0047] A2. Input the target domain image in the target domain training sample of the current iteration into the student network model to be trained to obtain the label information output by the student network model to be trained.

[0048] In this embodiment, the student network model to be trained can be understood as a student network model that needs to be trained. The label information can be understood as information used to identify privacy in the target domain image.

[0049] The student network model to be trained is constructed in advance, the target domain training samples under the current iteration are determined, the target domain images in the target domain training samples are input into the student network model to be trained, and the student network model to be trained detects and recognizes the target domain images and outputs label information.

[0050] A3. Input the target domain image in the target domain training sample under the current iteration into the predetermined target teacher network model to obtain the soft label output by the target teacher network model.

[0051] In this embodiment, the target teacher network model can be understood as a student network model that needs to be trained. The teacher network model is trained in advance to obtain a target teacher network model that meets the requirements. The target domain training sample under the current iteration is determined, and the target domain image in the target domain training sample is input into the target teacher network model. The target teacher network model detects and recognizes the target domain image and outputs a soft label. The soft label is used to identify the privacy in the target domain image. The soft label can be a foreground label and / or a background label.

[0052] Optionally, the target teacher network model is trained based on a set of predetermined source domain training samples;

[0053] The source domain training sample includes a first source domain image, a first privacy label of the first source domain image, a second source domain image, and a second privacy label of the second source domain image;

[0054] The foreground image in the first source domain image has privacy information;

[0055] The background image in the second source domain image has private information.

[0056] In this embodiment, the source domain training samples can be understood as training samples used to train the teacher network model; the first source domain image and the second source domain image can be understood as images used to train the teacher network model; the first privacy label and the second privacy label can be understood as information for identifying the privacy included in the image.

[0057] The source domain training sample includes a first source domain image and a second source domain image, wherein the foreground image in the first source domain image has privacy information, and the background image in the second source domain image has privacy information, that is, each source domain training sample contains two images, wherein the foreground image of one image has privacy information, and the background image of the other image has privacy information. The first privacy label of the first source domain image is a foreground label, which is used to identify that the foreground image in the first source domain image has privacy information; the second privacy label of the second source domain image is a background label, which is used to identify that the background image in the second source domain image has privacy information. Since each source domain training sample includes two source domain images, it can be ensured that the subsequent training of the teacher network model has a high recognition effect on both the foreground image and the background image.

[0058] The initial teacher network model is constructed in advance, and the source domain training samples are input into the teacher network model in sequence for training, and finally a target teacher network model that meets the requirements is obtained.

[0059] A4. Calculate a first loss function based on the label information and the target privacy label, calculate a second loss function based on the label information and the soft label, and fit the first loss function and the second loss function to obtain a fitted loss function.

[0060] In this embodiment, the first loss function and the second loss function are both loss functions, and the first loss function and the second loss function can be calculated in the same way. The fitting loss function can be understood as a loss function obtained by fitting multiple loss functions, which is used to adjust the parameters of the student network model and guide the optimization of the entire student network model.

[0061] A loss function expression is determined in advance, and the label information and the target privacy label are substituted into the function expression for calculation to obtain a first loss function, and the label information and the soft label are substituted into the function expression for calculation to obtain a second loss function. The first loss function and the second loss function are fitted, for example, weighted operations are performed according to corresponding weights, and a fitting loss function is obtained by fitting.

[0062] A5. Back-propagate the student network model to be trained based on the fitting loss function to obtain the student network model to be trained for the next iteration until the iterative convergence condition is met to obtain the target student network model.

[0063] The loss function is fitted to the student network model to be trained for back propagation, guiding the optimization of the student network model to be trained, and continuing to input the target domain training samples of the next iteration for network adjustment optimization until the model convergence conditions are met to obtain the target student network model. The model convergence conditions can be that the model accuracy meets the requirements or the number of training samples reaches a certain threshold, etc. The student network is guided to optimize by fitting the loss function to complete the distillation learning of the student network. The student network model obtained after the distillation learning is the target student network model, which can be used to identify privacy information in the image. The privacy prediction results output by the target student network model can be no label, foreground label, background label, foreground label and background label, respectively indicating whether there is privacy information in the image.

[0064] S203: If it is determined according to the privacy prediction result that the image to be processed has privacy information, segment the image to be processed according to a predetermined segmentation model to determine a contour curve between the foreground area and the background area.

[0065] In this embodiment, the segmentation model can be understood as a neural network model for segmenting the foreground and background in an image. The segmentation model is pre-trained. If the privacy prediction result includes a foreground label and / or a background label, it is determined that the image to be processed has privacy information, and the image to be processed is input into the segmentation model for segmentation. The segmentation model segments the foreground and background of the image to be processed based on the pre-learned knowledge, and outputs a contour curve between the foreground area and the background area. The contour curve can be represented by the coordinates of the pixel points, or by a segmented function expression, etc., which is not limited in this application.

[0066] S204 , segmenting the foreground area and the background area according to the contour curve to obtain the foreground area and the background area.

[0067] The foreground area and the background area are segmented according to the contour curve. For example, the contour curve is drawn in the image to be processed, and the different areas segmented by the contour curve are respectively used as the foreground area and the background area. The foreground and the background can be distinguished by judging the pixel values ​​of the image in different areas, or by detecting through a neural network model or algorithm to distinguish the foreground and the background. The image to be processed is segmented into the foreground area and the background area by segmentation.

[0068] S205: Determine the area to be desensitized based on the privacy prediction result in combination with the foreground area and the background area.

[0069] Based on the privacy prediction results, the area where the privacy data exists is determined. That is, according to the privacy prediction results, it is judged whether there is privacy data in the foreground area and the background area, and the area where the privacy data exists is determined as the area to be desensitized.

[0070] As an optional embodiment, this optional embodiment further optimizes the determination of the area to be desensitized based on the privacy prediction result combined with the foreground area and the background area, including:

[0071] B1. If the privacy prediction result includes a foreground prediction label, the foreground area is used as the area to be desensitized.

[0072] In this embodiment, the foreground prediction label can be understood as a label for privacy identification of the foreground area, which is used to indicate that there is privacy data in the foreground area. If the privacy prediction result includes the foreground prediction label, it is considered that there is privacy data in the foreground area, and the foreground area is used as the area to be desensitized.

[0073] B2. If the privacy prediction result includes a background prediction label, the background area is used as the area to be desensitized.

[0074] In this embodiment, the background prediction label can be understood as a label for privacy identification of the background area, which is used to indicate that there is privacy data in the background area. If the privacy prediction result includes the background prediction label, it is considered that there is privacy data in the background area, and the background area is used as a desensitized area.

[0075] B3. If the privacy prediction result includes foreground prediction labels and background prediction labels, the foreground area and background area are taken as areas to be desensitized.

[0076] If the privacy prediction result includes a foreground prediction label and a background prediction label, it is considered that both the foreground area and the background area contain privacy data, and both the foreground area and the background area are regarded as areas to be desensitized.

[0077] A segmentation model is used to determine the contour curve between the detected foreground area and the background area. There must be a turning point between the foreground area and the background area, and there is also a change in the curve trend near the turning point. The embodiment of the present application can also use this change to obtain a vector by connecting two pixel points, and then calculate the angle between the vectors to determine whether their size meets the corresponding conditions to achieve the segmentation of the foreground area and the background area.

[0078] S206, performing desensitization processing on the area to be desensitized.

[0079] Optionally, performing desensitization processing on the area to be desensitized includes at least one of the following:

[0080] Erase sensitive information from the desensitized area;

[0081] Cover the area to be desensitized with a set color block;

[0082] The desensitized area is processed by fuzzy algorithm;

[0083] Replace the pixels in the desensitized area.

[0084] To erase sensitive information in the desensitized area, the sensitive area in the image can be erased directly, which is simple and direct. During the erasing process, the overall coordination of the image is considered.

[0085] The area to be desensitized can be covered by a set color block. The color, size, shape, etc. of the color block can be pre-set. When one color block cannot cover the area to be desensitized, multiple color blocks can be generated to ensure that the area to be desensitized can be completely covered; or color blocks can be generated according to the size and shape of the area to be desensitized; the color of the color block can also be generated randomly, or selected according to the color of the image, the business scenario corresponding to the image, etc. Using a specific color block to cover the sensitive area can more flexibly control the desensitization effect while avoiding excessive impact on the overall image.

[0086] Preset blur algorithms, such as box blur, motion blur, etc. The blur algorithm is used to process sensitive areas so that they cannot be clearly identified. This method effectively protects sensitive information while preserving the overall structure of the image.

[0087] Replace the pixels in the area to be desensitized with other non-sensitive pixels. For example, for the pixels in the area to be desensitized, record this pixel as the first pixel, randomly select a pixel from other areas outside the area to be desensitized in the image to be processed, record it as the second pixel, and replace the first pixel with the selected second pixel; or, replace the first pixel with a set pixel value; or, through a replacement control algorithm, determine the second pixel corresponding to the first pixel, and use the pixel value of the second pixel to replace the pixel value of the first pixel. Replace all pixels in the area to be desensitized in the above manner to achieve the purpose of desensitization.

[0088] The desensitized area may be treated in one or more of the above-mentioned ways. When multiple ways are used for treatment, they may be treated in a set order or in a random order.

[0089] The present invention provides an image processing method, which uses a transfer learning algorithm to train a target student network model. Through knowledge distillation, the network can be transformed from a large network into a small network, and the performance close to that of the large network is retained, thereby improving the generalization of the model and saving computing power. The target student network model is used to identify whether there is privacy information in the image to be processed. When there is privacy information in the image to be processed, the foreground area and the background area of ​​the image to be processed are segmented, and the area to be desensitized is determined according to the predicted labels of the foreground area and the background area. One or more methods are used to desensitize the privacy information in the desensitized area, thereby improving data security and effectively protecting data privacy.

[0090] Embodiment 3

[0091] Figure 3 FIG. 1 is a schematic diagram of the structure of an image processing device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: a to-be-processed image acquisition module 31, a privacy prediction module 32 and a desensitization module 33.

[0092] The to-be-processed image acquisition module 31 is used to acquire the to-be-processed image generated during the business handling process;

[0093] A privacy prediction module 32, used for inputting the image to be processed into a predetermined target student network model, and determining a privacy prediction result of the image to be processed according to an output of the target student network model, wherein the target student network model is generated by knowledge distillation;

[0094] The desensitization module 33 is used to detect the image to be processed, determine the area to be desensitized, and perform desensitization processing on the area to be desensitized if it is determined that the image to be processed has privacy information according to the privacy prediction result.

[0095] The embodiment of the present invention provides an image processing method, which solves the problem of user privacy leakage. A target student network model is generated in advance through knowledge distillation, and the image to be processed generated in the business processing process is identified through the target student network model. The privacy prediction result of the image to be processed is determined according to the output of the target student network model, and the privacy detection of the image to be processed is realized; if it is determined according to the privacy prediction result that the image to be processed has privacy information, the image to be processed is detected, the area to be desensitized is determined, and the desensitized area is desensitized to avoid the image to be processed from leaking privacy data, protect user privacy, and improve business security.

[0096] Optionally, the device further comprises:

[0097] A set acquisition module, used to acquire a set of target domain training samples, wherein the target domain training samples include a target domain image and a target privacy label, wherein the target privacy label is a foreground label and / or a background label;

[0098] A first prediction module is used to input the target domain image in the target domain training sample in the current iteration into the student network model to be trained, and obtain the label information output by the student network model to be trained;

[0099] A second prediction module is used to input the target domain image in the target domain training sample in the current iteration into a predetermined target teacher network model to obtain a soft label output by the target teacher network model;

[0100] a loss function calculation module, configured to calculate a first loss function based on the tag information and the target privacy tag, calculate a second loss function based on the tag information and the soft tag, and fit the first loss function and the second loss function to obtain a fitting loss function;

[0101] The back propagation module is used to perform back propagation on the student network model to be trained based on the fitting loss function to obtain the student network model to be trained for the next iteration until the iterative convergence condition is met to obtain the target student network model.

[0102] Optionally, the target teacher network model is trained based on a set of predetermined source domain training samples;

[0103] The source domain training sample includes a first source domain image, a first privacy label of the first source domain image, a second source domain image, and a second privacy label of the second source domain image;

[0104] The foreground image in the first source domain image has privacy information;

[0105] The background image in the second source domain image has private information.

[0106] Optionally, the desensitization module 33 includes:

[0107] A contour curve determining unit, used to segment the image to be processed according to a predetermined segmentation model, and determine a contour curve between a foreground area and a background area;

[0108] A region segmentation unit, used for segmenting the foreground region and the background region according to the contour curve to obtain the foreground region and the background region;

[0109] A region determination unit is used to determine a region to be desensitized based on the privacy prediction result in combination with the foreground region and the background region.

[0110] Optionally, the region determination unit is specifically used to: if the privacy prediction result includes a foreground prediction label, use the foreground area as the area to be desensitized; if the privacy prediction result includes a background prediction label, use the background area as the area to be desensitized; if the privacy prediction result includes a foreground prediction label and a background prediction label, use the foreground area and the background area as the area to be desensitized.

[0111] Optionally, the desensitizing the area to be desensitized includes at least one of the following:

[0112] Erasing sensitive information from the area to be desensitized;

[0113] Covering the area to be desensitized with a set color block;

[0114] Processing the area to be desensitized by using a fuzzy algorithm;

[0115] The pixels of the area to be desensitized are replaced.

[0116] The image processing device provided by the embodiment of the present invention can execute the image processing method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0117] Embodiment 4

[0118] Figure 4 A schematic diagram of the structure of an electronic device 40 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0119] like Figure 4As shown, the electronic device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0120] A number of components in the electronic device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0121] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 performs the various methods and processes described above, such as an image processing method.

[0122] In some embodiments, the image processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the processor 41 may be configured to perform the image processing method in any other appropriate manner (e.g., by means of firmware).

[0123] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0124] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0125] An embodiment of the present invention provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the image processing method described in any embodiment of the present invention is implemented.

[0126] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0128] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0129] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0130] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0131] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: include: Obtaining the images to be processed generated during the business handling process; Inputting the image to be processed into a predetermined target student network model, and determining a privacy prediction result of the image to be processed according to an output of the target student network model, wherein the target student network model is generated by knowledge distillation; If it is determined according to the privacy prediction result that the image to be processed contains privacy information, the image to be processed is detected, the area to be desensitized is determined, and the desensitization processing is performed on the area to be desensitized.

2. The method according to claim 1, characterized in that The step of determining the target student network model includes: Acquire a set of target domain training samples, where the target domain training samples include a target domain image and a target privacy label, where the target privacy label is a foreground label and / or a background label; Inputting the target domain image in the target domain training sample of the current iteration into the student network model to be trained, and obtaining the label information output by the student network model to be trained; Inputting the target domain image in the target domain training sample in the current iteration into a predetermined target teacher network model to obtain a soft label output by the target teacher network model; Calculate a first loss function based on the label information and the target privacy label, calculate a second loss function based on the label information and the soft label, and fit the first loss function and the second loss function to obtain a fitting loss function; The student network model to be trained is back-propagated based on the fitting loss function to obtain the student network model to be trained for the next iteration, until the iterative convergence condition is met to obtain the target student network model.

3. The method according to claim 2, characterized in that The target teacher network model is trained based on a set of predetermined source domain training samples; The source domain training sample includes a first source domain image, a first privacy label of the first source domain image, a second source domain image, and a second privacy label of the second source domain image; The foreground image in the first source domain image has privacy information; The background image in the second source domain image has private information.

4. The method according to claim 1, characterized in that: The detecting the image to be processed and determining the area to be desensitized includes: Segmenting the image to be processed according to a predetermined segmentation model to determine a contour curve between a foreground area and a background area; Segment the foreground area and the background area according to the contour curve to obtain the foreground area and the background area; Based on the privacy prediction result and the foreground area and the background area, the area to be desensitized is determined.

5. The method according to claim 4, characterized in that The determining the area to be desensitized based on the privacy prediction result in combination with the foreground area and the background area includes: If the privacy prediction result includes a foreground prediction label, the foreground area is used as a to-be-desensitized area; If the privacy prediction result includes a background prediction label, the background area is used as the area to be desensitized; If the privacy prediction result includes a foreground prediction label and a background prediction label, the foreground area and the background area are used as areas to be desensitized.

6. The method according to claim 1, characterized in that The desensitizing treatment of the area to be desensitized includes at least one of the following: Erasing sensitive information from the area to be desensitized; Covering the area to be desensitized with a set color block; Processing the area to be desensitized by using a fuzzy algorithm; The pixels of the area to be desensitized are replaced.

7. An image processing device, characterized in that: include: The module for acquiring images to be processed is used to acquire images to be processed generated during the business handling process; A privacy prediction module, used for inputting the image to be processed into a predetermined target student network model, and determining a privacy prediction result of the image to be processed according to an output of the target student network model, wherein the target student network model is generated by knowledge distillation; The desensitization module is used to detect the image to be processed, determine the area to be desensitized, and perform desensitization processing on the area to be desensitized if it is determined that the image to be processed has privacy information according to the privacy prediction result.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor, and a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image processing method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the image processing method according to any one of claims 1 to 6.

Images