Image processing method, device and electronic equipment

Abstract

The present application discloses an image processing method, device and electronic device, belonging to the technical field of electronic devices. The image processing method includes: acquiring a first image; detecting a target area in the first image; wherein the target area includes: at least one text area and at least one image area; determining a privacy area according to the target area; performing privacy processing on the privacy area to obtain a processed second image.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to an image processing method, device and electronic equipment. Background Art

[0002] At present, after users obtain an image (such as a screenshot), they often share the image. However, since the image contains a lot of user privacy information, for information security reasons, this privacy information needs to be hidden before sharing. However, the existing technology cannot automatically locate the privacy information in the image and code it with one click. Users need to manually use image cropping tools to cut out the privacy area, or manually use image smearing tools to mosaic each area designed for privacy, which is time-consuming and laborious. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide an image processing method, device and electronic device, which can solve the problem that the existing technology of encoding private information in images is time-consuming and labor-intensive.

[0004] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0005] acquiring a first image;

[0006] Detecting a target area in the first image; wherein the target area includes: at least one text area and at least one image area;

[0007] Determining a privacy area according to the target area;

[0008] Perform privacy processing on the privacy area to obtain a processed second image.

[0009] In a second aspect, an embodiment of the present application provides an image processing device, including:

[0010] An acquisition module, used for acquiring a first image;

[0011] A detection module, configured to detect a target area in the first image; wherein the target area includes: at least one text area and at least one image area;

[0012] A first determining module, configured to determine a privacy area according to the target area;

[0013] The processing module is used to perform privacy processing on the privacy area to obtain a processed second image.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0017] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0018] In an embodiment of the present application, an image processing method includes: acquiring a first image; detecting a target area in the first image; wherein the target area includes: at least one text area and at least one image area; determining a privacy area according to the target area; performing privacy processing on the privacy area to obtain a processed second image. In this way, it can help users automatically process privacy information in images, save users' operation time, and improve users' operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flowchart of an image processing method according to an embodiment of the present application;

[0020] Figure 2 It is one of the structural diagrams of the network model of the embodiment of the present application;

[0021] Figure 3 It is a schematic diagram of the structure of the SPPF of an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of the structure of CSP1_x of an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of the structure of the Res unit of an embodiment of the present application;

[0024] Figure 6 Schematic diagram of a multi-branch structure of three convolution kernels of different sizes according to an embodiment of the present application;

[0025] Figure 7is a schematic diagram of a detection head according to an embodiment of the present application;

[0026] Figure 8 is a schematic diagram of another detection head according to an embodiment of the present application;

[0027] Fig. 9 is one of the schematic diagrams of the first image of an embodiment of the present application;

[0028] Fig.10 is a second schematic diagram of the first image of an embodiment of the present application;

[0029] Fig.11 is a structural block diagram of an image processing device according to an embodiment of the present application;

[0030] Fig.12 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application;

[0031] Fig.13 It is a schematic diagram of the hardware structure of the electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0033] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0034] The image processing provided by the embodiments of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0035] See also Figure 1 , the embodiment of the present application provides an image processing method, including:

[0036] Step 101, acquiring a first image;

[0037] Optionally, the first image includes a screenshot image (such as a screenshot of a chat interface, a screenshot of a user details page, etc.), a saved image, a photo, etc.

[0038] Optionally, the style of the first image includes but is not limited to: a single-screen image style, a dual-screen image style and a long image style.

[0039] Step 102, detecting a target area in the first image; wherein the target area includes: at least one text area and at least one image area;

[0040] Exemplarily, the text area includes at least one of the following: user information area (such as basic personal information, device information, behavior information, location information, etc.), chat content area, chat member nickname area, group name area, time information area; the image area includes at least one of the following: portrait area and object image area, etc. For example, the image area for the chat interface screenshot includes: avatar area, chat expression area, chat picture area.

[0041] Step 103, determining a privacy area according to the target area;

[0042] Exemplarily, for a chat interface image, the privacy areas include: a chat member nickname area, an avatar area, and a group name area.

[0043] Step 104: perform privacy processing on the privacy area to obtain a processed second image.

[0044] Exemplarily, the privacy processing includes: mosaic processing, preset image overlay processing, hiding processing, erasing processing, etc.

[0045] In this embodiment, after acquiring the first image, the first image is detected to acquire the target area in the first image; the privacy area is determined according to the target area; and the privacy area is processed. In this way, the user can automatically hide the privacy information in the first image, saving the user's operation time and improving the user's operation efficiency.

[0046] In one embodiment, in the above step 102, detecting the target area in the first image includes:

[0047] Extracting the top, middle and bottom feature maps of the first image through the backbone network of the network model;

[0048] The feature maps are spliced ​​through a feature fusion network of the network model to obtain an enhanced feature map;

[0049] The enhanced feature map is input into the head prediction network of the network model to obtain the target area; wherein the head prediction network includes two detection heads; one detection head is used to predict the at least one image area, and the other detection head is used to predict the at least one text area, and the detection frame output by each detection head includes the following three feature information: the category of the detection frame is text or image, the position of the detection frame and the confidence of the detection frame.

[0050] like Figure 2 As shown in the figure, the network model consists of three parts: backbone network (Backbone), feature fusion network (Neck) and head prediction network (Head). Specifically:

[0051] First, for the backbone network Backbone, it is mainly used to extract features of different sizes in the first image. The network structure uses the CSPdarknet53 network structure, the input size is 640*640*3, and the output is three feature vectors of different sizes: 64*80*80, 128*40*40, and 256*20*20. The network structure parameter settings are the same as yolov5n. Among them, the network unit CBL in Backbone is a basic module commonly used in convolutional neural networks. CBL consists of three parts: convolution layer (Conv), batch normalization (Batch Normalization, BN) and linear rectified function (Rectified Linear Unit, relu); SPPF in Backbone is a module that is further improved on the basis of the SPP (spatial pyramid pooling) module. The specific structure of SPPF is as follows Figure 3 As shown in the figure, Max Pool refers to the maximum pooling; CSP1_1 is used to extract multi-scale features of the top layer, CSP1_2 is used to extract multi-scale features of the bottom layer, and CSP1_3 is used to extract multi-scale features of the middle layer. Figure 4 In the figure, the structure of CSP1_x (CSP1_1, CSP1_2, CSP1_3) is shown; the residual block Res unit has a specific structure as shown in Figure 5 shown.

[0052] Second, for the feature fusion network, the Neck module fuses the multi-scale features of the top, middle and bottom layers extracted by the backbone to obtain higher-level features.

[0053] Third, for the head module, the current common target detection model YOLO outputs the regression box and confidence in the same detection head through 1*1 convolution. However, in the solution of this application, the head image and text detection are decoupled, and the classification (cls) and regression (reg, conf) in each detection head are decoupled, and a network model that can detect head images and texts at the same time is proposed.

[0054] Specifically, the head module in the present application includes two detection blocks: DetBlock1 and DetBlock2, and each block is applied to the three features of different sizes output by the Neck network.

[0055] like Figure 7 As shown, in the first detection head DetBlock1_x, 1*1 convolution is first used to reduce the dimension, and then two 3*3 convolutions are connected in parallel to decouple the classification and regression task features. Finally, 1*1 convolution is connected on the three branches to reduce the dimension to the output dimension, which is suitable for detecting relatively square avatar frames.

[0056] like Figure 8 As shown, the second detection head DetBlock2_x is designed for long text lines in this application. First, 1*1 convolution is used for dimensionality reduction, and then a two-branch convolution structure is used to decouple the classification and regression task features. Because the text lines are multi-dimensional strips of different lengths, each branch uses a 1*5 convolution and a 1*3 convolution structure and stacks them together to extract features. Finally, each branch uses 1*1 convolution to reduce the dimension to the output dimension.

[0057] Exemplarily, the network layer parameter configuration of the Head module is shown in Table 1 below.

[0058]

[0059] Table 1

[0060] In an optional embodiment, the detection frame output by each detection head includes the following three feature information: the category of the detection frame, which is text or image, the position of the detection frame, and the confidence of the detection frame.

[0061] like Figure 7 and 8In the figure, cls is of size 1*C*H*W, indicating the category to which each detection box belongs; reg is of size 1*4*H*W, indicating the position of the detection box (i.e., the position in the screenshot); conf is of size 1*1*H*W, indicating the confidence of the detection box; where C is the number of categories, which is 2 in this application (i.e., text and image); H*W indicates the output size, such as 20*20; 4 contains the detection box position information (x, y, w, h), (x, y) indicates the coordinates of the center of the box, and (w, h) indicates the width and height of the detection box.

[0062] It should be pointed out that if Figure 2 In the Neck module, after outputting the features of each size, two detection heads are connected, one detection head (DetBlock1_x) is used to output the image detection box (img box), and the other detection head (DetBlock2_x) is used to output the text detection box (tex box). It should be noted that each detection box corresponds to a text area or an image area.

[0063] It should be noted that the targets to be detected in the first image include avatars and texts. Avatars are mostly square or circular in shape, with rich and continuous regional content, while text lines are mostly horizontal long strips of varying lengths with discontinuous content. Therefore, the characteristics of these two types of targets are very different. If the existing general target detection model is used directly to detect text lines, the effect is difficult to meet the requirements, and there are few existing target detection models that can directly detect these two types of targets at the same time. In view of this situation, in the above embodiments of the present application, the target detection of text and avatar categories is decoupled, and a network model that can detect avatars and text at the same time is proposed.

[0064] Optionally, in the post-processing part of the network model, NMS (non-maximum suppression) can be used to filter candidate boxes in the derivation stage to obtain the final detection result.

[0065] In an optional embodiment, the feature fusion network includes an FPN structure and a PAN structure connected in sequence; the cross-stage local network CspNet in the FPN structure and the PAN structure is replaced by a multi-branch structure (MutilBranchBlock), and each multi-branch structure includes three convolution kernels of different sizes; the feature fusion network of the network model splices the feature map to obtain an enhanced feature map, including:

[0066] like Figure 2 In the above example, the top-level feature map extracted by the backbone network (i.e. Figure 2 The output of CSP1_1 in the FPN structure is input to the first network unit CBL (i.e. Figure 2 CBL1 in the figure), the feature map of the middle layer extracted by the backbone network (i.e. Figure 2The output of CSP1_3 in the FPN structure is input into the first multi-branch structure (MutilBranchBlock1) of the FPN structure, and the output of the first multi-branch structure is used as the second CBL (i.e. Figure 2 The input of CBL2 in

[0067] After upsampling (i.e., Upsampling) the feature map output by the first CBL, it is compared with the underlying feature map (i.e., Figure 2 The output of CSP1_2 in the PAN structure) and the feature map output by the second CBL are concatenated (i.e., concat), and the concatenated feature map is used as the input of the second multi-branch structure (MutilBranchBlock2) in the PAN structure;

[0068] The output of the second multi-branch structure is used as the third CBL in the PAN structure (i.e. Figure 2 The output of the second CBL is used as the input of the third multi-branch structure (MutilBranchBlock3) in the PAN structure, and the output of the third multi-branch structure is used as the fourth CBL (i.e. Figure 2 The input of CBL4 in

[0069] After down sampling, the feature map output by the third CBL is concatenated with the feature map of the top layer extracted by the backbone network and the feature map output by the fourth CBL. The concatenated feature map is used as the input of the fourth multi-branch structure (MutilBranchBlock4), and the enhanced feature map is output by the fourth multi-branch structure.

[0070] Specifically, Figure 2 In the embodiment, the output ends of the second multi-branch structure, the third multi-branch structure and the fourth multi-branch structure are respectively connected to two detection heads, one detection head is used to predict the at least one image area, and the other detection head is used to predict the at least one text area.

[0071] In this embodiment, the feature fusion network uses the FPN+PAN structure used in yolo5, and makes the following two improvements:

[0072] Improvement 1: Designed a multi-branch structure (MutilBranchBlock) to replace CspNet in yolo5 to build the Neck module, such as Figure 2 In the block, three convolution kernels of different sizes (such as 3*3, 1*3 and 3*1) are used to extract richer target features from different horizontal and vertical directions.

[0073] Improvement 2: Added feature fusion of the bottom layer and top layer in the FPN+PAN structure, such as Figure 2 As shown in the figure, in the FPN structure, the input of the second multi-branch structure is the result of stacking (feature concatenation) the output of CSP1_2 and the output of CBL2, and the result of CBL1 will be stacked into the input of the second multi-branch structure after upsampling; similarly, in the PAN structure, the features of the output of CBL3 will be added to the input of the fourth multi-branch structure after downsampling. In this way, by adding network connections between the bottom and top structures, features of different sizes can be better integrated, thereby obtaining better detection results.

[0074] Exemplarily, the network layer parameters in the Neck module may be as shown in Table 2 below.

[0075]

[0076]

[0077] Table 2

[0078] In an optional embodiment, the method further includes: using at least one of the following loss functions to perform model training to obtain a network model:

[0079] Loss function 1: Binary cross entropy BCE loss function for classification.

[0080] In the embodiment of the present application, the classification loss uses BCE_Loss. The network model output contains the category information of each detection box (text detection box or avatar detection box), as well as the probability value of belonging to each category of text or avatar. The maximum probability is taken as the category of the corresponding detection box. Through the term loss function, the detection box output by the initial model will be more accurately classified into the corresponding category, reducing the possibility of false detection.

[0081] Loss function 2: Focal loss function used to calculate confidence;

[0082] In the embodiment of the present application, the confidence loss uses Focal_loss. The corresponding confidence value of each detection box in the output of the network model indicates the degree of credibility of it as a real target box. During the model optimization process, the detection box containing more target areas (calculated by IOU) has better confidence, and the confidence of the detection box that does not contain the target is 0. During the model reasoning process, each output detection box has a corresponding confidence. In the post-processing, the detection box with a confidence less than the set threshold will be deleted, thereby reducing the number of false detection boxes and improving the effect of target detection.

[0083] Loss function three: CIoU loss function used to constrain the position of the image detection box;

[0084] In the embodiment of the present application, CIoU is used to constrain the position of the avatar detection frame. This loss causes the detection frame output by the network model to have a higher area overlap with the actual detection frame, making the position of the detection frame more accurate.

[0085] Loss function 4: TIoU loss function used to constrain the position of the text detection box.

[0086] In the embodiment of the present application, TIoU is a loss function specially designed for long strips of text lines. The characteristic of text is that even if a small part of the area is missing, it may still be subjectively recognized by people. Only coding part of the text area cannot activate the privacy protection function. Therefore, for the text line detection result, in addition to requiring that the detection box output by the network model and the GT (GroundTruth, referring to the box as the standard answer) have a high overlap area, it is also required that the number of real texts that can be recognized by the optical character recognition (OCR) algorithm in the text box detected by the network model is as large as possible, so that the privacy information of the text can be more completely protected after privacy coding. In this application, by adding the TIOU loss function, the position of the text detection box output by the network model covering the text can be made more accurate.

[0087] Optionally, in the embodiment of the present application, the calculation formula of the TIoU loss function is:

[0088] TIoU=ln inter(char pred , char gt ) / union(char pred , char gt );

[0089] Among them, inter refers to finding the intersection, union refers to finding the union, char_pred refers to the characters in the predicted text box detected by OCR, and har_gt refers to the characters in the true value box detected by OCR. pred represents the result of network prediction, and gt represents the manually annotated true value. Here, the true value box can be used as the detection box of the standard answer.

[0090] In an optional embodiment, each detection head in the head prediction network includes: TIoU loss function, CIoU loss function, Focal loss function, and BCE loss function.

[0091] Exemplarily, the calculation formula of the overall loss function is:

[0092]

[0093] It should be pointed out that the above overall function calculation formula is only an example. When calculating the overall loss function corresponding to each detection head, different weights can be assigned to each loss function. Among them, for the TIoU loss function and CIoU loss function, the smaller the calculated loss value, the better, while for the Focal loss function and BCE loss function, the larger the calculated loss value, the better. In order to allow the calculation company of the overall loss function that combines the four loss functions to accurately measure the deviation between the model prediction results and the actual results, in the above overall loss function formula, CIoU (reg pred ,reg gt ) and TIoU(reg pred ,reg gt ) is preceded by a minus sign, BCEloss(cls pred ,cls gt ) and Focalloss(conf pred ,conf gt ) is preceded by a plus sign.

[0094] For the training process of the network model, the input size of the network model is 640*640*3, and the number of epochs is set to 150 during the training process. During the inference process, the first image is first scaled to a size of 640*640, and then input into the network model in sequence. After the network model is inferred, the avatar detection frame and the text detection frame are obtained, and then the position coordinates of these detection frames are mapped back to the original input first image. That is, the location of the avatar detection frame is the image area in the target area, and the location of the text detection frame is the text area in the target area.

[0095] In an optional embodiment, in the above step 103, determining the privacy area according to the target area includes:

[0096] According to a middle vertical line of the first image, dividing at least one image area in the target area into a first set located on the left side of the middle vertical line and a second set located on the right side of the middle vertical line;

[0097] Determining a first head portrait area set located on the left from the first set according to the horizontal coordinates of the image areas in the first set;

[0098] Determining a second head portrait area set located on the right from the second set according to the horizontal coordinates of the image areas in the second set;

[0099] The privacy area is determined according to the first avatar area set and the second avatar area set.

[0100] In specific implementation, when the first image is a chat interface image, after detecting at least one image area in the first image through the network model, in order to identify the avatar area on the left side of the chat interface and the avatar area on the right side of the chat interface, this embodiment uses the middle vertical line of the first image to divide the at least one image area output by the network model into a first set on the left side of the middle vertical line and a second set on the right side of the middle vertical line. That is, the avatar on the left side of the chat interface is in the first set, and the avatar on the right side of the chat interface is in the second set.

[0101] Furthermore, for the chat interface image, since the horizontal coordinate range of the avatar area located on the left side of the chat interface screenshot is the same, the horizontal coordinate range of the avatar area located on the right side of the chat interface screenshot is also the same, therefore, for the image areas in the first set and the second set, the present application projects all the image areas in the first set onto the horizontal line to obtain the first overlapping area with the largest horizontal coordinate overlap; for each image area in the first set, if the projected image area includes the first overlapping area, then the image area is judged to be an avatar area, and finally the first avatar area set is obtained; similarly, all the image areas in the second set are projected onto the horizontal line to obtain the second overlapping area with the largest horizontal coordinate overlap; for each image area in the second set, if the projected image area includes the second overlapping area, then the image area is judged to be an avatar area, and finally the second avatar area set is obtained.

[0102] In the above embodiment, the acquired first avatar area set and second avatar area set can be determined as the user avatar part in the privacy area, the avatar area can be further screened based on the first avatar area set and the second avatar area set, and the user nickname can be detected based on the first avatar area set and the second avatar area set.

[0103] Among them, for further screening and detection of user avatars, in an optional embodiment, determining the privacy area according to the first avatar area set and the second avatar area set includes:

[0104] Using a sliding window, slide in the target longitudinal area of ​​the first image according to a preset step length to obtain N pixel points in the sliding window; N is a positive integer; wherein the target longitudinal area corresponding to the first head portrait area set is a first longitudinal area located on the left, and the target longitudinal area corresponding to the second head portrait area set is a second longitudinal area located on the left;

[0105] Determining whether the image in the sliding window matches the first area according to the three-channel color differences between the N pixels and the pixels at corresponding positions in the first area; wherein the first area is any one of the first avatar area set and the second avatar area set;

[0106] When the image in the sliding window matches the first area, traversing the avatar areas in the first avatar area set and the second avatar area set to determine an overlap ratio between each avatar area and the image area in the sliding window;

[0107] When the overlap ratio between the avatar area in the first avatar area set and the second avatar area set and the image area in the sliding window is less than a second preset value, the image area in the sliding window, the first avatar area set and the second avatar area set are determined as the user avatar part in the privacy area.

[0108] like Fig. 9 As shown, for the avatar area detection on the left side of the chat interface, a sliding window c with the same size as the avatar frame is used to slide from top to bottom in the first longitudinal area b1 according to a preset step size. For each step, N pixels in the sliding window c are randomly obtained, and the RGB three-channel color difference between the N pixels and any avatar area A in the first avatar area set is calculated; similarly, for the avatar area detection on the right side of the chat interface, a sliding window c with the same size as the avatar frame is used to slide from top to bottom in the second longitudinal area b2 according to a preset step size. For each step, N pixels in the sliding window c are randomly obtained, and the RGB three-channel color difference between the N pixels and any avatar area A in the second avatar area set is calculated.

[0109] Specifically, the calculation formula for the color difference of the three RGB channels is as follows:

[0110] D(A,c)=Σ i∈A，J∈c |r i -r j +|b i -b j |+|g i -g j |; Among them, the relative position of pixel i in the avatar area A is the same as the relative position of pixel j in the sliding window c; the three RGB channels are R (red), G (green), and B (blue).

[0111] Further, taking the first longitudinal area b1 as an example, if the average color difference between all pixel points j and pixel point i extracted from the sliding window c is less than the first preset threshold (such as 20), it is determined that the two are matched. In the case that the sliding window c and the avatar area A are matched, the overlapping ratio between the sliding window c and each avatar area C in the avatar area set, i.e., IOU, is calculated. If the IOU is less than the second preset threshold (such as 0.1), it is determined that the area in the sliding window c also belongs to the avatar area and should be added to the first avatar area set, and the area in the sliding window c and the avatar area A are recorded as the avatar of the same contact; otherwise, if the IOU is greater than or equal to the second preset threshold, the avatar area A and the avatar area C are recorded as the avatar of the same contact and do not need to be added to the first avatar area set.

[0112] Among them, for nickname detection, in an optional embodiment, after determining the image area in the sliding window, the first avatar area set and the second avatar area set as the user avatar part in the privacy area, the method further includes:

[0113] Determining the nickname area of ​​the chat group member from at least one text area in the target area according to the positional relationship between the user avatar area and the nickname area of ​​the chat group member;

[0114] The nickname area of ​​the chat group member is determined as the nickname part in the privacy area.

[0115] In this embodiment, for nickname detection, it is known in advance based on APP statistical history information or historical probability that a nickname can only appear next to an avatar and have a fixed positional relationship with it. Therefore, the embodiment of the present application determines the nickname area of ​​the chat group member from at least one text area in the target area based on the positional relationship between the user avatar area and the nickname area of ​​the chat group member.

[0116] In the specific implementation, for a chat interface image, first, record the coordinates of the upper left corner of the avatar area as (head left , head up ), the width and height of the avatar area are head_w and head_h respectively, and the coordinates of the upper left corner of the text area are (text left ,text up ), the lower right corner coordinates are (text right ,text bottom ).

[0117] Secondly, according to the APP type to which the chat interface screenshot belongs, the corresponding historical information is obtained as follows:

[0118] (1) The minimum distance l_min between the avatar frame (each avatar area corresponds to an avatar frame) and the nickname frame (each text area corresponds to a nickname frame); wherein the avatar frame is the avatar detection frame, and the nickname frame is the nickname detection frame.

[0119] (2) The relative position between the upper boundary of the nickname frame and the upper boundary of the avatar frame is: the upper boundary coordinates of the nickname frame text up In [head up +a*head_h,head up +b*head_h] coordinates, such as a=-0.25, b=0.5.

[0120] (3) The relative position of the bottom border of the nickname frame and the top border of the avatar frame is: The bottom border coordinates of the nickname frame are text bottom In [head up +c*head_h,head up +d*head_h] coordinates, such as c=0.25, d=0.75.

[0121] Again, for an avatar on the left side of the chat interface screenshot, a text box (i.e., text area) is considered to be its corresponding nickname area if it meets the following conditions at the same time:

[0122]

[0123] Similarly, for the avatar in the screenshot on the right side of the chat interface, a full text box (i.e., text area) is considered to be its corresponding nickname area and should meet the following conditions at the same time:

[0124]

[0125] It should be noted that for each avatar, if there are multiple text boxes that meet the conditions, the text box closest to it is selected as the corresponding nickname position. After screening in this step, each avatar corresponds to at most one nickname box corresponding to it.

[0126] In an optional embodiment, after the avatar area set is obtained, for multiple avatar frames belonging to the same contact, the left and right horizontal coordinates and vertical coordinates of the nickname frames corresponding to the avatar frames are aligned.

[0127] Specifically, let nick1 be the nickname of head1 (avatar 1), nick2 be the nickname of head2 (avatar 2), (left, up) be the coordinates of the upper left corner of the nickname box, and (right, bottom) be the coordinates of the lower right corner of the nickname box. The result to be aligned is shown in the following formula:

[0128]

[0129] It should be noted that if, when aligning the nickname box, it is determined that there is an undetected nickname area, the corresponding completion is performed. After the alignment operation, the detection accuracy and recall of the nickname area can be significantly improved.

[0130] In an optional embodiment, in the above step 103, determining the privacy area according to the target area includes:

[0131] Selecting a candidate region from at least one text region in the target region, the candidate region being a text region located above a boundary line of the first image;

[0132] Determining a group name area of ​​a chat group from the candidate areas according to the vertical line in the first image and historical information of an application to which the first image belongs;

[0133] The group name area of ​​the chat group is determined as the group name part in the privacy area.

[0134] In a specific implementation, when the first image is a screenshot of a chat interface, the historical information of the application can be obtained according to the type of application to which the first image belongs; and the target boundary line can be obtained according to the historical information of the application. Fig. 9 In the existing chat software, after obtaining multiple avatar areas in the first image, the target avatar area located at the top in the first image can be selected from the multiple avatar areas; the straight line where the upper boundary of the target avatar area is located is used as the target dividing line a.

[0135] Furthermore, since the group name area of ​​the chat group is generally located on the left and center of the chat interface. Therefore, it is possible to determine whether the group name is in the center or on the left based on the historical information of the application. Further, the group name area of ​​the chat group is determined from the candidate area based on the middle vertical line of the first image and the historical information of the application to which the first image belongs. For example, if the group name is located in the center, the text area covering the coordinates of the middle vertical line is selected from the candidate area as the group name candidate; if the group name is located on the left, the text area covering 1 / 4 of the screenshot width is selected from the candidate area as the group name candidate. Finally, based on statistical historical information, it can be seen that the font of the group name is larger than the font of other information (such as time) in the top area, so the text area with the largest text box height can be selected from the group name alternatives as the group name area of ​​the chat group.

[0136] In an optional embodiment, the image processing method further includes:

[0137] When acquiring the first image, correspondingly acquiring the name of the application to which the first image belongs;

[0138] The application name is written into the attribute information of the first image.

[0139] In a specific implementation, when the first image is obtained, the name of the application to which the chat interface belongs is obtained at the same time, and the name of the application is written into the attribute information of the first image. In this way, the name of the application can be obtained according to the attribute information of the first image, thereby obtaining the historical information of the application.

[0140] In an optional embodiment, before detecting the target area in the first image, the method further includes:

[0141] Determine the image type of the first image according to the attribute information of the first image; the image type includes: single-screen image style, dual-screen image style and long image style;

[0142] When it is determined that the picture style of the first image is the dual-screen image style or the long image style, the first image is processed into the single-screen image style.

[0143] For example, when the first image is a screenshot of a chat interface, different applications display different styles, such as some chat interfaces directly tile a single interface to two screens, while some chat interfaces display a chat list or a contact list on the left screen and a chat interface on the right screen. Therefore, when cropping the first image in a dual-screen image style, the cropping method can be determined according to the type of application.

[0144] Exemplarily, for the first image of the long image style, such as Fig.10 As shown, a single-screen picture can be extracted by sliding in the first image through a longitudinal sliding window d with a step size of 200 and a height of 1000, that is, the image area within each longitudinal sliding window d is regarded as a single-screen picture. In this way, a first image in a long image style can be processed into multiple single-screen pictures.

[0145] For example, for a long picture in a wide screen, the cropping logic of the dual-screen image is first executed, and then the cropping logic of the vertical sliding window is executed on the cropped result. After such processing, the input first image can be processed into multiple single-screen pictures.

[0146] In specific implementation, the type of application can be determined based on the attribute information of the first image; based on the type of application, it is determined whether to crop the first image; if the first image is not to be cropped, the multi_window value is false; if the first image needs to be cropped into the left screen area and the right screen area according to the middle line, the multi_window value is true.

[0147] In the above embodiment, before inputting the first image into the network model, a better detection effect can be obtained by processing the first image into a single-screen image style.

[0148] The image processing method provided in the embodiment of the present application can be executed by an image processing device. In the embodiment of the present application, an image processing device executing the image processing method is taken as an example to illustrate the image processing device provided in the embodiment of the present application.

[0149] See also Fig.11 , the embodiment of the present application provides an image processing device 1100, comprising:

[0150] A first acquisition module 1101 is used to acquire a first image;

[0151] A detection module 1102 is used to detect a target area in the first image; wherein the target area includes: at least one text area and at least one image area;

[0152] A first determining module 1103, configured to determine a privacy area according to the target area;

[0153] The processing module 1104 is used to perform privacy processing on the privacy area to obtain a processed second image.

[0154] Optionally, the detection module 1102 includes:

[0155] A first detection submodule, used for extracting feature maps of the top layer, middle layer and bottom layer of the first image through a backbone network of a network model;

[0156] A second detection submodule is used to splice the feature map through a feature fusion network of the network model to obtain an enhanced feature map;

[0157] The third detection submodule is used to input the enhanced feature map into the head prediction network of the network model to obtain the target area; wherein the head prediction network includes two detection heads; one detection head is used to predict the at least one image area, and the other detection head is used to predict the at least one text area, and the detection frame output by each detection head includes the following three feature information: the category of the detection frame is text or image, the position of the detection frame and the confidence of the detection frame.

[0158] Optionally, the feature fusion network includes an FPN structure and a PAN structure connected in sequence; the cross-stage local network CspNet in the FPN structure and the PAN structure is replaced by a multi-branch structure with three convolution kernels of different sizes;

[0159] The second detection submodule is specifically used for:

[0160] Input the feature map of the top layer extracted by the backbone network into the first network unit CBL of the FPN structure, input the feature map of the middle layer extracted by the backbone network into the first multi-branch structure of the FPN structure, and use the output of the first multi-branch structure as the input of the second CBL;

[0161] After upsampling the feature map output by the first CBL, the feature map is spliced ​​with the bottom feature map extracted by the backbone network and the feature map output by the second CBL, and the spliced ​​feature map is used as the input of the second multi-branch structure in the PAN structure;

[0162] Using the output of the second multi-branch structure as the input of the third CBL in the PAN structure, using the output of the second CBL as the input of the third multi-branch structure in the PAN structure, and using the output of the third multi-branch structure as the input of the fourth CBL in the PAN structure;

[0163] After downsampling the feature map output by the third CBL, it is spliced ​​with the feature map of the top layer extracted by the backbone network and the feature map output by the fourth CBL, and the spliced ​​feature map is used as the input of the fourth multi-branch structure, and the enhanced feature map is output by the fourth multi-branch structure.

[0164] Optionally, the device 1100 further includes:

[0165] When training the network model, the TIoU loss function is used to constrain the position of the text detection box;

[0166] Among them, the calculation formula of the TIoU loss function is:

[0167] TIoU=ln inter(char pred , char gt ) / union(char pred , char gt );

[0168] Among them, inter refers to finding the intersection, union refers to finding the union, char_pred refers to the characters detected by OCR in the predicted text box, and har_gt refers to the characters detected by OCR in the true value box.

[0169] Optionally, the first determining module 1103 includes:

[0170] A first determination submodule, configured to divide at least one image region in the target region into a first set located on the left side of the middle vertical line and a second set located on the right side of the middle vertical line according to the middle vertical line of the first image;

[0171] A second determining submodule, configured to determine a first head portrait area set located on the left side from the first set according to the horizontal coordinates of the image areas in the first set;

[0172] A third determining submodule, configured to determine a second head portrait area set located on the right side from the second set according to the horizontal coordinates of the image areas in the second set;

[0173] The fourth determining submodule is used to determine the privacy area according to the first avatar area set and the second avatar area set.

[0174] Optionally, the fourth determining submodule includes:

[0175] The first determining unit is used to use a sliding window to slide in the target longitudinal area of ​​the first image according to a preset step length to obtain N pixel points in the sliding window; N is a positive integer; wherein the target longitudinal area corresponding to the first head portrait area set is the first longitudinal area located on the left, and the target longitudinal area corresponding to the second head portrait area set is the second head portrait longitudinal area located on the left;

[0176] A second determining unit is used to determine whether the image in the sliding window matches the first area according to a three-channel color difference between the N pixels and the pixels at corresponding positions in the first area; wherein the first area is any one of the first avatar area set and the second avatar area set;

[0177] a third determining unit, configured to, when the image in the sliding window matches the first area, traverse the avatar areas in the first avatar area set and the second avatar area set, and determine an overlap ratio between each avatar area and the image area in the sliding window;

[0178] The fourth determination unit is used to determine the image area in the sliding window, the first avatar area set and the second avatar area set as the user avatar part in the privacy area when the overlap ratio between the avatar area and the image area in the sliding window in the first avatar area set and the second avatar area set is less than a second preset value.

[0179] Optionally, the device further comprises:

[0180] a fifth determining unit, configured to determine the nickname area of ​​the chat group member from at least one text area in the target area according to a positional relationship between the user avatar area and the nickname area of ​​the chat group member;

[0181] The sixth determining unit is used to determine the nickname area of ​​the chat group member as the nickname part in the privacy area.

[0182] Optionally, the first determining module 1103 includes:

[0183] a fifth determination submodule, configured to select a candidate region from at least one text region in the target region, wherein the candidate region is a text region located above a boundary line of the first image;

[0184] a sixth determination submodule, configured to determine a group name area of ​​the chat group from the candidate areas according to the vertical line in the first image and historical information of an application to which the first image belongs;

[0185] The seventh determination submodule is used to determine the group name area of ​​the chat group as the group name part in the privacy area.

[0186] Optionally, the device 1100 further includes:

[0187] A second acquisition module, configured to acquire a name of an application to which the first image belongs when acquiring the first image;

[0188] The third acquisition module is used to write the application name into the attribute information of the first image.

[0189] Optionally, the device 1100 includes:

[0190] A second determination module, configured to determine a picture type of the first image according to the attribute information of the first image; the picture type includes: a single-screen image style, a dual-screen image style, and a long image style;

[0191] The third determining module is used to process the first image into the single-screen image style when it is determined that the picture style of the first image is the dual-screen image style or the long image style.

[0192] In the embodiment of the present application, a first image is acquired; a target area in the first image is detected; wherein the target area includes: at least one text area and at least one image area; a privacy area is determined according to the target area; and privacy processing is performed on the privacy area to obtain a processed second image. In this way, it can help users hide privacy information with one click, save users' operation time, and facilitate user operations.

[0193] The image processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0194] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0195] The image processing device provided in the embodiment of the present application can achieve Figures 1 to 10 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0196] Alternatively, if Fig.12 As shown, an embodiment of the present application further provides an electronic device 1200, including a processor 1201 and a memory 1202, wherein the memory 1202 stores programs or instructions that can be executed on the processor 1201, and when the program or instructions are executed by the processor 1201, the various steps of the above-mentioned image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0197] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0198] Fig.13 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0199] The electronic device 1300 includes but is not limited to: a radio frequency unit 1301, a network module 1302, an audio output unit 1303, an input unit 1304, a sensor 1305, a display unit 1306, a user input unit 1307, an interface unit 1308, a memory 1309, and a processor 1310 and other components.

[0200] Those skilled in the art will appreciate that the electronic device 1300 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1310 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.13 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0201] Among them, the processor 1310 is used to obtain a first image; detect a target area in the first image; wherein the target area includes: at least one text area and at least one image area; determine a privacy area based on the target area; perform privacy processing on the privacy area to obtain a processed second image.

[0202] Optionally, the processor 1310 is specifically used to extract feature maps of the top layer, middle layer and bottom layer of the first image through the backbone network of the network model; splice the feature maps through the feature fusion network of the network model to obtain an enhanced feature map; input the enhanced feature map to the head prediction network of the network model to obtain the target area; wherein the head prediction network includes two detection heads; one detection head is used to predict the at least one image area, and the other detection head is used to predict the at least one text area, and the detection box output by each detection head includes the following three feature information: the category of the detection box is text or image, the position of the detection box and the confidence of the detection box.

[0203] Optionally, the feature fusion network includes an FPN structure and a PAN structure connected in sequence; the cross-stage local network CspNet in the FPN structure and the PAN structure is replaced with a multi-branch structure with three convolution kernels of different sizes; the processor 1310 is specifically used to:

[0204] Input the feature map of the top layer extracted by the backbone network into the first network unit CBL of the FPN structure, input the feature map of the middle layer extracted by the backbone network into the first multi-branch structure of the FPN structure, and use the output of the first multi-branch structure as the input of the second CBL;

[0205] After upsampling the feature map output by the first CBL, the feature map is spliced ​​with the bottom feature map extracted by the backbone network and the feature map output by the second CBL, and the spliced ​​feature map is used as the input of the second multi-branch structure in the PAN structure;

[0206] Using the output of the second multi-branch structure as the input of the third CBL in the PAN structure, using the output of the second CBL as the input of the third multi-branch structure in the PAN structure, and using the output of the third multi-branch structure as the input of the fourth CBL in the PAN structure;

[0207] After downsampling the feature map output by the third CBL, it is spliced ​​with the feature map of the top layer extracted by the backbone network and the feature map output by the fourth CBL, and the spliced ​​feature map is used as the input of the fourth multi-branch structure, and the enhanced feature map is output by the fourth multi-branch structure.

[0208] Optionally, the processor 1310 is specifically configured to:

[0209] According to a middle vertical line of the first image, dividing at least one image area in the target area into a first set located on the left side of the middle vertical line and a second set located on the right side of the middle vertical line;

[0210] Determining a first head portrait area set located on the left from the first set according to the horizontal coordinates of the image areas in the first set;

[0211] Determining a second head portrait area set located on the right from the second set according to the horizontal coordinates of the image areas in the second set;

[0212] The privacy area is determined according to the first avatar area set and the second avatar area set.

[0213] Optionally, the processor 1310 is specifically configured to:

[0214] Using a sliding window, slide in the target longitudinal area of ​​the first image according to a preset step length to obtain N pixel points in the sliding window; N is a positive integer; wherein the target longitudinal area corresponding to the first head portrait area set is a first longitudinal area located on the left, and the target longitudinal area corresponding to the second head portrait area set is a second longitudinal area located on the left;

[0215] Determining whether the image in the sliding window matches the first area according to the three-channel color differences between the N pixels and the pixels at corresponding positions in the first area; wherein the first area is any one of the first avatar area set and the second avatar area set;

[0216] When the image in the sliding window matches the first area, traversing the avatar areas in the first avatar area set and the second avatar area set to determine an overlap ratio between each avatar area and the image area in the sliding window;

[0217] When the overlap ratio between the avatar area in the first avatar area set and the second avatar area set and the image area in the sliding window is less than a second preset value, the image area in the sliding window, the first avatar area set and the second avatar area set are determined as the user avatar part in the privacy area.

[0218] Optionally, the processor 1310 is further configured to:

[0219] Determining the nickname area of ​​the chat group member from at least one text area in the target area according to the positional relationship between the user avatar area and the nickname area of ​​the chat group member;

[0220] The nickname area of ​​the chat group member is determined as the nickname part in the privacy area.

[0221] Optionally, the processor 1310 is specifically configured to:

[0222] Selecting a candidate region from at least one text region in the target region, the candidate region being a text region located above a boundary line of the first image;

[0223] Determining a group name area of ​​a chat group from the candidate areas according to the vertical line in the first image and historical information of an application to which the first image belongs;

[0224] The group name area of ​​the chat group is determined as the group name part in the privacy area.

[0225] Optionally, the processor 1310 is specifically configured to:

[0226] When acquiring the first image, correspondingly acquiring the name of the application to which the first image belongs;

[0227] The application name is written into the attribute information of the first image.

[0228] Optionally, the processor 1310 is further configured to:

[0229] Determine the image type of the first image according to the attribute information of the first image; the image type includes: single-screen image style, dual-screen image style and long image style;

[0230] When it is determined that the picture style of the first image is the dual-screen image style or the long image style, the first image is processed into the single-screen image style.

[0231] In the embodiment of the present application, the electronic device acquires a first image; detects a target area in the first image; wherein the target area includes: at least one text area and at least one image area; determines a privacy area according to the target area; performs privacy processing on the privacy area to obtain a processed second image. In this way, it can help users hide privacy information with one click, save users' operation time, and facilitate user operations.

[0232] It should be understood that in the embodiment of the present application, the input unit 1304 may include a graphics processor (Graphics Processing Unit, GPU) 13041 and a microphone 13042, and the graphics processor 13041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1306 may include a display panel 13061, and the display panel 13061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1307 includes a touch panel 13071 and at least one of other input devices 13072. The touch panel 13071 is also called a touch screen. The touch panel 13071 may include two parts: a touch detection device and a touch controller. Other input devices 13072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0233] The memory 1309 can be used to store software programs and various data. The memory 1309 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1309 may include a volatile memory or a non-volatile memory, or the memory 1309 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1309 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0234] The processor 1310 may include one or more processing units; optionally, the processor 1310 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1310.

[0235] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0236] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0237] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0238] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0239] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0240] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0241] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0242] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for processing an image, characterized in that: include: acquiring a first image; Detecting a target area in the first image; wherein the target area includes: at least one text area and at least one image area; Determining a privacy area according to the target area; Performing privacy processing on the privacy area to obtain a processed second image; The step of detecting the target area in the first image includes: Extracting the top, middle and bottom feature maps of the first image through the backbone network of the network model; The feature maps are spliced ​​through a feature fusion network of the network model to obtain an enhanced feature map; The enhanced feature map is input into the head prediction network of the network model to obtain the target area; wherein the head prediction network includes two detection heads; one detection head is used to predict the at least one image area, and the other detection head is used to predict the at least one text area, and the detection frame output by each detection head includes the following three feature information: the category of the detection frame is text or image, the position of the detection frame and the confidence of the detection frame; The feature fusion network includes an FPN structure and a PAN structure connected in sequence; the cross-stage local network CspNet in the FPN structure and the PAN structure is replaced by a multi-branch structure, and each of the multi-branch structures includes three convolution kernels of different sizes; The step of splicing the feature maps through the feature fusion network of the network model to obtain an enhanced feature map includes: Input the feature map of the top layer extracted by the backbone network into the first network unit CBL of the FPN structure, input the feature map of the middle layer extracted by the backbone network into the first multi-branch structure of the FPN structure, and use the output of the first multi-branch structure as the input of the second CBL; After upsampling the feature map output by the first CBL, the feature map is spliced ​​with the bottom feature map extracted by the backbone network and the feature map output by the second CBL, and the spliced ​​feature map is used as the input of the second multi-branch structure in the PAN structure; Using the output of the second multi-branch structure as the input of the third CBL in the PAN structure, using the output of the second CBL as the input of the third multi-branch structure in the PAN structure, and using the output of the third multi-branch structure as the input of the fourth CBL in the PAN structure; After downsampling the feature map output by the third CBL, it is spliced ​​with the feature map of the top layer extracted by the backbone network and the feature map output by the fourth CBL, and the spliced ​​feature map is used as the input of the fourth multi-branch structure, and the enhanced feature map is output by the fourth multi-branch structure.

2. The image processing method according to claim 1, characterized in that: The step of determining the privacy area according to the target area includes: According to a middle vertical line of the first image, dividing at least one image area in the target area into a first set located on the left side of the middle vertical line and a second set located on the right side of the middle vertical line; Determining a first head portrait area set located on the left from the first set according to the horizontal coordinates of the image areas in the first set; Determining a second head portrait area set located on the right from the second set according to the horizontal coordinates of the image areas in the second set; The privacy area is determined according to the first avatar area set and the second avatar area set.

3. The image processing method according to claim 2, characterized in that: The determining the privacy area according to the first head portrait area set and the second head portrait area set includes: Using a sliding window, slide in the target longitudinal area of ​​the first image according to a preset step length to obtain N pixel points in the sliding window; N is a positive integer; wherein the target longitudinal area corresponding to the first head portrait area set is a first longitudinal area located on the left, and the target longitudinal area corresponding to the second head portrait area set is a second longitudinal area located on the right; Determining whether the image in the sliding window matches the first area according to the three-channel color differences between the N pixels and the pixels at corresponding positions in the first area; wherein the first area is any one of the first avatar area set and the second avatar area set; When the image in the sliding window matches the first area, traversing the avatar areas in the first avatar area set and the second avatar area set to determine an overlap ratio between each avatar area and the image area in the sliding window; When the overlap ratio between the avatar area in the first avatar area set and the second avatar area set and the image area in the sliding window is less than a second preset value, the image area in the sliding window, the first avatar area set and the second avatar area set are determined as the user avatar part in the privacy area.

4. The image processing method according to claim 3, characterized in that: After determining the image area in the sliding window, the first avatar area set, and the second avatar area set as the user avatar part in the privacy area, the method further includes: Determining the nickname area of ​​the chat group member from at least one text area in the target area according to the positional relationship between the user avatar area and the nickname area of ​​the chat group member; The nickname area of ​​the chat group member is determined as the nickname part in the privacy area.

5. The image processing method according to claim 1, characterized in that: The step of determining the privacy area according to the target area includes: Selecting a candidate region from at least one text region in the target region, the candidate region being a text region located above a boundary line of the first image; Determining a group name area of ​​a chat group from the candidate areas according to the vertical line in the first image and historical information of an application to which the first image belongs; The group name area of ​​the chat group is determined as the group name part in the privacy area.

6. The image processing method according to claim 1, characterized in that: The method further comprises: When acquiring the first image, correspondingly acquiring the name of the application to which the first image belongs; The application name is written into the attribute information of the first image.

7. The image processing method according to claim 6, characterized in that: Before detecting the target area in the first image, the method further includes: Determine the image type of the first image according to the attribute information of the first image; the image type includes: single-screen image style, dual-screen image style and long image style; When it is determined that the picture style of the first image is the dual-screen image style or the long image style, the picture type of the first image is processed into the single-screen image style.

8. An image processing device, characterized in that: include: A first acquisition module, used to acquire a first image; A detection module, configured to detect a target area in the first image; wherein the target area includes: at least one text area and at least one image area; A first determining module, configured to determine a privacy area according to the target area; A processing module, used for performing privacy processing on the privacy area to obtain a processed second image; The detection module comprises: A first detection submodule, used to extract feature maps of the top layer, middle layer and bottom layer of the first image through a backbone network of a network model; A second detection submodule is used to splice the feature map through a feature fusion network of the network model to obtain an enhanced feature map; The third detection submodule is used to input the enhanced feature map into the head prediction network of the network model to obtain the target area; wherein the head prediction network includes two detection heads; one detection head is used to predict the at least one image area, and the other detection head is used to predict the at least one text area, and the detection frame output by each detection head includes the following three feature information: the category of the detection frame is text or image, the position of the detection frame and the confidence of the detection frame; The feature fusion network includes an FPN structure and a PAN structure connected in sequence; the cross-stage local network CspNet in the FPN structure and the PAN structure is replaced by a multi-branch structure with three convolution kernels of different sizes; The second detection submodule is specifically used for: Input the feature map of the top layer extracted by the backbone network into the first network unit CBL of the FPN structure, input the feature map of the middle layer extracted by the backbone network into the first multi-branch structure of the FPN structure, and use the output of the first multi-branch structure as the input of the second CBL; After upsampling the feature map output by the first CBL, the feature map is spliced ​​with the bottom feature map extracted by the backbone network and the feature map output by the second CBL, and the spliced ​​feature map is used as the input of the second multi-branch structure in the PAN structure; Using the output of the second multi-branch structure as the input of the third CBL in the PAN structure, using the output of the second CBL as the input of the third multi-branch structure in the PAN structure, and using the output of the third multi-branch structure as the input of the fourth CBL in the PAN structure; After downsampling the feature map output by the third CBL, it is spliced ​​with the feature map of the top layer extracted by the backbone network and the feature map output by the fourth CBL, and the spliced ​​feature map is used as the input of the fourth multi-branch structure, and the enhanced feature map is output by the fourth multi-branch structure.

9. The image processing device according to claim 8, characterized in that: The first determining module includes: A first determination submodule, configured to divide at least one image region in the target region into a first set located on the left side of the middle vertical line and a second set located on the right side of the middle vertical line according to the middle vertical line of the first image; A second determining submodule, configured to determine a first head portrait area set located on the left side from the first set according to the horizontal coordinates of the image areas in the first set; A third determining submodule, configured to determine a second head portrait area set located on the right side from the second set according to the horizontal coordinates of the image areas in the second set; The fourth determining submodule is used to determine the privacy area according to the first avatar area set and the second avatar area set.

10. The image processing device according to claim 9, characterized in that: The fourth determining submodule includes: The first determining unit is configured to use a sliding window to slide in a target longitudinal region of the first image according to a preset step length to obtain N pixel points in the sliding window; N is a positive integer; wherein the target longitudinal region corresponding to the first head portrait region set is a first longitudinal region located on the left, and the target longitudinal region corresponding to the second head portrait region set is a second longitudinal region located on the right; A second determining unit is used to determine whether the image in the sliding window matches the first area according to a three-channel color difference between the N pixels and the pixels at corresponding positions in the first area; wherein the first area is any one of the first avatar area set and the second avatar area set; a third determining unit, configured to, when the image in the sliding window matches the first area, traverse the avatar areas in the first avatar area set and the second avatar area set, and determine an overlap ratio between each avatar area and the image area in the sliding window; The fourth determination unit is used to determine the image area in the sliding window, the first avatar area set and the second avatar area set as the user avatar part in the privacy area when the overlap ratio between the avatar area and the image area in the sliding window in the first avatar area set and the second avatar area set is less than a second preset value.

11. The image processing device according to claim 10, characterized in that: The device also includes: a fifth determining unit, configured to determine the nickname area of ​​the chat group member from at least one text area in the target area according to a positional relationship between the user avatar area and the nickname area of ​​the chat group member; The sixth determining unit is used to determine the nickname area of ​​the chat group member as the nickname part in the privacy area.

12. The image processing device according to claim 8, characterized in that: The first determining module includes: a fifth determination submodule, configured to select a candidate region from at least one text region in the target region, wherein the candidate region is a text region located above a boundary line of the first image; a sixth determination submodule, configured to determine a group name area of ​​the chat group from the candidate areas according to the vertical line in the first image and historical information of an application to which the first image belongs; The seventh determination submodule is used to determine the group name area of ​​the chat group as the group name part in the privacy area.

13. The image processing device according to claim 8, characterized in that: The device further comprises: A second acquisition module, configured to acquire a name of an application to which the first image belongs when acquiring the first image; The third acquisition module is used to write the application name into the attribute information of the first image.

14. The image processing device according to claim 13, characterized in that: The device comprises: A second determination module, configured to determine a picture type of the first image according to the attribute information of the first image; the picture type includes: a single-screen image style, a dual-screen image style, and a long image style; The third determining module is used to process the first image into the single-screen image style when it is determined that the picture style of the first image is the dual-screen image style or the long image style.

15. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the image processing method according to any one of claims 1 to 7 are implemented.

16. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the image processing method according to any one of claims 1 to 7 are implemented.

Images