Image processing method and processing device, electronic device and readable storage medium

Abstract

The present application discloses an image processing method and processing device, an electronic device, and a readable storage medium. The image processing method includes: locating a target area in a first image to obtain location information of the target area; erasing image information of a preset area in the first image to obtain an erased second image, wherein the second image has the same size as the first image and the preset area includes the target area; based on the location information, intercepting a regional image corresponding to the target area in the second image; and generating a processed target image based on the regional image and the first image.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to an image processing method and processing device, an electronic device, and a readable storage medium. Background Art

[0002] In the related art, users sometimes have the need to erase or hide specific content in a picture, such as erasing text in the picture.

[0003] Existing image processing methods, such as image inpainting, which erases specific areas, only utilize the texture information in the erased area, resulting in poor results and leaving noticeable inpainting artifacts in the image. While deep learning can reduce inpainting artifacts, it can also easily erase other areas of the image with similar content, resulting in poor results. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide an image processing method and processing device, an electronic device and a readable storage medium, which can solve the problem of poor processing effect of erasing image content in the prior art.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0006] Locating a target area in the first image to obtain position information of the target area;

[0007] Erasing image information of a preset area in the first image through an image processing model to obtain an erased second image, wherein the second image has the same size as the first image, and the preset area includes the target area;

[0008] intercepting an area image corresponding to the target area in the second image according to the position information;

[0009] A processed target image is generated according to the region image and the first image.

[0010] In a second aspect, an embodiment of the present application provides an image processing device, comprising:

[0011] a positioning module, configured to locate a target area in the first image and obtain position information of the target area;

[0012] an erasing module, configured to erase image information of a preset area in the first image through an image processing model to obtain an erased second image, wherein the second image has the same size as the first image, and the preset area includes the target area;

[0013] A capture module, configured to capture a region image corresponding to the target region in the second image according to the position information;

[0014] The processing module is used to generate a processed target image according to the regional image and the first image.

[0015] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method of the first aspect are implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method of the first aspect are implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps of the method of the first aspect.

[0018] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the method described in the first aspect.

[0019] In the embodiment of the present application, the image area to be processed is first located and the location information is recorded. Then, the entire first image is processed using an adversarial model. This image processing step is based on image recognition technology and deep learning technology. The model automatically identifies the portion to be processed and, based on the global information of the first image, erases the preset area to be processed, maintaining the image of the erased preset area consistent with the overall image.

[0020] After the erasing is completed, according to the position information of the marked target area, the area image at the same position is captured in the second image after erasing through the image processing model, and the area image is combined with the original first image, that is, only the part of the second image that the user needs to eliminate is selected to process the original image. Therefore, the final target image erases the content of the target area that needs to be processed while retaining the coordination and consistency of the overall image, while ensuring that other similar areas in the image will not be erased by mistake, thereby improving the processing efficiency when erasing specific content in the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 shows one of the flow charts of the image processing method according to an embodiment of the present application;

[0022] Figure 2 One of the schematic diagrams of the image processing method according to an embodiment of the present application is shown;

[0023] Figure 3 A second schematic diagram of an image processing method according to an embodiment of the present application is shown;

[0024] Figure 4 A third schematic diagram of an image processing method according to an embodiment of the present application is shown;

[0025] Figure 5 A schematic diagram of the structure of an image processing model according to an embodiment of the present application is shown;

[0026] Figure 6 The second flowchart of the image processing method according to the embodiment of the present application is shown;

[0027] Figure 7 shows a structural block diagram of an image processing device according to an embodiment of the present application;

[0028] Figure 8 shows a structural block diagram of an electronic device according to an embodiment of the present application;

[0029] Figure 9 A schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0031] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0032] The image processing method and processing device, electronic device, and readable storage medium provided in the embodiments of the present application are described in detail below with reference to specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0033] In some embodiments of the present application, an image processing method is provided. Figure 1 FIG. 1 shows one of the flow charts of the image processing method according to an embodiment of the present application. Figure 1 As shown, the method includes:

[0034] Step 102: locate the target area in the first image to obtain position information of the target area;

[0035] Step 104: Erasing image information of a preset area in the first image through an image processing model to obtain an erased second image;

[0036] In step 104, the size of the second image is the same as that of the first image, and the preset area includes the target area;

[0037] Step 106, intercepting an area image corresponding to the target area in the second image according to the position information;

[0038] Step 108: Generate a processed target image based on the regional image and the first image.

[0039] In an embodiment of the present application, when a user wishes to hide or erase specific content in an image, such as text content in an image, first, a text region in the image can be located using, for example, a text recognition algorithm, and the location information corresponding to the region can be recorded. The location information can be coordinate information.

[0040] After obtaining the location information of the target area to be erased, the original image of the first image is input as a whole into a preset image processing model. Specifically, the image processing model can automatically identify the target information to be erased in the first image according to the type of information that the user needs to erase, and process the first image as a whole according to the global information of the target image.

[0041] Specifically, Figure 2 One of the schematic diagrams of the image processing method according to an embodiment of the present application is shown, Figure 2 As shown, the first image 200 includes text information 202, and the user needs to erase the text information 202. First, by means such as optical character recognition (OCR), the area where the text information 202 is located is marked in the first image 200, that is, the target area 204, and the coordinate information of the target area 204 is recorded, thereby recording the position information of the target area 204.

[0042] After obtaining the location information of the target area, the first image is input into a preset image processing model. The image processing model can automatically identify the image information content to be erased by the user, such as text information, and erase it based on the global information of the target image, including color information, pixel information, etc.

[0043] Figure 3A second schematic diagram of the image processing method according to an embodiment of the present application is shown. Figure 3 As shown, the image processing model processes the entire first image 300 and erases several preset areas that are identified as containing text. Figure 3 As shown, the image processing model erases both preset areas, wherein the first preset area 302 contains text information that the user needs to erase, and the second preset area 304 contains QR code information. Since the features of the QR code information and the text information are similar, the QR code information is mistakenly recognized as text and is erased.

[0044] Furthermore, after the image processing model outputs the second image, based on the position information of the identified target area, the corresponding area image is cut out in the second image according to the same coordinates, and the area image is superimposed on the original image of the first image according to the coordinates of the target area, thereby covering the target area of the first image and generating the target image.

[0045] Figure 4 A third schematic diagram of an image processing method according to an embodiment of the present application is shown. Figure 4 As shown, on the target image 400 , the text information in the target area 402 is erased, while the QR code 404 is retained.

[0046] The embodiment of the present application captures the area image at the same position in the second image after erasing through the image processing model based on the position information of the marked target area, and combines the area image with the original first image, that is, only the part of the second image that the user needs to eliminate is selected to process the original image. Therefore, the target image finally obtained erases the content of the target area that needs to be processed while retaining the coordination and consistency of the overall image, and at the same time ensures that other similar areas in the image will not be erased by mistake, thereby improving the processing efficiency when erasing specific content in the image.

[0047] In some embodiments of the present application, generating a processed target image based on the regional image and the first image includes: overlaying the regional image on the target region according to the position information to obtain the target image.

[0048] In an embodiment of the present application, after the second image is obtained through the image processing model, the area image corresponding to the position of the target area is cut out from the second image based on the position information of the target area, such as coordinate information.

[0049] After obtaining the regional image, the regional image obtained by interception and processing by the image processing model is overlaid on the original first image according to the coordinate information of the target area, so that the image content in the target area is completely replaced by the regional image. In this way, in the replaced target image, only the area that needs to be erased is replaced by the processed image. While retaining the coordination and consistency of the overall image, the content of the target area that needs to be processed is erased, and at the same time, it is ensured that other similar areas in the image will not be erased by mistake, thereby improving the processing efficiency when erasing specific content in the image.

[0050] In some embodiments of the present application, before erasing image information of a preset area in the first image, the method further includes:

[0051] Acquire a first training image and a second training image, wherein the second training image is an image obtained by removing preset image information from the first training image;

[0052] Training a preset model using the first training image and the second training image to obtain a trained image processing model, where the image processing model includes a first network and a second network;

[0053] Erasing image information of a preset area in the first image, including:

[0054] performing erasing processing on the first image through the first network to obtain a processed third image;

[0055] downsampling the third image to obtain a fourth image;

[0056] performing erasing processing on the fourth image through the second network to obtain a processed fifth image;

[0057] Upsampling is performed on the fifth image to obtain a second image.

[0058] In an embodiment of the present application, a preset adversarial model is trained to obtain a trained image processing model. Specifically, a first image is first collected and manually processed to generate a second image. The first image is the original image, and the second image is obtained by erasing a preset area such as text content using image editing or image modification software.

[0059] Using the first and second images as training sets, a pre-set generative adversarial network is trained to perform end-to-end whole-image text erasure. Specifically, a network model is built, using the UNET network structure for the generative network. The image is first downsampled to obtain semantic information, and then upsampled to its original size to obtain the output image. Considering that a single model optimization may not yield good results for complex scenes, a lightweight UNET result is added after the UNET (a dense, predictive, and segmented U-shaped network) structure for further processing to obtain the final result.

[0060] Furthermore, a discriminant network is constructed. The discriminant network includes a two-scale network, where the first scale network includes several cascaded convolutions, where the convolution step size can be set to 1, and no pooling layer is introduced to ensure that the resolution of the image does not decrease.

[0061] After the first scale network, a second scale network is added with the same scale, but the input of the second scale network is obtained by downsampling the output of the first network by one factor. The ground truth of the two-scale network is obtained by taking the erased image and the downsampled image of the erased image by one factor.

[0062] The output of the generative network is input into the discriminative network, which determines the difference between the currently generated erased image and the original annotated erased image (i.e., the second image). This difference is used as the loss to backpropagate and optimize the network parameters, ultimately obtaining an optimized network structure.

[0063] In this optimized network structure, the model obtained after removing the discriminant network is the above-mentioned image processing model. Figure 5 FIG. 4 shows a schematic diagram of the structure of an image processing model according to an embodiment of the present application. Figure 5 Therefore, the image processing model 500 includes a two-layer UNET network structure, namely a first network 502 and a second network 504.

[0064] When erasing specific content in the first image 506 through the image processing model 500 , first, the first image 506 to be processed is input into the first network 502 , and the specific information therein is erased through the first network 502 .

[0065] After erasing, a processed third image 508 is obtained, and the third image 508 is downsampled to obtain a fourth image 510 with reduced resolution, so as to obtain the image semantic definition, and the downsampled fourth image 510 is used as the input of the second layer network, that is, the second network 504.

[0066] Finally, the fifth image 512 output by the second network 504 is upsampled and restored to its original size to obtain the final second image 514.

[0067] It is understood that the content contained in the first image is the content that the user needs to erase. For example, if the user needs to erase text information in the image, the first image contains text information. If the user needs to erase a face, the first image contains facial information.

[0068] This application trains the image recognition model so that the image recognition model can erase the corresponding content area in the first image according to the image content in the training set, and can make the erased image maintain the coordination and consistency of the overall image, thereby improving image processing efficiency.

[0069] In some embodiments of the present application, the target area is a character image area, and locating the target area in the first image includes:

[0070] Performing optical character recognition on the first image to obtain a character detection frame in the first image;

[0071] The target area is located in the first image according to the coordinate information of the character detection frame.

[0072] In the embodiment of the present application, the target area specifically includes a character image area, that is, the user needs to erase the character area in the first image. Specifically, optical character recognition (OCR) is first performed on the first image to detect the positions of the characters in the first image, and a character detection frame is formed based on the positions of the characters.

[0073] Specifically, the first image is preprocessed. In some embodiments, a denoising algorithm is used to remove noise from the first image. Then, based on a trained OCR detection model, the text or characters are captured and their coordinates are located using an OCR detection algorithm. The OCR detection algorithm can capture character detection frames in various scenarios, such as horizontal, vertical, and curved.

[0074] For horizontal and vertical regular character detection boxes, a four-point coordinate box can be used; for curved irregular character detection boxes, an eight-point coordinate box can be used. If there is no text or character information in the current image, an empty character coordinate box is returned.

[0075] The character detection frame is annotated with coordinate information, which refers to the coordinates of the character detection frame in the first image. The coordinate information of the character detection frame is used to locate the target area in the first image. After the second image is obtained through the image processing model, the processed area image is overlaid on the target area based on the coordinate information. This ensures that the generated target image not only maintains the overall image consistency, but also effectively erases the content of the target area that needs to be processed, while ensuring that other similar areas in the image are not mistakenly erased, thereby improving the processing efficiency when erasing specific content in the image.

[0076] In some embodiments of the present application, Figure 6 FIG. 2 shows a second flow chart of an image processing method according to an embodiment of the present application. Figure 6 As shown, the method includes:

[0077] Step 602: locate the text in the original image and record the coordinate information of the text in the original image;

[0078] In step 602, the image is first preprocessed, using a denoising algorithm to remove noise. An OCR detection model is then trained, and text positioning information is obtained using the OCR detection algorithm. This OCR detection algorithm can detect text frames in various scenarios, including horizontal, vertical, and curved. Horizontal and vertical text frames are represented using a four-point coordinate frame, while curved text frames are represented using an eight-point coordinate frame. If the current image contains no text, an empty text coordinate frame is returned.

[0079] Step 604: collect paired data and train a generative adversarial network for text erasure.

[0080] In step 604, pairs of <original image, text-erased image> are collected, and a generative adversarial approach is used to perform model training to obtain an image processing model.

[0081] Specifically, the original image is first acquired, and the corresponding text-erased image is acquired through PS, thereby obtaining a pair of <original image, text-erased image>.

[0082] A generative adversarial network is then trained to perform end-to-end text erasure on the entire image. Specifically, a network model is constructed. The generative network uses the UNET network structure, first downsampling the image to obtain semantic information, then upsampling it back to its original size to obtain the image output. Considering that a single model optimization may not yield good results for complex scenes, a lightweight UNET layer is added after the UNET structure for further processing to obtain the final result. A discriminative network is constructed, consisting of a dual-scale network. The first scale network consists of several convolutional cascades with a stride of 1. No pooling layer is introduced to ensure that the image resolution does not decrease. A second scale network is added after the first scale network, but the input to the second scale network is a double-sampled output of the first network. The ground truth of the dual-scale network is obtained from the erased image and the double-sampled image of the erased image.

[0083] The output of the generative network is input into the discriminative network to determine the difference between the currently generated erased image and the originally labeled erased image, and the difference is used as the loss to backpropagate and optimize the network parameters, ultimately obtaining the optimized network structure.

[0084] Step 606: Use the optimized model to infer the original image to obtain the erased image result;

[0085] In step 606, the optimized generative adversarial model is used to infer the original image to obtain the erased image result. The specific method is to input the original image into the trained model. At this time, the discriminator needs to be removed and only the generator is retained. The output image obtained after the generator is the erased image result.

[0086] Step 608: Map the acquired text coordinate information to the erased image, and crop the erased area where the text coordinates are located;

[0087] In step 608, the text coordinate information obtained in the step is mapped to the erased image result obtained, and the erased area where the text coordinates are located is cropped out; considering that some areas that are very similar to text, such as fences, patterns, flowers and plants, are easily erased as text, in order to ensure that this information is not mistakenly painted, it is necessary to map the text coordinate information recorded from the original image to the erased image obtained. Because the coordinates are non-rectangular frames, it is necessary to set a mask image that is the same size as the input image. The mask image is originally pure black. The area where the coordinate box is located is set to white, and the erased area where the text coordinates to be cropped are located can be obtained.

[0088] Step 610: Paste the erased area back to the original image to obtain the final text-erased image.

[0089] In step 610, the cropped erased area is pasted back to the original image to obtain the final text-erased image.

[0090] The specific method is to obtain the mask image corresponding to the text area that needs to be erased. At this time, you only need to paste the erased area corresponding to the pure white area in the mask image back to the original image. In this way, you can get the final text-erased image you need, thereby avoiding accidental painting of areas that are very similar to text.

[0091] In some embodiments of the present application, an image processing device is provided. Figure 7 FIG. 1 shows a structural block diagram of an image processing device according to an embodiment of the present application. Figure 7 As shown, the image processing device 700 includes:

[0092] A positioning module 702 is configured to locate a target area in the first image and obtain position information of the target area;

[0093] an erasing module 704 for erasing image information of a preset area in the first image using an image processing model to obtain an erased second image, wherein the second image has the same size as the first image, and the preset area includes the target area;

[0094] A capture module 706 is configured to capture an area image corresponding to the target area in the second image according to the position information;

[0095] The processing module 708 is configured to generate a processed target image according to the regional image and the first image.

[0096] The embodiment of the present application captures the area image at the same position in the second image after erasing through the image processing model based on the position information of the marked target area, and combines the area image with the original first image, that is, only the part of the second image that the user needs to eliminate is selected to process the original image. Therefore, the target image finally obtained erases the content of the target area that needs to be processed while retaining the coordination and consistency of the overall image, and at the same time ensures that other similar areas in the image will not be erased by mistake, thereby improving the processing efficiency when erasing specific content in the image.

[0097] In some embodiments of the present application, the image processing apparatus further includes: an overlay module configured to overlay the regional image on the target region according to the position information to obtain a target image.

[0098] After obtaining the regional image, the embodiment of the present application also overlays the captured regional image processed by the image processing model onto the original first image based on the coordinate information of the target area, so that the image content in the target area is completely replaced by the regional image. In this way, in the replaced target image, only the area that needs to be erased is replaced by the processed image. While retaining the coordination and consistency of the overall image, the content of the target area that needs to be processed is erased, and at the same time, it is ensured that other similar areas in the image will not be erased by mistake, thereby improving the processing efficiency when erasing specific content in the image.

[0099] In some embodiments of the present application, the processing device further includes:

[0100] an acquisition module, configured to acquire a first training image and a second training image, wherein the second training image is an image obtained by removing preset image information from the first training image;

[0101] A training module, configured to train a preset model using a first training image and a second training image to obtain a trained image processing model, wherein the image processing model includes a first network and a second network;

[0102] The erasing module is further configured to perform an erasing process on the first image through the first network to obtain a processed third image;

[0103] a sampling module, configured to perform downsampling processing on the third image to obtain a fourth image;

[0104] The erasing module is further configured to perform an erasing process on the fourth image through the second network to obtain a processed fifth image;

[0105] The sampling module is further configured to perform upsampling processing on the fifth image to obtain a second image.

[0106] This application trains the image recognition model so that the image recognition model can erase the corresponding content area in the first image according to the image content in the training set, and can make the erased image maintain the coordination and consistency of the overall image, thereby improving image processing efficiency.

[0107] In some embodiments of the present application, the target area is a character image area, and the processing device further includes:

[0108] a recognition module, configured to perform optical character recognition on the first image and obtain a character detection frame in the first image;

[0109] The positioning module is further used to locate the target area in the first image according to the coordinate information of the character detection frame.

[0110] The embodiment of the present application locates the target area in the first image, and after obtaining the second image through the image processing model, the processed area image is overlaid on the target area according to the coordinate information, so that the generated target image not only ensures the coordination and consistency of the overall image, but also effectively erases the content of the target area that needs to be processed, while ensuring that other similar areas in the image will not be erased by mistake, thereby improving the processing efficiency when erasing specific content in the image.

[0111] The image processing device in the embodiment of the present application can be an electronic device or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or a device other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. It can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., and the embodiment of the present application does not specifically limit it.

[0112] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0113] The image processing device provided in the embodiment of the present application can implement each process implemented in the above method embodiment. To avoid repetition, it will not be described here.

[0114] Optionally, an embodiment of the present application further provides an electronic device, Figure 8 FIG. 1 shows a structural block diagram of an electronic device according to an embodiment of the present application. Figure 8 As shown, the electronic device 800 includes a processor 802, a memory 804, and a program or instruction stored in the memory 804 and executable on the processor 802. When the program or instruction is executed by the processor 802, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, they will not be described here.

[0115] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0116] Figure 9 A schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.

[0117] The electronic device 900 includes but is not limited to: a radio frequency unit 901, a network module 902, an audio output unit 903, an input unit 904, a sensor 905, a display unit 906, a user input unit 907, an interface unit 908, a memory 909 and a processor 910.

[0118] Those skilled in the art will understand that the electronic device 900 may also include a power source (such as a battery) to power each component, and the power source may be logically connected to the processor 910 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system. Figure 9 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be repeated here.

[0119] The processor 910 is configured to locate a target area in the first image and obtain location information of the target area;

[0120] Erasing image information of a preset area in the first image through an image processing model to obtain an erased second image, wherein the second image has the same size as the first image, and the preset area includes the target area;

[0121] intercepting an area image corresponding to the target area in the second image according to the position information;

[0122] A processed target image is generated according to the region image and the first image.

[0123] The embodiment of the present application captures the area image at the same position in the second image after erasing through the image processing model based on the position information of the marked target area, and combines the area image with the original first image, that is, only the part of the second image that the user needs to eliminate is selected to process the original image. Therefore, the target image finally obtained erases the content of the target area that needs to be processed while retaining the coordination and consistency of the overall image, and at the same time ensures that other similar areas in the image will not be erased by mistake, thereby improving the processing efficiency when erasing specific content in the image.

[0124] Optionally, the processor 910 is further configured to overlay the regional image on the target region according to the position information to obtain a target image.

[0125] After obtaining the regional image, the embodiment of the present application also overlays the captured regional image processed by the image processing model onto the original first image based on the coordinate information of the target area, so that the image content in the target area is completely replaced by the regional image. In this way, in the replaced target image, only the area that needs to be erased is replaced by the processed image. While retaining the coordination and consistency of the overall image, the content of the target area that needs to be processed is erased, and at the same time, it is ensured that other similar areas in the image will not be erased by mistake, thereby improving the processing efficiency when erasing specific content in the image.

[0126] Optionally, the processor 910 is further configured to obtain a first training image and a second training image, wherein the second training image is image data obtained by erasing a preset area in the first training image;

[0127] Training a preset model using the first training image and the second training image to obtain a trained image processing model, where the image processing model includes a first network and a second network;

[0128] Erasing image information of a preset area in the first image, including:

[0129] performing erasing processing on the first image through the first network to obtain a processed third image;

[0130] downsampling the third image to obtain a fourth image;

[0131] performing erasing processing on the fourth image through the second network to obtain a processed fifth image;

[0132] Upsampling is performed on the fifth image to obtain a second image.

[0133] This application trains the image recognition model so that the image recognition model can erase the corresponding content area in the first image according to the image content in the training set, and can make the erased image maintain the coordination and consistency of the overall image, thereby improving image processing efficiency.

[0134] Optionally, the target area is a character image area, and the processor 910 is further configured to perform optical character recognition on the first image to obtain a character detection frame in the first image;

[0135] The target area is located in the first image according to the coordinate information of the character detection frame.

[0136] The embodiment of the present application locates the target area in the first image, and after obtaining the second image through the image processing model, the processed area image is overlaid on the target area according to the coordinate information, so that the generated target image not only ensures the coordination and consistency of the overall image, but also effectively erases the content of the target area that needs to be processed, while ensuring that other similar areas in the image will not be erased by mistake, thereby improving the processing efficiency when erasing specific content in the image.

[0137] It should be understood that in an embodiment of the present application, the input unit 904 may include a graphics processing unit (GPU) 9041 and a microphone 9042, and the graphics processor 9041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 906 may include a display panel 9061, and the display panel 9061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 907 includes a touch panel 9071 and at least one of other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 may include two parts: a touch detection device and a touch controller. Other input devices 9072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and an operating stick, which will not be repeated here.

[0138] The memory 909 can be used to store software programs and various data. The memory 909 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 909 may include a volatile memory or a non-volatile memory, or the memory 909 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 909 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0139] Processor 910 may include one or more processing units. Optionally, processor 910 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 910.

[0140] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0141] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.

[0142] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0143] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0144] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0145] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0147] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. An image processing method, characterized in that: include: Locating a target area in the first image to obtain position information of the target area; Erasing image information of a preset area in the first image to obtain an erased second image, wherein the size of the second image is the same as that of the first image, and the preset area includes the target area; intercepting an area image corresponding to the target area in the second image according to the position information; According to the position information, the regional image is overlaid on the target area to obtain a target image.

2. The image processing method according to claim 1, wherein: Before erasing the image information of the preset area in the first image, the method further includes: Acquire a first training image and a second training image, wherein the first training image includes preset image information, and the second training image is an image obtained by removing the preset image information from the first training image; Training a preset model using the first training image and the second training image to obtain a trained image processing model, wherein the image processing model includes a first network and a second network; Erasing image information of a preset area in the first image includes: performing erasing processing on the first image through the first network to obtain a processed third image; performing downsampling processing on the third image to obtain a fourth image; performing erasing processing on the fourth image through the second network to obtain a processed fifth image; Upsampling is performed on the fifth image to obtain the second image.

3. The image processing method according to claim 1 or 2, characterized in that: The target area is a character image area, and locating the target area in the first image includes: performing optical character recognition on the first image to obtain a character detection frame in the first image; The target area is located in the first image according to the coordinate information of the character detection frame.

4. An image processing device, characterized in that include: a positioning module, configured to locate a target area in the first image and obtain position information of the target area; an erasing module, configured to erase image information of a preset area in the first image to obtain an erased second image, wherein the size of the second image is the same as that of the first image, and the preset area includes the target area; a capture module, configured to capture, in the second image, an image of a region corresponding to the target region based on the position information; a processing module, configured to generate a processed target image based on the regional image and the first image; The covering module is used to cover the regional image on the target area according to the position information to obtain the target image.

5. The image processing device according to claim 4, wherein Also includes: an acquisition module, configured to acquire a first training image and a second training image, wherein the second training image is image data obtained by erasing the preset area in the first training image; A training module, configured to train a preset model using the first training image and the second training image to obtain a trained image processing model, wherein the image processing model includes a first network and a second network; The erasing module is further configured to perform an erasing process on the first image through the first network to obtain a processed third image; a sampling module, configured to perform downsampling processing on the third image to obtain a fourth image; The erasing module is further configured to perform an erasing process on the fourth image through the second network to obtain a processed fifth image; The sampling module is further configured to perform upsampling processing on the fifth image to obtain the second image.

6. The image processing device according to claim 4 or 5, characterized in that The target area is a character image area, and the processing device further includes: a recognition module, configured to perform optical character recognition on the first image and obtain a character detection frame in the first image; The positioning module is further configured to locate the target area in the first image according to the coordinate information of the character detection frame.

7. An electronic device, characterized in that: The image processing method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the image processing method according to any one of claims 1 to 3 are implemented.

8. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the image processing method according to any one of claims 1 to 3 are implemented.

Images