Image processing method, apparatus, device, medium, and program product

Abstract

The present disclosure provides an image processing method, device, equipment, storage medium and program product. The method comprises: obtaining an original image to be processed; performing high-pass filtering on a target region of the original image to obtain an original high-pass filtered image, and performing low-pass filtering on the target region to obtain an original low-pass filtered image; processing the original image based on a first network to determine a target attribute of the target region; processing the original high-pass filtered image and the original low-pass filtered image based on the target attribute to obtain a target high-pass filtered image and a target low-pass filtered image; obtaining a target region image based on the target high-pass filtered image and the target low-pass filtered image; and fusing the target region image into the original image to obtain a target image.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to an image processing method and device, equipment, medium and program product. BACKGROUND

[0002] In real life, people often need to beautify the portrait in the image. However, the existing beautification method is based on the whole image to perform skin beautification, instead of processing only specific parts, resulting in poor beautification effect of specific parts, for example, unable to meet the user's requirement for wrinkle removal effect of the neck in the image. SUMMARY

[0003] The present disclosure provides an image processing method, device, equipment, storage medium and program product to solve the technical problem of poor wrinkle removal effect of the neck in the image to some extent.

[0004] In a first aspect, the present disclosure provides an image processing method, comprising:

[0005] obtaining an original image to be processed;

[0006] performing high-pass filtering on a target region of the original image to obtain an original high-pass filtered image, and performing low-pass filtering on the target region to obtain an original low-pass filtered image;

[0007] processing the original image based on a first network to determine a target attribute of the target region;

[0008] processing the original high-pass filtered image and the original low-pass filtered image based on the target attribute to obtain a target high-pass filtered image and a target low-pass filtered image;

[0009] obtaining a target region image based on the target high-pass filtered image and the target low-pass filtered image;

[0010] fusing the target region image into the original image to obtain a target image.

[0011] In a second aspect, the present disclosure provides an image processing device, comprising:

[0012] an acquisition module configured to obtain an original image to be processed;

[0013] a filtering module configured to perform high-pass filtering on a target region of the original image to obtain an original high-pass filtered image, and perform low-pass filtering on the target region to obtain an original low-pass filtered image;

[0014] The first network is configured to process the original image, determine a target attribute of the target region, process the original high-pass filtered image and the original low-pass filtered image based on the target attribute to obtain a target high-pass filtered image and a target low-pass filtered image, obtain a target region image based on the target high-pass filtered image and the target low-pass filtered image, and fuse the target region image into the original image to obtain a target image.

[0015] In a third aspect, the present disclosure provides an electronic device, comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the programs comprise instructions for performing the method according to the first aspect or the second aspect.

[0016] In a fourth aspect, the present disclosure provides a non-volatile computer readable storage medium containing a computer program, which, when executed by one or more processors, causes the processors to perform the method according to the first aspect or the second aspect.

[0017] In a fifth aspect, the present disclosure provides a computer program product comprising computer program instructions, which, when executed on a computer, cause the computer to perform the method according to the first aspect.

[0018] As can be seen from the above, the present disclosure provides an image processing method, device, equipment, medium and program product, which processes a target region in an original image that has been subjected to high-pass filtering and low-pass filtering based on a target attribute adaptively determined from the original image by a first network, obtains a target high-pass filtered image and a target low-pass filtered image, obtains a target region image based on the target high-pass filtered image and the target low-pass filtered image, and fuses the target region image into the original image to obtain a corresponding target image. Thus, the target region in a portrait image is processed without changing the skin quality and other features of other regions in the image, so that the processed image is more natural and the processing effect of image processing is improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present disclosure or the related art, the drawings needed to be used in the embodiments or the related art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor.

[0020] Figure 1 FIG. 1 is a schematic diagram of an image processing architecture according to an embodiment of the present disclosure.

[0021] Figure 2A schematic diagram of a hardware structure of an exemplary electronic device of an embodiment of the present disclosure.

[0022] Figure 3 A schematic diagram of an image processing method of an embodiment of the present disclosure.

[0023] Figure 4 A flowchart of an image processing method of an embodiment of the present disclosure.

[0024] Figure 5 A schematic diagram of an image processing apparatus of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to make the objects, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below with reference to specific embodiments and drawings.

[0026] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood as the common meanings understood by those skilled in the art to which the present disclosure belongs. The terms “first”, “second” and similar terms used in the embodiments of the present disclosure do not represent any order, number or importance, but are only used to distinguish different components. The terms “include”, “contain” and similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms “connect” or “connected” and similar terms do not mean physical or mechanical connection, but can include electrical connection, whether direct or indirect. The terms “upper”, “lower”, “left”, “right” and the like only represent relative positional relationships, and when the absolute positions of the described objects change, the relative positional relationships may also change accordingly.

[0027] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type, use range, use scenario and the like of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained in a proper manner according to relevant laws and regulations.

[0028] For example, in response to receiving the active request of the user, the user is sent prompt information to explicitly prompt the user that the operation requested to be performed will require obtaining and using the personal information of the user. Thus, the user can voluntarily choose whether to provide the personal information to the electronic device, application program, server or storage medium and the like software or hardware that performs the operation of the technical solution of the present disclosure according to the prompt information.

[0029] As an optional but non-limiting implementation, in response to receiving the active request of the user, the manner of sending the prompt information to the user may, for example, be a pop-up window manner, in which the prompt information may be presented in the form of text. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0030] It can be understood that the above notification and user authorization obtaining process is only illustrative and does not limit the implementation of the present disclosure. Other manners that meet relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0031] Figure 1 A schematic diagram of an image processing architecture of an embodiment of the present disclosure is shown. Referring to Figure 1 The image processing architecture 100 can include a server 110, a terminal 120, and a network 130 providing a communication link. The server 110 and the terminal 120 can be connected through the wired or wireless network 130. The server 110 can be a stand-alone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, security services, CDN, and other basic cloud computing services.

[0032] The terminal 120 can be implemented in hardware or software. For example, when the terminal 120 is implemented in hardware, it can be various electronic devices with a display screen and supporting page display, including but not limited to smartphones, tablet computers, e-book readers, laptop computers, and desktop computers, etc. When the terminal 120 is implemented in software, it can be installed in the above-mentioned electronic devices; it can be implemented as multiple software or software modules (such as software or software modules for providing distributed services), or as a single software or software module, which is not specifically limited here.

[0033] It should be noted that the image processing method provided by the embodiments of the present disclosure can be executed by the terminal 120 or the server 110. It should be understood that Figure 1 The number of terminals, networks, and servers in

[0034] Figure 2 A hardware structure schematic diagram of an exemplary electronic device 200 provided by the embodiments of the present disclosure is shown. As Figure 2As shown, the electronic device 200 can include a processor 202, a memory 204, a network module 206, a peripheral interface 208, and a bus 210. The processor 202, the memory 204, the network module 206, and the peripheral interface 208 are communicatively connected to each other within the electronic device 200 via the bus 210.

[0035] The processor 202 can be a central processing unit (CPU), a graphics processor, a neural processing unit (NPU), a microcontroller (MCU), a programmable logic device, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or one or more integrated circuits. The processor 202 can be configured to perform functions related to the techniques described in the present disclosure. In some embodiments, the processor 202 can also include multiple processors integrated as a single logical component. For example, as shown in FIG. 2, the processor 202 can include multiple processors 202a, 202b, and 202c. Figure 2 As shown, the processor 202 can include multiple processors 202a, 202b, and 202c.

[0036] The memory 204 can be configured to store data (e.g., instructions, computer code, etc.). As shown, the memory 204 stores data can include program instructions (e.g., program instructions for implementing the image processing method of the embodiments of the present disclosure) and data to be processed (e.g., the memory can store configuration files of other modules, etc.). The processor 202 can also access the program instructions and data stored in the memory 204, and execute the program instructions to operate on the data to be processed. The memory 204 can include volatile storage or non-volatile storage. In some embodiments, the memory 204 can include random access memory (RAM), read-only memory (ROM), optical disc, magnetic disc, hard disk, solid state disk (SSD), flash memory, memory stick, etc. Figure 2 As shown, the memory 204 stores data can include program instructions (e.g., program instructions for implementing the image processing method of the embodiments of the present disclosure) and data to be processed (e.g., the memory can store configuration files of other modules, etc.). The processor 202 can also access the program instructions and data stored in the memory 204, and execute the program instructions to operate on the data to be processed. The memory 204 can include volatile storage or non-volatile storage. In some embodiments, the memory 204 can include random access memory (RAM), read-only memory (ROM), optical disc, magnetic disc, hard disk, solid state disk (SSD), flash memory, memory stick, etc.

[0037] The network module 206 can be configured to provide communication between the electronic device 200 and other external devices via a network. The network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, near field communication (NFC), etc.), a cellular network, the Internet, or a combination thereof. It can be understood that the type of network is not limited to the specific examples described above. In some embodiments, the network module 306 can include any combination of any number of network interface controllers (NICs), radio frequency modules, transceivers, modems, routers, gateways, adapters, cellular network chips, etc.

[0038] The peripheral interface 208 can be configured to connect the electronic device 200 with one or more peripheral devices to realize information input and output. For example, the peripheral devices can include input devices such as keyboards, mice, touchpads, touch screens, microphones, various sensors, and the like, and output devices such as displays, speakers, vibrators, indicator lights, and the like.

[0039] The bus 210 can be configured to transmit information between various components (e.g., the processor 202, the memory 204, the network module 206, and the peripheral interface 208) of the electronic device 200, such as an internal bus (e.g., a processor-memory bus), an external bus (a USB port, a PCI-E bus), and the like.

[0040] It should be noted that although the architecture of the electronic device 200 described above only shows the processor 202, the memory 204, the network module 206, the peripheral interface 208, and the bus 210, in the specific implementation process, the architecture of the electronic device 200 can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the architecture of the electronic device 200 described above can also only contain components necessary for implementing the embodiments of the present disclosure, and does not necessarily contain all the components shown in the figure.

[0041] In order to obtain better image effects, people often use application programs with beautifying functions to process target regions of images, for example, to process wrinkles on the neck, and the neck is the target region. Since the texture of the target region (for example, the wrinkles on the neck) is often relatively small compared to the entire image, it is easy to be mistaken by the image processing model as image noise for processing, resulting in that when the target region is processed, it is not the texture of the target region itself, but the same processing method as other noise of the entire image, which cannot achieve appropriate image processing of the target region of the image, so that the image processing effect is not good. Therefore, how to improve the image processing effect of the target region in the image has become a technical problem to be solved.

[0042] In view of this, the embodiments of the present disclosure provide an image processing method, device, equipment, storage medium and program product. Based on the target attribute adaptively determined by the first network according to the original image, the target region in the original image subjected to high-pass filtering and low-pass filtering is processed to obtain a target high-pass filtered image and a target low-pass filtered image, and then a target region image is obtained according to the target high-pass filtered image and the target low-pass filtered image, and is fused into the original image to obtain a corresponding target image. Thus, the target region in the portrait image is processed without changing the skin quality and other characteristics of other regions in the image, so that the processed image is more natural, and the image processing effect is improved.

[0043] Specifically, based on the neck line removal processing on the neck in the image, the target high-pass filtered image and the target low-pass filtered image can be obtained by performing the neck line removal processing on the neck region image in the original image which has been high-pass filtered and low-pass filtered based on the target neck attribute adaptively determined by the first network from the original image, and then the target neck image can be obtained based on the target high-pass filtered image and the target low-pass filtered image, and fused into the original image to obtain the corresponding target image. Thus, the neck line removal processing on the neck region in the portrait image is realized without changing the skin quality and other features in the image, so that the processed image is more natural, and the neck line removal effect of the image processing is improved.

[0044] Referring to Figure 3 , Figure 3 A schematic diagram of an image processing method according to an embodiment of the present disclosure is shown. Figure 3 In the embodiment, the model architecture of the image processing model 300 can adopt a generative adversarial network (GAN), including a generative network 310 and a discriminative network 320. The initial adversarial network can be trained to obtain the image processing model 300 by using the first training sample and a preset supervision strategy. The generative network 310 in the image processing model 300 can be used as the neck line removal network 330 in actual application to perform the neck line removal processing on the original image to be processed and obtain the target image after the neck line removal.

[0045] In some embodiments, the first training sample can include at least one sample pair, each sample pair including an original training image A, a neck correction image B corresponding to the original training image A, and an original neck mask region corresponding to the original training image A, and a corrected neck mask region corresponding to the neck correction image B. In some embodiments, the neck feature correction can be performed on the original training image A based on a preset neck line removal standard to obtain the corresponding neck correction image B. Specifically, a first number (for example, 2800) of original training images A can be obtained, and the original training images A include neck images meeting the image quality requirements, for example, the resolution of the neck image is not less than 512 1024 pixels. In some embodiments, key point detection can be performed on the original training image A, and an original neck region can be determined based on the neck key points; the pixels of the original neck region in the original training image A can be set to a first pixel (e.g., 255, i.e., white), and the pixels of the non-original neck region can be set to a second pixel (e.g., 0, i.e., black), to obtain a corresponding original neck mask region. In some embodiments, key point detection can be performed on the neck correction image B, and a correction neck region can be determined based on the neck key points; the pixels of the correction neck region in the neck correction image B can be set to a first pixel (e.g., 255, i.e., white), and the pixels of the non-neck region can be set to a second pixel (e.g., 0, i.e., black), to obtain a corresponding correction neck mask region.

[0046] The original training image A in the first training sample can cover as many types of portrait images as possible, for example, the necks of men and women, the necks of various races, the necks of various age groups, and the necks of various angles, so as to ensure the richness of the training data and improve the accuracy of the model training. Then, according to the preset neck wrinkle removal effect standard, the features of the neck are manually corrected, and the neck wrinkles of the original training image A are removed, so as to obtain the corresponding neck correction image B. It should be understood that when the neck features are manually corrected, other features such as skin color, facial features, and skin quality outside the neck can not be processed, so as to ensure that the image processing model obtained by training is only targeted to remove wrinkles in the neck region, so that the neck maintains a smooth and delicate skin texture, and the entire image is not affected. other areas are changed, affecting the authenticity of the image, resulting in unnatural image processing effect.

[0047] In some embodiments, the generative adversarial network can be trained based on the first training sample and the preset supervision strategy to obtain the neck wrinkle removal network. In order to ensure that the generative network 310 in the image processing model 300 can adaptively learn the network parameters of the neck wrinkle removal effect during the training process, the preset supervision strategy can be set, and the image processing model 300 can be supervised and trained using the first training sample, so as to obtain the generative network 310 that can adaptively match the input image with appropriate neck wrinkle removal attributes as the neck wrinkle removal network 330.

[0048] In some embodiments, the preset supervision strategy can comprise: setting a light condition parameter of an input image of the generative adversarial network to simulate a light condition of the input image. Further, the light condition parameter can be randomly set. In the training phase, the input image input to the generative adversarial network can be an image in the first training sample, for example, the original training image A and the corresponding face correction image B. Wherein, the light condition of the input image in the image processing model 300 is simulated based on the light condition parameter, which can increase the richness of the input data, so that the image processing model 300 can process more diversified input images in the training process, thereby improving the accuracy of the image processing model 300. Specifically, for example, the gamma correction algorithm can be used to realize the light simulation, that is, the light condition parameter can be set by setting the related parameters of the gamma correction algorithm.

[0049] In some embodiments, the preset supervision strategy can comprise:

[0050] The generative network generates a corresponding first image and a first neck mask region of the first image based on the original training image in the first training sample;

[0051] Calculate the cross-entropy loss function of the generative network and the discriminative network based on the first image, the original training image, the first neck mask region and the corrected neck mask region;

[0052] Adjust the first discriminative network parameter of the discriminative network based on the cross-entropy loss function to maximize the cross-entropy loss function, and adjust the first generative network parameter of the generative network based on the cross-entropy loss function to minimize the cross-entropy loss function.

[0053] Further, in some embodiments, the cross-entropy loss function comprises an image cross-entropy loss function and / or a mask cross-entropy loss function; wherein,

[0054] The image cross-entropy loss function is calculated based on the first image and the original training image;

[0055] The mask cross-entropy loss function is calculated based on the first neck mask region and the original neck mask region.

[0056] Specifically, the cross-entropy loss function of the generative network 310 and the discriminative network 320 Can include an image cross-entropy loss function And a mask cross-entropy loss function The sum of. Wherein, the image cross-entropy loss function The image cross-entropy loss function can include a sum of a first expected function about a first logarithmic function and a second expected function about a second logarithmic function, where the first logarithmic function includes a logarithmic function about a first discrimination result of the original training image x, and the second logarithmic function includes a logarithmic function about a difference between a first preset value (e.g., 1) and a second discrimination result of a generated result for the original training image x. The mask cross-entropy loss function can include a sum of a third expected function about a third logarithmic function and a fourth expected function about a fourth logarithmic function, where the third logarithmic function includes a logarithmic function about a third discrimination result of the original neck mask region x_mask, and the fourth logarithmic function includes a logarithmic function about a difference between a second preset value (e.g., 1) and a fourth discrimination result of a generated result for the first neck mask region G(x)_mask.

[0057] The training process of the image processing model 300 can be that the two networks of the generation network 310 and the discrimination network 320 are trained separately and alternately. For example, the generation network 310 can be fixed first, and the discrimination network 320 can be trained to update the first discrimination network parameters of the discrimination network 320. At this time, the first discrimination network parameters of the discrimination network 320 can be adjusted so that the discrimination network 320 outputs 1 when its input is the face correction image B (i.e., D(B)=1), outputs 0 when its input is the first image A' (i.e., D(A')=0), and outputs 1 when its input is the correction neck mask region of the face correction image B (i.e., D(B_mask)=1). Then, in the update stage of the discrimination network 320, the training target of the image processing model 300 is to maximize the image cross-entropy loss function cross-entropy function and the mask cross-entropy loss function .

[0058] Then, the generation network 310 can be fixed, and the generation network 310 can be trained to update the first generation network parameters of the generation network 310. At this time, the first generation network parameters of the generation network 310 can be adjusted so that the discrimination network 320 outputs 1 when the first image A' output by the generation network 310 is input to the discrimination network 320 (i.e., D(A')=1), and the discrimination network 320 outputs 1 when the first neck mask region A'_mask of the first image A' is input to the discrimination network 320 (i.e., D(A'_mask)=1). Then, the training target of the image processing model 300 is to minimize the cross-entropy loss function , that is, to minimize the image cross-entropy loss function cross-entropy function and the mask cross-entropy loss function . Since the first network parameters of the discrimination network 320 are unchanged at this time, the first expected function and the third expected function are also unchanged, and the cross-entropy function .

[0059] The training of the discriminative network 320 is performed repeatedly while fixing the generative network 310, and the training of the generative network 310 is performed repeatedly while fixing the discriminative network 320, until a Nash equilibrium is reached. In this way, the results generated by the generative network 310 can be made more realistic.

[0060] In some embodiments, the preset supervision strategy can include:

[0061] The generative network generates a corresponding first image based on the original training image in the first training sample;

[0062] A feature correction loss function is calculated based on the first image and the neck correction image in the first training sample;

[0063] Second generative network parameters of the generative network and second discriminative network parameters of the discriminative network are adjusted based on the feature correction loss function, so as to minimize the feature correction loss function.

[0064] Further, in some embodiments, the calculation of the feature correction loss function based on the first image and the neck correction image in the first training sample further includes:

[0065] Feature extraction is performed on the first image to obtain first high-dimensional semantic features and first low-dimensional texture features, and feature extraction is performed on the neck correction image in the first training sample to obtain second high-dimensional semantic features and second low-dimensional texture features;

[0066] A first high-dimensional feature loss function is calculated based on the first high-dimensional semantic features and the second high-dimensional semantic features, and a first low-dimensional feature loss function is calculated based on the first low-dimensional texture features and the second low-dimensional texture features;

[0067] The feature correction loss function is obtained based on the sum of the first high-dimensional feature loss function and the second low-dimensional feature loss function.

[0068] Wherein, the high-dimensional semantic features can refer to features obtained based on deep layers of an image processing model, which can be close to the output layer, have low resolution, small size of feature maps, high degree of abstraction, contain more global information, and the like. The low-dimensional texture features can refer to features obtained based on shallow layers of the image processing model, which can be close to the input layer, have high resolution, large size of feature maps, contain more detailed information, are easy to align with the original training image, and the like. By correcting these two kinds of features, the advantages of both can be combined, thereby improving the training effect of the image processing model and the effect of face neck wrinkle removal processing.

[0069] Specifically, the generation network 310 generates a corresponding first image A' based on the original training image A. The first image A' can be subjected to feature extraction by a visual processor in the image processing model 300 to obtain a first high-dimensional semantic feature F1_A' and a first low-dimensional texture feature F2_A' of the first image; and a neck correction image in the first training sample is subjected to feature extraction to obtain a second high-dimensional semantic feature F3 and a second low-dimensional texture feature F4. Then, a feature correction loss function L_F = first high-dimensional feature loss function l1(F1_A', F3) + first low-dimensional feature loss function l1(F2_A', F4) can be calculated, where l1 is a mean absolute error function. In some embodiments, the preset supervision strategy can include:

[0070] The generation network generates a corresponding first image and a first neck mask region of the first image based on the original training image in the first training sample;

[0071] A pixel correction loss function is calculated based on the first image, the first neck mask region, a neck correction image in the first training sample, and the corrected neck mask.

[0072] The third generation network parameters of the generation network and the third discriminant network parameters of the discriminant network are adjusted based on the pixel correction loss function to minimize the pixel correction loss function.

[0073] Further, in some embodiments, the pixel correction loss function includes a neck pixel correction loss function; wherein the neck pixel correction loss function is calculated based on the first neck mask region and the corrected neck mask region in the first training sample. Specifically, the neck pixel correction loss function L_JX = l1(A'_mask, B_mask) = l1(F_A' mask_A', F_B mask_B). l1 is a mean absolute error function, A'_mask is a neck mask feature of the first image A', and B_mask is a neck mask feature of the neck correction image B. The neck mask feature A'_mask of the first image A' can be obtained based on the image feature F_A' of the first image A' and the neck mask part mask_A' of the first image A', for example, A'_mask = F_A' mask_A'. The neck mask feature B_mask of the neck correction image B can be obtained based on the image feature F_B of the neck correction image B and the neck mask part mask_B of the neck correction image B, for example, B_mask = F_B mask_B.

[0074] In some embodiments, the pixel correction loss function can further include an image pixel correction loss function, wherein the image pixel correction loss function is calculated based on the first image and the neck correction image in the first training sample. Specifically, the image pixel correction loss function L_TX= l1(A', B) = l1(F_A', F_B). l1 is a mean absolute error function, F_A' is an image feature of the first image A', and F_B is an image feature of the neck correction image B.

[0075] In some embodiments, the pixel correction loss function can include a neck pixel correction loss function and a corresponding first weight, and an image pixel correction loss function and a corresponding second weight, the first weight being greater than the second weight. For example, the pixel correction loss function L_XSJZ= L_JX Q1+L_TX Q2, Q1 is the first weight, and Q2 is the second weight.

[0076] Since the neck lines are more similar to noise for the image processing model, in order to prevent the neck line features from being mistaken for noise for processing, the importance of the neck mask area can be emphasized by setting the neck pixel correction loss function, and the wrinkle removal effect in the mask area can be increased. Further, by assigning different weights to the image pixel correction loss function reflecting the pixel loss of the overall image feature and the neck pixel correction loss function, the image processing model can pay more attention to the neck pixel loss while paying attention to the overall image pixel loss in the training stage, thereby further increasing the wrinkle removal effect in the mask area.

[0077] In some embodiments, the preset supervision strategy can include:

[0078] determining an original attention area of the original training image in the first training sample, and removing the original attention area in the original training image to obtain a first intermediate image; and determining a correction attention area of the neck correction image, and removing the correction attention area in the neck correction image to obtain a second intermediate image;

[0079] obtaining an attention loss function based on the first intermediate image and the second intermediate image;

[0080] adjusting fourth generation network parameters of the generation network and fourth discriminant network parameters of the discriminant network based on the attention loss function, so as to minimize the attention loss function.

[0081] The attention region refers to the local information area that the image processing model focuses on during image processing based on the attention mechanism. The original attention region L of the original training image A can be determined based on the attention region mask (Attmask). A The original attention region L A Remove the original training image A, and the remaining image is used as the first intermediate image T. A =A (1.0-Attmask). The correction attention region L of the neck correction image B can be determined based on the attention region mask Attmask. B The attention area L to be corrected B Remove the image from the neck correction image B, and use the remaining image as the second intermediate image T. B =B (1.0-Attmask). By using the first intermediate image T A Second intermediate image T B The attention loss function is obtained as L_Att=l1(T) A T B ) = l1(A (1.0-Attmask), B (1.0-Attmask)), making the first intermediate image T A Second intermediate image T B Maintaining consistency between the two allows the image processing model to be more sensitive to attention regions during training, enabling it to more accurately determine attention regions.

[0082] It should be understood that the first to fourth generator network parameters in this disclosure can all represent the model parameters of the generator network, and they can be the same or different; the first to fourth discriminator network parameters can all represent the model parameters of the discriminator network, and they can be the same or different.

[0083] In some embodiments, the preset supervision strategy may include:

[0084] The original training image is low-pass filtered to obtain the corresponding original low-pass filtered image.

[0085] The generative adversarial network is trained using the original training image and the corresponding original low-pass filtered image.

[0086] The original training image after low-pass filtering will become blurred, and the neck wrinkles will appear to be removed after blurring. By inputting the original training image and its corresponding original low-pass filtered image as prior knowledge for neck wrinkle removal into the image processing model for training, the training efficiency of the image processing model can be further improved.

[0087] In some embodiments, the preset supervision strategy can include:

[0088] The generation network generates a corresponding first image and a first neck mask region of the first image based on the original training image in the first training sample;

[0089] The first neck mask region is subjected to high-pass filtering to obtain a first high-pass filtered image, and low-pass filtering to obtain a first low-pass filtered image, and the corrected neck mask region in the first sample is subjected to high-pass filtering to obtain a second high-pass filtered image, and low-pass filtering to obtain a second low-pass filtered image;

[0090] The filter loss function is calculated based on the first high-pass filtered image, the first low-pass filtered image, the second high-pass filtered image, and the second low-pass filtered image;

[0091] The parameters of the image processing model are adjusted based on the filter loss function, so that the filter loss function is minimized.

[0092] In some embodiments, the filter loss function includes a high-pass filter loss function and a low-pass filter loss function. Further, in some embodiments, the high-pass filter loss function can be obtained based on the first high-pass filtered image and the second high-pass filtered image. In some embodiments, the low-pass filter loss function can be obtained based on the first low-pass filtered image and the second low-pass filtered image. Specifically, the filter loss function L_filter = high-pass filter loss function L_gt + low-pass filter loss function L_dt. The high-pass filter loss function L_gt = l1(gt(A'_mask), gt(B_mask)), gt(A'_mask) is the first high-pass filtered image obtained by high-pass filtering the first neck mask region A'_mask, and gt(B_mask) is the second high-pass filtered image obtained by high-pass filtering the corrected neck mask region B_mask. The low-pass filter loss function L_dt = l1(dt(A'_mask), dt(B_mask)), dt(A'_mask) is the first low-pass filtered image obtained by low-pass filtering the first neck mask region A'_mask, and gt(B_mask) is the second low-pass filtered image obtained by low-pass filtering the corrected neck mask region B_mask, and l1 is the mean absolute error function.

[0093] Wherein, the high-pass filter correction can enable the image processing model to learn the details of the neck wrinkle texture, and the low-pass filter correction can enable the image processing model to learn the overall feeling of the neck wrinkle. Through the combination of the two, the wrinkle removal is performed on the neck region from the overall and the details, which can further improve the wrinkle removal effect of the neck.

[0094] Through one or more of the above preset supervision strategies, the initial generative adversarial network is trained to obtain a trained image processing model. The trained image processing model learns the network parameters of the neck wrinkle removal of the training sample in the training process, and has the adaptive neck wrinkle removal attribute. The generation network in it is used as the neck wrinkle removal network for face neck wrinkle removal processing of the image in actual application, and the original image input by the user is processed to achieve the processing effect of making the neck skin younger.

[0095] In some embodiments, the generative adversarial network is trained based on the first training sample and the preset supervision strategy to obtain the neck wrinkle removal network, which can further include:

[0096] The generative adversarial network is trained based on the first training sample and the preset supervision strategy to obtain a preliminary image processing model;

[0097] The preliminary image processing model is trained based on the second training sample and the preset supervision strategy to obtain the neck wrinkle removal network.

[0098] Further, in some embodiments, the second training sample can be obtained based on the preliminary image processing model, specifically including:

[0099] Obtain a training data set including a plurality of neck images, and input the neck images into a generation network in the generative adversarial network to obtain a second image;

[0100] Input the second image into the preliminary image processing model to obtain a third image corresponding to the second image and subjected to preliminary neck wrinkle removal processing;

[0101] Based on the second image and the corresponding third image, the second training sample is obtained.

[0102] Because the amount of data in the first training sample is relatively small, the preliminary image processing model (including the preliminary generator network and the preliminary discriminator network) trained based on the first training sample is not very stable. In order to increase the stability of the generator image processing model, it is necessary to use the preliminary image processing model to process a large amount of data to obtain a large amount of secondary training dataset as the second training sample. Then, the preliminary image processing model is trained again using the second training sample and a preset supervision strategy to obtain a more stable image processing model. The generator network in this more stable image processing model is used as the neck wrinkle removal network, thereby improving the stability of neck wrinkle removal in practical applications.

[0103] In some embodiments, the third image may undergo preset image processing to obtain a preprocessed third image; the preprocessed third image and the corresponding second image are used as the second training samples. For example, the second image may be subjected to skin smoothing processing, which can further enhance the wrinkle removal effect.

[0104] Specifically, a large number of images, including neck images (e.g., an open-source dataset containing human neck images), can be input into the initial generative network of the initial bio-adversarial network to generate a second image D, resulting in a large dataset set_D. This large dataset set_D is then input into a preliminary image processing model trained based on the first training samples to obtain the output third image E. In some embodiments, the third image E can be further processed to obtain a processed third image E'. The second image D and the corresponding third image E or the processed third image E' can be combined to form a new training data pair, serving as a second training sample. This pair, combined with a pre-defined supervision strategy, is then used to retrain the preliminary image processing model to obtain a new image processing model. The generative network in this new image processing model can be used as a practical neck wrinkle removal network (e.g., [missing information]). Figure 3 The neck wrinkle removal network 330 (in the middle).

[0105] See Figure 4 , Figure 4 A schematic flowchart of an image processing method according to an embodiment of the present disclosure is shown. Figure 4 In the image processing method 400, the following steps may be included.

[0106] Step S410: Obtain the original image to be processed;

[0107] Step S420: Perform high-pass filtering on the target region of the original image to obtain an original high-pass filtered image, and perform low-pass filtering on the target region to obtain an original low-pass filtered image;

[0108] Step S430: Process the original image based on the first network to determine the target attributes of the target region;

[0109] Step S440: Process the original high-pass filtered image and the original low-pass filtered image based on the target attributes to obtain the target high-pass filtered image and the target low-pass filtered image;

[0110] In step S450, a target region image is obtained based on the target high-pass filtered image and the target low-pass filtered image;

[0111] In step S460, the target region image is fused into the original image to obtain the target image.

[0112] Specifically, for the original image imageA to be processed, the user wants to perform image processing (e.g., neck wrinkle removal) on the target region (e.g., neck region) in the original image imageA. Target keypoints (e.g., neck keypoints) can be detected in the original image imageA to obtain the target keypoint P (e.g., neck keypoint P). Based on the target keypoint P (e.g., neck keypoint P), the target region image (e.g., neck region image image_Neck) in the original image imageA can be determined. High-pass filtering can be applied to the target region image to obtain the original high-pass filtered image image_filter_H, and low-pass filtering can be applied to the target region image to obtain the original low-pass filtered image image_filter_L. The original image imageA is then input into a trained first network (e.g., ...). Figure 3 The first network (330) can adaptively determine the target attributes (e.g., target neck attributes) of the input data based on the input data. Therefore, image processing (e.g., neck wrinkle removal) can be performed on the original high-pass filtered image (image_filter_H) and the original low-pass filtered image (image_filter_L) based on these target attributes, resulting in the target high-pass filtered image (image_target_H) and the target low-pass filtered image (image_target_L). The first network can also obtain the target region image (e.g., the target neck image) image' after image processing (e.g., neck wrinkle removal) based on the target high-pass filtered image (image_target_H) and the target low-pass filtered image (image_target_L). Furthermore, the target region image image' can be fused into the original image image A to obtain the target image image A' after image processing. Thus, high-pass filtering improves the detail and texture of the image processing (e.g., neck wrinkle removal), while low-pass filtering improves the overall feel of the image processing (e.g., neck wrinkle removal). By combining these two methods, image processing (e.g., neck wrinkle removal) is performed on the neck region from both an overall and detailed perspective, further enhancing the effect of image processing (e.g., neck wrinkle removal).

[0113] According to the image processing method, the target region in the original image that has undergone high-pass filtering and low-pass filtering is processed based on the target attribute of the original image adaptively determined by the first network to obtain a target high-pass filtered image and a target low-pass filtered image, and then a target region image is obtained according to the target high-pass filtered image and the target low-pass filtered image, and is fused into the original image to obtain a corresponding target image. Thus, the target region in the portrait image is processed without changing the skin quality and other features of other regions in the image, so that the processed image is more natural, and the processing effect of image processing is improved. In the application program of image processing, the skin of the neck region can be improved by one key based on the adaptive smoothing attribute of the neck wrinkle removal network, so that the skin of the neck region is rejuvenated.

[0114] In some embodiments, the target attribute of the target region is determined based on processing of the original image by the first network, including:

[0115] The original image (for example, the original image imageA in method 400) is low-pass filtered to obtain an original filtered image;

[0116] The first network determines the target attribute (for example, the target neck attribute in method 400) based on the original filtered image and the original image.

[0117] The original image that has undergone low-pass filtering will become blurred, and the original image after blurring looks like it has undergone image processing. Using the original filtered image corresponding to the original image as prior knowledge of image processing and combining the original image to determine the target attribute is beneficial to improve the accuracy of the target attribute, so that the target attribute can be more adapted to the original image.

[0118] In some embodiments, the high-pass filtering of the target region of the original image to obtain an original high-pass filtered image and the low-pass filtering of the target region to obtain an original low-pass filtered image include:

[0119] The target feature is extracted based on the second network to obtain a target feature;

[0120] The target feature is corrected based on the target attribute to obtain a corrected target feature;

[0121] The corrected target region image is obtained based on the corrected target feature;

[0122] The original high-pass filtered image is obtained by high-pass filtering the corrected target region image of the first image, and the original low-pass filtered image is obtained by low-pass filtering the corrected target region image.

[0123] The first network and the second network can be used for neck wrinkle removal processing of the image, and the neck region in the original image can be first subjected to feature correction and then subjected to high-pass filtering processing and low-pass filtering processing. The second network extracts a neck feature from the neck region in the original image, corrects the neck feature based on a target neck attribute, and obtains a corrected neck region image. The corrected neck region image is subjected to high-pass filtering processing and low-pass filtering processing to obtain an original high-pass filtered image and an original low-pass filtered image. In this way, the features of the neck region can be focused on while the overall processing effect of the image is concerned, and the effect of removing wrinkles in the neck region is further increased.

[0124] In some embodiments, the high-pass filtering of the target region of the original image to obtain an original high-pass filtered image and the low-pass filtering of the target region to obtain an original low-pass filtered image include:

[0125] detecting an attention region based on the third network;

[0126] correcting the attention region to obtain a corrected attention region;

[0127] fusing the corrected attention region to the original image to obtain an original corrected image including the corrected attention region;

[0128] high-pass filtering of a target region image of the original corrected image to obtain an original high-pass filtered image, and low-pass filtering of the target region image to obtain an original low-pass filtered image.

[0129] Similarly, the first network can also be encapsulated with the third network for processing the original image. The first network and the third network can be used for neck wrinkle removal processing of the image, and the attention region in the original image can be first corrected and then subjected to high-pass filtering processing and low-pass filtering processing. In this way, the attention region of the original image can be more accurately determined, thereby improving the effect of neck wrinkle removal.

[0130] In some embodiments, the generative adversarial network is trained based on a first training sample and a preset supervision strategy to obtain the neck wrinkle removal network (for example, the neck wrinkle removal network 330 in Figure 3 );

[0131] The first training sample includes at least one sample pair, and each sample pair includes an original training image (for example, the original training image A in Figure 3 ), a neck correction image corresponding to the original training image (for example, the neck correction image B in Figure 3The face correction image B in the image), and the original neck mask region corresponding to the original training image ( Figure 3 (not shown in the image) and the corrected neck mask area corresponding to the corrected neck image ( Figure 3 (Not shown in the image).

[0132] In some embodiments, the generative adversarial network includes a generative network (e.g., Figure 3 The generator network 310 and the discriminator network (e.g., the generator network 310) and the discriminator network (e.g., the discriminator network 310) Figure 3 The preset supervision strategy includes: (discriminative network 320 in the middle)

[0133] The generative network generates a corresponding first image (e.g., first image A' or G(x)) and a first neck mask region (e.g., G(x)_mask) of the first image based on the original training image in the first training sample.

[0134] Based on the first image, the original training image, the first neck mask region, and the corrected neck mask region, calculate the cross-entropy loss function (e.g., cross-entropy loss function) of the generator network and the discriminator network. );

[0135] Based on the cross-entropy loss function, the parameters of the first discriminant network of the discriminant network are adjusted to maximize the cross-entropy loss function (e.g., ...). ), and adjust the parameters of the first generator network of the generator network based on the cross-entropy loss function to minimize the cross-entropy loss function (e.g. ).

[0136] In some embodiments, the cross-entropy loss function includes an image cross-entropy loss function and / or a mask cross-entropy loss function; wherein,

[0137] The image cross-entropy loss function is calculated based on the first image and the original training image;

[0138] The mask cross-entropy loss function is calculated based on the first neck mask region (e.g., G(x)_mask) and the original neck mask region (e.g., x_mask).

[0139] In some embodiments, the generative adversarial network includes a generator network and a discriminator network, and the preset supervision strategy includes:

[0140] The generative network generates a corresponding first image based on the original training image in the first training sample.

[0141] Calculate a feature correction loss function (e.g., feature correction loss function L_F) based on the first image and the neck correction image in the first training sample.

[0142] adjusting second generation network parameters of the generation network and second discrimination network parameters of the discrimination network based on the feature correction loss function, so as to minimize the feature correction loss function.

[0143] In some embodiments, the feature correction loss function is calculated based on the first image and the neck corrected image in the first training sample, further comprising:

[0144] performing feature extraction on the first image to obtain first high-dimensional semantic features and first low-dimensional texture features, and performing feature extraction on the neck corrected image in the first training sample to obtain second high-dimensional semantic features and second low-dimensional texture features;

[0145] calculating a first high-dimensional feature loss function (e.g., l1(F1_A', F3)) based on the first high-dimensional semantic features and the second high-dimensional semantic features, and calculating a first low-dimensional feature loss function (e.g., l1(F2_A', F4)) based on the first low-dimensional texture features and the second low-dimensional texture features;

[0146] obtaining the feature correction loss function based on a sum of the first high-dimensional feature loss function and the second low-dimensional feature loss function.

[0147] In some embodiments, the generative adversarial network comprises a generation network and a discrimination network, and the preset supervision strategy comprises:

[0148] the generation network generates a corresponding first image and a first neck mask region of the first image based on the original training image in the first training sample;

[0149] calculating a pixel correction loss function (e.g., a sum of a pixel correction loss function L_JX and an image pixel correction loss function L_TX) based on the first image, the first neck mask region, the neck corrected image in the first training sample, and the corrected neck mask;

[0150] adjusting third generation network parameters of the generation network and third discrimination network parameters of the discrimination network based on the pixel correction loss function, so as to minimize the pixel correction loss function.

[0151] In some embodiments, the pixel correction loss function comprises a neck pixel correction loss function (e.g., a neck pixel correction loss function L_JX); wherein the neck pixel correction loss function is calculated based on the first neck mask region and the corrected neck mask region in the first training sample.

[0152] In some embodiments, the pixel correction loss function further comprises an image pixel correction loss function (e.g., image pixel correction loss function L_TX); wherein the image pixel correction loss function is calculated based on the first image and the neck correction image in the first training sample.

[0153] In some embodiments, the pixel correction loss function (e.g., pixel correction loss function L_XSJZ) comprises a neck pixel correction loss function and a corresponding first weight, and an image pixel correction loss function and a corresponding second weight, the first weight being greater than the second weight.

[0154] In some embodiments, the preset supervision strategy comprises:

[0155] determining an original attention region (e.g., original attention region L A ) of the original training image in the first training sample, and removing the original attention region in the original training image to obtain a first intermediate image (e.g., first intermediate image T A ); and determining a correction attention region (e.g., correction attention region L B ) of the neck correction image, and removing the correction attention region in the neck correction image to obtain a second intermediate image (e.g., second intermediate image T B );

[0156] obtaining an attention loss function (e.g., attention loss function L_Att) based on the first intermediate image and the second intermediate image;

[0157] adjusting fourth generation network parameters of the generation network and fourth discriminant network parameters of the discriminant network based on the attention loss function, so as to minimize the attention loss function.

[0158] In some embodiments, the preset supervision strategy comprises:

[0159] performing low-pass filtering on the original training image to obtain a corresponding original low-pass filtered image;

[0160] training the generative adversarial network on the original training image and the corresponding original low-pass filtered image.

[0161] In some embodiments, the generative adversarial network comprises a generation network and a discriminant network, and the preset supervision strategy comprises:

[0162] the generation network generates a corresponding first image and a first neck mask region of the first image based on the original training image in the first training sample;

[0163] performing high-pass filtering on the first neck mask region to obtain a first high-pass filtered image and performing low-pass filtering on the first neck mask region to obtain a first low-pass filtered image, and performing high-pass filtering on the corrected neck mask region in the first sample to obtain a second high-pass filtered image and performing low-pass filtering on the corrected neck mask region to obtain a second low-pass filtered image;

[0164] calculating a filter loss function (e.g., filter loss function L_filter) based on the first high-pass filtered image (e.g., gt(A'mask)), the first low-pass filtered image (e.g., dt(A'mask)), the second high-pass filtered image (e.g., gt(B_mask)), and the second low-pass filtered image (e.g., dt(B_mask));

[0165] adjusting parameters of the image processing model based on the filter loss function so as to minimize the filter loss function.

[0166] In some embodiments, the filter loss function includes a high-pass filter loss function and a low-pass filter loss function; wherein the high-pass filter loss function (e.g., high-pass filter loss function L_gt) is based on the first high-pass filtered image and the second high-pass filtered image, and the low-pass filter loss function (e.g., low-pass filter loss function L_dt) is based on the first low-pass filtered image and the second low-pass filtered image.

[0167] In some embodiments, the generation of the neck wrinkle removal network based on the first training sample and a preset supervision strategy includes:

[0168] training a generative adversarial network based on the first training sample and a preset supervision strategy to obtain a preliminary image processing model;

[0169] training the preliminary image processing model based on a second training sample and the preset supervision strategy to obtain the neck wrinkle removal network.

[0170] In some embodiments, the second training sample is obtained based on the preliminary image processing model, and specifically includes:

[0171] obtaining a training data set including a plurality of neck images, and inputting the neck images into a generative network in the generative adversarial network to obtain a second image (e.g., second image D);

[0172] inputting the second image into the preliminary image processing model to obtain a third image (e.g., third image E) corresponding to the second image and subjected to preliminary neck wrinkle removal processing;

[0173] The second training sample is obtained based on the second image and a corresponding third image.

[0174] In some embodiments, the preset supervision strategy comprises:

[0175] The light condition parameter of the input image of the generative adversarial network is set to simulate the light condition of the input image.

[0176] It should be noted that the method of the embodiments of the present disclosure can be executed by a single device, such as a computer or a server. The method of the embodiments can also be applied to a distributed scenario, and be completed by multiple devices cooperating with each other. In the distributed scenario, one of the multiple devices can only execute one or more steps in the method of the embodiments of the present disclosure, and the multiple devices can interact with each other to complete the method.

[0177] It should be noted that some embodiments of the present disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order and still achieve desirable results. Additionally, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0178] Based on the same technical concept, the present disclosure also provides an image processing device corresponding to the method of any of the above embodiments, referring to Figure 5 , the image processing device comprises:

[0179] An acquisition module is configured to acquire an original image to be processed;

[0180] A filtering module is configured to perform high-pass filtering on a target region of the original image to obtain an original high-pass filtered image, and perform low-pass filtering on the target region to obtain an original low-pass filtered image;

[0181] A first network is configured to process the original image to determine a target attribute of the target region, process the original high-pass filtered image and the original low-pass filtered image based on the target attribute to obtain a target high-pass filtered image and a target low-pass filtered image, obtain a target region image based on the target high-pass filtered image and the target low-pass filtered image, and fuse the target region image into the original image to obtain a target image.

[0182] For the convenience of description, the above device is described in various modules in terms of functions. Of course, the functions of the modules can be implemented in one or more software and / or hardware when implementing the present disclosure.

[0183] The apparatus of the above embodiments is used to implement the corresponding image processing method of any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not repeated here.

[0184] Based on the same technical concept, the disclosure also provides a non-transitory computer-readable storage medium, which stores computer instructions for causing the computer to execute the image processing method of any of the above embodiments.

[0185] The computer-readable medium of the present embodiment includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0186] The storage medium of the above embodiments stores computer instructions for causing the computer to execute the image processing method of any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not repeated here.

[0187] Those skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure (including claims) is limited to these examples; under the idea of the present disclosure, the above embodiments or technical features between different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of the embodiments of the present disclosure as described above. In order to be brief, they are not provided in detail.

[0188] Additionally, to simplify the description and discussion, and so as not to obscure the understanding of the embodiments of the disclosure, the well-known power / ground connections of the integrated circuits (ICs) and other components can or can not be shown in the provided figures. Furthermore, the apparatus can be shown in block diagram form in order to simplify and advance the description of such embodiments and also to highlight the fact that the details regarding how the apparatus is implemented, e.g., in terms of its working details, are highly dependent on the platform within which the embodiments of the disclosure are to be implemented (i.e., these details should be well within the understanding of one of ordinary skill in the art). In situations where detailed circuitry is set forth in order to describe the exemplary embodiments of the disclosure, it should be understood that the disclosure can be practiced with the full understanding and

[0189] Although the present disclosure has been described in connection with certain embodiments, numerous alternatives, modifications, and variations can become apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0190] The embodiments of this disclosure are intended to cover all such alternatives, modifications, and variations as come within the scope of the appended claims. Accordingly, any and all such modifications, variations, and equivalents that fall within the spirit and scope of the embodiments of the disclosure are intended to be embraced herein.

Claims

1. An image processing method, comprising: Obtain the original image to be processed; A high-pass filter is applied to the target region of the original image to obtain an original high-pass filtered image, and a low-pass filter is applied to the target region to obtain an original low-pass filtered image; The original image is processed based on a first network to adaptively determine the target attributes of the target region in the original image; the first network includes a generator network and a discriminator network. Based on the target attributes, the original high-pass filtered image and the original low-pass filtered image are processed to obtain the target high-pass filtered image and the target low-pass filtered image; A target region image is obtained based on the target high-pass filtered image and the target low-pass filtered image; The target region image is obtained by fusing it into the original image; The generator network generates a first image and a first target mask region of the first image based on the original training images in the first training samples. A first high-pass filtered image is obtained by performing high-pass filtering on the first target mask region, and a first low-pass filtered image is obtained by performing low-pass filtering on the first target mask region; a second high-pass filtered image is obtained by performing high-pass filtering on the corrected target mask region in the first training sample, and a second low-pass filtered image is obtained by performing low-pass filtering on the corrected target mask region. The generator network and the discriminator network are trained based on the first high-pass filtered image, the second high-pass filtered image, the first low-pass filtered image, and the second low-pass filtered image.

2. The method according to claim 1, wherein, The original image is processed based on the first network to determine the target attributes of the target region, including: The original image is low-pass filtered to obtain the original filtered image; The first network determines the target attribute based on the original filtered image and the original image.

3. The method according to claim 1, wherein performing high-pass filtering on the target region of the original image to obtain an original high-pass filtered image, and performing low-pass filtering on the target region to obtain an original low-pass filtered image, comprises: Based on the second network, feature extraction is performed on the target region to obtain target features; Based on the target attributes, the target features are corrected to obtain the corrected target features; The image of the target region is obtained based on the aforementioned target features; The original high-pass filtered image is obtained by performing high-pass filtering on the target region image of the original image, and the original low-pass filtered image is obtained by performing low-pass filtering on the target region image of the original image.

4. The method according to claim 1, wherein performing high-pass filtering on the target region of the original image to obtain an original high-pass filtered image, and performing low-pass filtering on the target region to obtain an original low-pass filtered image, comprises: Attention regions are detected in the original image based on a third network. The attention region is corrected to obtain the corrected attention region; The corrected attention region is fused into the original image to obtain an original corrected image including the corrected attention region; A high-pass filter is applied to the target region image of the original corrected image to obtain an original high-pass filtered image, and a low-pass filter is applied to the target region image to obtain an original low-pass filtered image.

5. The method according to claim 1, further comprising: A high-pass filtering loss function is obtained based on the first high-pass filtered image and the second high-pass filtered image, and a low-pass filtering loss function is obtained based on the first low-pass filtered image and the second low-pass filtered image. The filtering loss function is obtained by summing the high-pass filtering loss function and the low-pass filtering loss function; The first generator network parameters of the generator network and the second discriminator network parameters of the discriminator network are adjusted based on the filtering loss function to minimize the filtering loss function. The first training sample includes at least one sample pair, and each sample pair includes an original training image, a target correction image corresponding to the original training image, a first target mask region corresponding to the original training image, and a correction target mask region corresponding to the target correction image.

6. The method according to claim 5, wherein, Also includes: The target pixel correction loss function is calculated based on the first target mask region and the corrected target mask region in the first training sample; The image pixel correction loss function is calculated based on the first image and the target corrected image in the first training sample; The pixel correction loss function is obtained based on the target pixel correction loss function and the corresponding first weight, and the image pixel correction loss function and the corresponding second weight; The second generator network parameters of the generator network and the second discriminator network parameters of the discriminator network are adjusted based on the pixel correction loss function to minimize the pixel correction loss function.

7. The method according to claim 5, wherein, Also includes: Determine the original attention region of the original training image in the first training sample, and remove the original attention region from the original training image to obtain the first intermediate image; And determine the correction attention region of the target corrected image, and remove the correction attention region from the target corrected image to obtain a second intermediate image; An attention loss function is obtained based on the first intermediate image and the second intermediate image; The third generator network parameters of the generator network and the third discriminator network parameters of the discriminator network are adjusted based on the attention loss function to minimize the attention loss function.

8. An image processing apparatus, comprising: The acquisition module is used to acquire the original image to be processed; A filtering module is used to perform high-pass filtering on the target region of the original image to obtain an original high-pass filtered image, and to perform low-pass filtering on the target region to obtain an original low-pass filtered image; A first network is used to process the original image and adaptively determine the target attributes of the target region of the original image; Based on the target attributes, the original high-pass filtered image and the original low-pass filtered image are processed to obtain the target high-pass filtered image and the target low-pass filtered image; A target region image is obtained based on the target high-pass filtered image and the target low-pass filtered image; and the target region image is fused into the original image to obtain a target image; the first network includes a generator network and a discriminator network; The generator network generates a first image and a first target mask region of the first image based on the original training images in the first training samples. A first high-pass filtered image is obtained by performing high-pass filtering on the first target mask region, and a first low-pass filtered image is obtained by performing low-pass filtering on the first target mask region; a second high-pass filtered image is obtained by performing high-pass filtering on the corrected target mask region in the first training sample, and a second low-pass filtered image is obtained by performing low-pass filtering on the corrected target mask region. The generator network and the discriminator network are trained based on the first high-pass filtered image, the second high-pass filtered image, the first low-pass filtered image, and the second low-pass filtered image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as claimed in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1 to 7.

11. A computer program product comprising computer program instructions that, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 7.

Images