Image processing method and device based on style migration and storage medium

Abstract

The invention discloses an image processing method and device based on style migration and a storage medium. The method comprises the following steps: acquiring an original image uploaded by a user; obtaining a plurality of groups of image processing parameters, wherein the plurality of groups of image processing parameters are obtained by training an image data set containing a plurality of cartoon styles through a pre-trained convolutional neural network model; and performing style migration processing on the original image according to the plurality of groups of image processing parameters and an image processing algorithm to obtain a plurality of target style images. Through the image style migration method and device, the problem that the image style migration effect is poor due to the fact that a filter with the corresponding processing effect is usually added to the image to achieve image style migration in the related technology is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to an image processing method, device and storage medium based on style transfer. Background Art

[0002] In the current era of mobile Internet, mobile banking applications have become one of the main channels for users to conduct daily financial transactions. As users' demand for personalized experience continues to increase, the interface design and functional services of mobile banking are also continuously innovating to meet the diverse needs of different user groups. In this context, the personalized setting of user avatars has become one of the important means to improve user experience and enhance user stickiness. However, the traditional mobile banking avatar setting function is often limited to uploading real photos or selecting preset graphics, lacking artistry and style diversity, and unable to meet users' dual needs for image creativity and privacy protection.

[0003] The emergence of image style transfer technology has made it possible to solve the above problems. Image style transfer is a deep learning technology that can transfer the style of one image (such as color, texture, lines, etc.) to another image while maintaining the integrity of the content. This technology is widely used in the fields of artistic creation, image enhancement and personalized services. It can convert ordinary photos into various art forms such as painting style and anime style, providing users with more diverse image choices.

[0004] However, existing technologies have some challenges and shortcomings in image style transfer. Traditional style transfer methods mostly rely on specific filters or predefined style templates, which limits the diversity and flexibility of styles and cannot meet users' pursuit of personalized and innovative styles. Moreover, the filter only adds a layer to the image without modifying the pixels of the image, that is, the image itself does not really achieve style transfer.

[0005] Currently, no effective solution has been proposed to solve the problem that in the related art, a filter with corresponding processing effects is usually added to the image to achieve the style transfer of the image, which leads to poor effect of the style transfer of the image. Summary of the invention

[0006] The main purpose of the present application is to provide an image processing method, device and storage medium based on style transfer, so as to solve the problem that in the related art, a filter with corresponding processing effect is usually added to the image to realize the style transfer of the image, resulting in poor effect of the image style transfer.

[0007] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for image processing based on style transfer is provided. The method comprises: obtaining an original image uploaded by a user; obtaining multiple sets of image processing parameters, wherein the multiple sets of image processing parameters are obtained by training a pre-trained convolutional neural network model on an image data set containing multiple comic styles; performing style transfer processing on the original image according to the multiple sets of image processing parameters and the image processing algorithm to obtain multiple target style images.

[0008] Furthermore, multiple sets of image processing parameters are obtained through the following steps: optimizing the convolutional neural network model according to the image data set to obtain an optimized convolutional neural network model; extracting the feature matrix of each image from the image data set using the optimized convolutional neural network model to obtain a feature matrix set; dividing the feature matrix set into a training matrix set and a verification matrix set; training the parameter generation network according to the training matrix set to obtain a trained parameter generation network; inputting the verification matrix set into the trained parameter generation network to perform prediction processing, and outputting multiple sets of image processing parameters.

[0009] Furthermore, the image processing algorithm at least includes: a homomorphic filtering algorithm, a bilateral filtering algorithm and an edge processing algorithm, and each set of image processing parameters in the multiple sets of image processing parameters at least includes: a filter kernel, a spatial domain standard deviation, a grayscale domain standard deviation, an edge detection threshold, a custom filter radius, and a saturation index. The style migration process is performed on the original image according to the multiple sets of image processing parameters and the image processing algorithm to obtain multiple target style images, including: according to the homomorphic filtering algorithm and the filter kernel in each set of image processing parameters, the original image is enhanced in contrast to obtain a homomorphic filtered image; according to the spatial domain standard deviation in the bilateral filtering algorithm and each set of image processing parameters , the grayscale domain standard deviation in each set of image processing parameters is used to smooth the homomorphic filtered image to obtain a bilateral filtered image; according to the edge processing algorithm and the edge detection threshold in each set of image processing parameters and the custom filter radius in each set of image processing parameters, the edge processing is performed on the bilateral filtered image to obtain an edge-thickened image; the homomorphic filtered image and the edge-thickened image are merged to obtain a merged image; according to the saturation index in each set of image processing parameters, the saturation of the merged image is enhanced to obtain a target style image corresponding to each set of image processing parameters; multiple target style images are obtained based on the target style image corresponding to each set of image processing parameters.

[0010] Furthermore, edge processing is performed on the bilateral filtered image according to the edge processing algorithm, the edge detection threshold in each set of image processing parameters, and the custom filter radius in each set of image processing parameters to obtain an edge-thickened image, including: grayscale conversion processing is performed on the bilateral filtered image according to the grayscale processing function to obtain a grayscale image; edge detection is performed on the grayscale image according to the edge detection function and the edge detection threshold in each set of image processing parameters to obtain an edge detected image; the edge detection image is processed according to the edge extraction method to obtain an edge extracted image; and edge thickening processing is performed on the edge extracted image according to the edge thickening algorithm and the custom filter radius in each set of image processing parameters to obtain an edge thickened image.

[0011] Furthermore, according to the saturation index in each set of image processing parameters, the merged image is subjected to saturation enhancement processing to obtain the target style image corresponding to each set of image processing parameters, including: traversing all pixels of the merged image to obtain the intensity value of each pixel, multiplying the intensity value by the saturation index image processing parameter in each set of image processing parameters to enhance the intensity value of each pixel, and obtaining the target style image corresponding to each set of image processing parameters.

[0012] Furthermore, according to the edge thickening algorithm and the custom filter radius in each set of image processing parameters, the edge extracted image is subjected to edge thickening processing to obtain the edge thickening image, which includes: traversing all pixel points of the edge extracted image, taking each pixel point as a center pixel point, using the custom filter radius in each set of image processing parameters as a radius, defining a neighborhood range, searching for all pixel points within the neighborhood range, and if there is a pixel point with a pixel value of 1 among all the pixel points, setting the value of the center pixel point to 1 to thicken the edge of the edge extracted image to obtain the edge thickening image.

[0013] Furthermore, after obtaining multiple target style images according to the target style images corresponding to each set of image processing parameters, the method also includes: displaying the multiple target style images on the target interface; if it is detected on the target interface that the user selects a target style image from the multiple target style images, using the target style image as the user's avatar image in the target application.

[0014] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a style transfer-based image processing device is provided. The device comprises: a first acquisition unit, used to acquire the original image uploaded by the user; a second acquisition unit, which acquires multiple sets of image processing parameters, wherein the multiple sets of image processing parameters are obtained by training a pre-trained convolutional neural network model on an image data set containing multiple comic styles; a processing unit, which performs style transfer processing on the original image according to the multiple sets of image processing parameters and the image processing algorithm to obtain multiple target style images.

[0015] Furthermore, the device also includes: a model optimization unit, which is used to generate multiple sets of image processing parameters, optimize the convolutional neural network model according to the image data set, and obtain the optimized convolutional neural network model; a feature extraction unit, which uses the optimized convolutional neural network model to extract the feature matrix of each image from the image data set to obtain a feature matrix set; a division unit, which divides the feature matrix set into a training matrix set and a verification matrix set; a training unit, which trains the parameter generation network according to the training matrix set to obtain a trained parameter generation network; a prediction unit, which inputs the verification matrix set into the trained parameter generation network to perform prediction processing, and outputs multiple sets of image processing parameters.

[0016] Furthermore, the processing unit also includes: a homomorphic filtering subunit, which performs contrast enhancement processing on the original image according to the homomorphic filtering algorithm and the filter kernel in each set of image processing parameters to obtain a homomorphic filtered image; a bilateral filtering subunit, which performs smoothing processing on the homomorphic filtered image according to the bilateral filtering algorithm and the spatial domain standard deviation in each set of image processing parameters and the grayscale domain standard deviation in each set of image processing parameters to obtain a bilateral filtered image; an edge processing subunit, which performs edge processing on the bilateral filtered image according to the edge processing algorithm and the edge detection threshold in each set of image processing parameters and the custom filter radius in each set of image processing parameters to obtain an edge-thickened image; a merging subunit, which merges the homomorphic filtered image with the edge-thickened image to obtain a merged image; a saturation enhancement subunit, which performs saturation enhancement processing on the merged image according to the saturation index in each set of image processing parameters to obtain a target style image corresponding to each set of image processing parameters; and an image generation subunit, which obtains multiple target style images based on the target style image corresponding to each set of image processing parameters.

[0017] Furthermore, the edge processing subunit also includes: a grayscale conversion module; performing grayscale conversion processing on the bilateral filtered image according to the grayscale processing function to obtain a grayscale image; an edge detection module, performing edge detection on the grayscale image according to the edge detection function and the edge detection threshold in each group of image processing parameters to obtain an edge detection image; an edge extraction module, processing the edge detection image according to the edge extraction method to obtain an edge extracted image; an edge thickening module, performing edge thickening processing on the edge extracted image according to the edge thickening algorithm and the custom filter radius in each group of image processing parameters to obtain an edge thickened image.

[0018] Furthermore, the saturation enhancement subunit also includes: an acquisition module, which traverses all pixel points of the merged image to obtain the intensity value of each pixel point; an image generation module, which multiplies the intensity value with the saturation index image processing parameter in each set of image processing parameters to enhance the intensity value of each pixel point and obtain the target style image corresponding to each set of image processing parameters.

[0019] Furthermore, the edge thickening module also includes: a search range delimiting submodule, which traverses all pixel points of the edge extraction image, takes each pixel point as a center pixel point, and uses the custom filter radius in each set of image processing parameters as a radius to delimit the neighborhood range; an edge thickening submodule, which searches for all pixel points in the neighborhood range, and if there are pixel points with a pixel value of 1 among all the pixel points, the value of the center pixel point is set to 1, so as to thicken the edge of the edge extraction image and obtain an edge thickened image.

[0020] Furthermore, the device also includes: an output unit, which displays the multiple target style images on the target interface after obtaining multiple target style images corresponding to each set of image processing parameters; and an avatar setting unit, which uses the target style image as the user's avatar image in the target application if it is detected on the target interface that the user selects a target style image from the multiple target style images.

[0021] According to another aspect of the present application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein when the program is executed, a device where the computer-readable storage medium is located is controlled to execute any image processing method based on style transfer.

[0022] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the image processing methods based on style transfer.

[0023] According to another aspect of the present application, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the steps of any one of the above-mentioned image processing methods based on style transfer.

[0024] In an embodiment of the present application, an original image uploaded by a user is obtained; multiple sets of image processing parameters are obtained by training a pre-trained convolutional neural network model on an image data set containing multiple comic styles; style transfer processing is performed on the original image according to the multiple sets of image processing parameters and the image processing algorithm to obtain multiple target style images, thereby solving the problem that the style transfer of the image is usually achieved by adding filters with corresponding processing effects to the image in the related art, resulting in poor effect of image style transfer. In the present application, since multiple sets of image processing parameters are obtained by training a pre-trained convolutional neural network model on an image data set containing multiple comic styles, multiple sets of image processing parameters and the image processing algorithm are used to perform style transfer processing on the original image, and multiple target style images can be obtained at the same time, thereby improving the effect of image style transfer. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0026] Figure 1 A hardware structure block diagram of a computer terminal for implementing an image processing method based on style transfer is shown;

[0027] Figure 2 is a flowchart of an image processing method based on style transfer provided in an embodiment of the present application;

[0028] Figure 3 is a flowchart of generating image processing parameters provided according to an embodiment of the present application;

[0029] Figure 4 It is a flowchart of generating multiple target style images according to the image processing algorithm and image processing parameters provided in an embodiment of the present application;

[0030] Figure 5 It is a flow chart of generating an edge-thickened image according to an edge processing algorithm and image processing parameters provided in an embodiment of the present application;

[0031] Figure 6 is a schematic diagram of an image processing device based on style transfer provided according to an embodiment of the present application;

[0032] Figure 7 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0036] Style transfer: Use certain methods to convert an image from its original style to another style while ensuring that the image content does not change, similar to adding a filter with a specific style.

[0037] Homomorphic filtering: Homomorphic filtering is an image enhancement technique that can repair changes in illumination and contrast. Homomorphic filters can process both low-frequency and high-frequency information. By performing a logarithmic transformation on the image in the frequency domain, the product domain is converted into an additive domain, thereby adjusting brightness and contrast. Homomorphic filters are often used to deal with problems such as uneven illumination and shadows, and also have certain denoising and enhancement effects.

[0038] Bilateral filtering: Bilateral filtering is a nonlinear filtering method. It is a compromise between the spatial proximity and pixel value similarity of an image, while considering spatial information and grayscale similarity to achieve edge-preserving denoising.

[0039] It should be noted that the collected information involved in this application (including but not limited to user device information, original images, etc.) is authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant information comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions to provide users with corresponding operation entrances for users to choose to agree or refuse the results of automated decision-making; if the user chooses to refuse, the expert decision-making process will be entered.

[0040] Example 1

[0041] According to an embodiment of the present application, an embodiment of an image processing method based on style transfer is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0042] The image processing method embodiment based on style transfer provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method based on style transfer. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0043] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0044] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the image processing method based on style transfer in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned image processing method based on style transfer. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0045] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0046] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0047] Under the above operating environment, this application provides Figure 2 The image processing method based on style transfer shown. Figure 2 This is a flowchart of an image processing method based on style transfer according to Example 1 of the present application.

[0048] Step S201, obtaining the original image uploaded by the user.

[0049] Optionally, the above-mentioned original image can be a personal photo, a landscape photo or other photo uploaded or taken by the user, which is not limited in this application.

[0050] Step S202, obtaining multiple sets of image processing parameters, wherein the multiple sets of image processing parameters are obtained by training a pre-trained convolutional neural network model on an image data set containing multiple comic styles.

[0051] Optionally, the comics involved in the above-mentioned image data set may include fantasy comics, suspense comics, girls comics, campus comics, urban comics, workplace comics, heroism comics, etc., which are not limited in this application. Multiple groups of image processing parameters output by the pre-trained convolutional neural network model, each group of image processing parameters corresponds to a comic style, for example, the fantasy comic style image processing parameters are obtained by training the image data set of fantasy comic style through the pre-trained convolutional neural network model, and the suspense comic style image processing parameters are obtained by training the image data set of suspense comic style through the pre-trained convolutional neural network model.

[0052] Step S203 , performing style transfer processing on the original image according to multiple sets of image processing parameters and image processing algorithms to obtain multiple target style images.

[0053] Optionally, for each set of image processing parameters, the original image is processed using an image processing algorithm to generate a stylized target image. This process is repeated multiple times until each comic style has been tried, and multiple style-transferred images are generated for the user to choose from. For example, the original image A is processed using the image processing parameters corresponding to the fantasy comic style, and a fantasy comic style image A1 is generated as the target style image. The original image A is processed using the image processing parameters corresponding to the suspense comic style, and a suspense comic style image A2 is generated as the target style image. That is, multiple target style images are generated for the user to choose from, thereby improving the effect of image style transfer.

[0054] In summary, in the embodiments of the present application, by obtaining the original image uploaded by the user; obtaining multiple sets of image processing parameters, wherein the multiple sets of image processing parameters are obtained by training and processing a pre-trained convolutional neural network model on an image data set containing multiple comic styles; performing style transfer processing on the original image according to the multiple sets of image processing parameters and the image processing algorithm, a plurality of target style images are obtained, which solves the problem that the style transfer of the image is usually achieved by adding filters with corresponding processing effects to the image in the related art, resulting in poor effect of image style transfer. In the present application, since the multiple sets of image processing parameters are obtained by training and processing a pre-trained convolutional neural network model on an image data set containing multiple comic styles, the multiple sets of image processing parameters and the image processing algorithm are used to perform style transfer processing on the original image, and multiple target style images can be obtained at the same time, thereby improving the effect of image style transfer.

[0055] In an optional embodiment, if Figure 3 As shown, using the pre-trained convolutional neural network model to generate multiple sets of image processing parameters includes:

[0056] S301, optimizing the convolutional neural network model according to the image data set to obtain an optimized convolutional neural network model.

[0057] Optionally, the convolutional neural network model is retrained using an image dataset. During the training process, the model can be made better adapted to the new comic style classification task by adding adaptive layers, data enhancement, and learning rate parameter adjustment, thereby obtaining an optimized convolutional neural network model that performs better in comic style recognition.

[0058] For example, adding some adaptive layers to adapt to the characteristics of the image dataset, such as a global average pooling layer, a new fully connected layer, or a Dropout layer to reduce the risk of overfitting.

[0059] S302, using the optimized convolutional neural network model to extract the feature matrix of each image from the image data set to obtain a feature matrix set.

[0060] Optionally, use the optimized convolutional neural network model to extract features from each image in the image dataset, generate a series of image feature matrices, and obtain a feature matrix set containing a large number of comic style image features. For example, for each image, use the optimized VGG16 model to extract features. Assuming that features are extracted at the 13th layer, a 4096-dimensional feature vector will be obtained, and these vectors constitute a feature matrix set.

[0061] S303, dividing the feature matrix set into a training matrix set and a verification matrix set.

[0062] Optionally, the training matrix set is used for training the parameter generation network, and the verification matrix set is used as a verification matrix set to evaluate the generalization ability of the network.

[0063] For example, the feature matrix set may be divided into a verification matrix set and a training matrix set in a ratio of 8:2, and the specific division ratio is not limited in this application.

[0064] S304, training the parameter generation network according to the training matrix set to obtain a trained parameter generation network.

[0065] Optionally, the parameter generation network is a neural network structure used to generate model parameters or control signals in the field of deep learning. It is usually used as part of a more complex model to dynamically generate parameters of other network layers, or to generate transformation parameters for data such as images, sounds, and text to adapt to different inputs or achieve specific output effects. When constructing a parameter generation network, the network architecture can be a fully connected network or other sequence model. The network input is the above-mentioned training matrix set, and the output is the image processing parameters required for style migration. During the training process, the model can be trained by loss functions and optimizers. Commonly used optimizers include momentum optimizer, stochastic gradient descent, stochastic gradient descent with momentum, etc., which are not limited in this application.

[0066] S305, inputting the verification matrix set into the trained parameter generation network to perform prediction processing and output multiple sets of image processing parameters.

[0067] Optionally, a trained fully connected layer parameter generation network, the input layer is a set of verification matrices, and the output layer is a set of vectors containing the parameters required for the style transfer image processing algorithm. Each set of vectors includes the filter kernel parameters required for homomorphic filtering, the spatial domain standard deviation and grayscale domain standard deviation required for bilateral filtering, the edge detection threshold required for edge detection, and the saturation index required for color enhancement.

[0068] Through the above technical solution, it is possible to generate multiple sets of image processing parameters for stylizing images into comics based on artificial intelligence technology, thereby improving the accuracy and efficiency of style migration.

[0069] In an optional embodiment, if Figure 4 As shown, the original image is style-transferred according to multiple sets of image processing parameters and image processing algorithms to obtain multiple target style images including:

[0070] S401, performing contrast enhancement processing on the original image according to the homomorphic filtering algorithm and the filter kernel in each set of image processing parameters to obtain a homomorphic filtered image.

[0071] Optionally, the original image is first converted into the frequency domain, and the frequency domain of the image is filtered. The filtering process uses the filter kernel in each set of image processing parameters to identify and adjust the brightness and contrast in the above original image, and then converted back to the spatial domain to obtain a homomorphic filtered image with enhanced contrast.

[0072] Different comic styles correspond to different filter kernel values. For example, when using the filter kernel corresponding to the fantasy comic style to perform homomorphic filtering on the original image, a larger filter kernel with a more complex shape may be required. For example, using a filter kernel with a lower center weight and a higher edge weight to simulate a dreamy light and shadow effect can simulate a soft lighting and increase the layering of the image.

[0073] S402, smoothing the homomorphic filtered image according to the bilateral filtering algorithm and the spatial domain standard deviation in each set of image processing parameters and the gray domain standard deviation in each set of image processing parameters to obtain a bilateral filtered image.

[0074] Optionally, a bilateral filtering algorithm can be used to process an image based on the spatial proximity of image pixels and the similarity of grayscale values, that is, the smoothness of the image is adjusted according to the standard deviation in the spatial domain and the standard deviation in the grayscale domain, thereby removing noise while keeping edges clear, thereby improving the denoising effect of bilateral filtering and filtering the image bilaterally.

[0075] For example, when performing bilateral filtering on the homomorphic filtered image obtained in step S401 using the spatial domain standard deviation and grayscale domain standard deviation corresponding to the fantasy comic style, a larger spatial domain standard deviation may be required to ensure consistency and smoothness within a larger range, and a smaller grayscale domain standard deviation may be required to maintain details and color contrast.

[0076] S403, performing edge processing on the bilateral filtered image according to the edge processing algorithm, the edge detection threshold in each set of image processing parameters, and the custom filter radius in each set of image processing parameters to obtain an edge-thickened image.

[0077] Optionally, during edge processing, edge details of the image can be detected according to an edge detection threshold, and the edge can be thickened according to a custom filter radius, which can enhance the comic line effect of the image.

[0078] For example, to stylize the bilateral filtered image obtained in step S403 to a fantasy comic style, a lower edge detection threshold may be required to detect more edges and details, making the lines richer and suitable for the exaggerated features of the fantasy style, and a larger custom filter radius may be required to allow the lines to be thickened.

[0079] S404, merging the homomorphic filtered image and the edge-thickened image to obtain a merged image.

[0080] Optionally, the layer blending function in the image processing software can be used to use the homomorphic filtered image as the base image and the edge-thickened image as the top image, and a merged image can be obtained by an appropriate method to ensure that the edge lines are clear and blend naturally with the image content. Among them, the appropriate method can be superposition or mask blending, which is not limited in this application.

[0081] S405 , performing saturation enhancement processing on the merged image according to the saturation index in each set of image processing parameters to obtain a target style image corresponding to each set of image processing parameters.

[0082] Optionally, the color intensity of each pixel is adjusted according to the intelligently generated saturation index to achieve the effect of enhancing the color saturation of the image and generate a target style image.

[0083] Different comic styles have different color saturation. For example, if the merged image obtained in step S404 is further stylized into a fantasy comic style, a higher saturation index may be required to enhance color saturation and vividness. A suspense comic style may tend to be a simple black-white-gray style, corresponding to a lower saturation index to increase the sense of suspense.

[0084] S406 , obtaining a plurality of target style images according to the target style image corresponding to each set of image processing parameters.

[0085] Optionally, all target style images generated according to each set of parameters are collected to form a set of multiple target style images, and all images in the set are displayed to the user so that the user can select the most satisfactory comic style avatar.

[0086] The above technical solution significantly improves the technical effect of image style transfer, not only enhancing the contrast, smoothness and color vividness of the image, but also making the image more in line with the visual characteristics of comics through intelligent edge processing technology, providing users with a personalized, beautiful and privacy-protected comic avatar generation solution. At the same time, the selection of multiple styles increases the satisfaction of users' personalized needs and improves the overall user experience of mobile banking applications.

[0087] In an optional embodiment, if Figure 5 As shown, according to the edge processing algorithm and the edge detection threshold in each set of image processing parameters, and the custom filter radius in each set of image processing parameters, edge processing is performed on the bilateral filter image to obtain an edge-thickened image including:

[0088] S501, performing grayscale conversion processing on the bilateral filtered image according to a grayscale processing function to obtain a grayscale image.

[0089] Optionally, the bilaterally filtered image is processed using a grayscale processing function to convert the RGB color value of each pixel into a grayscale value to generate a grayscale image. The grayscale processing function may be a brightness average method, a maximum value method, a minimum value method, an average value method, a weighted average value method, a Luma method, a perceptual weighted method, a custom grayscale method, etc., which are not limited in this application.

[0090] S502, performing edge detection on the grayscale image according to the edge detection function and the edge detection threshold in each set of image processing parameters to obtain an edge detected image.

[0091] Optionally, the grayscale image is input into an edge detection algorithm, and the edges in the image are detected and marked according to the edge detection threshold in each set of image processing parameters to generate an edge detection image. The edge detection function can be based on different operators, such as Canny operator, Sobel operator, Prewitt operator, Laplacian operator, Roberts crossover operator, LOG operator, etc., which are not limited in this application.

[0092] S503: Process the edge detection image according to an edge extraction method to obtain an edge extracted image.

[0093] Optionally, an edge extraction method is used to process the edge detection image, remove non-edge noise, and refine edge lines to ultimately obtain a clear edge extraction image.

[0094] S504, performing edge thickening processing on the edge extracted image according to the edge thickening algorithm and the user-defined filter radius in each set of image processing parameters to obtain an edge thickened image.

[0095] Optionally, in image processing, edge thickening algorithms can be used to enhance or expand detected edges in an image, making them more visually obvious or numerically wider. Thickened edges can make images more expressive and visually impactful. Custom filter radius usually refers to the size of the neighborhood considered by the filter when processing an image, which directly affects the filtering effect. In comic style migration, different comic styles correspond to different custom filter radii. For example, suspense comic styles usually emphasize shadows, dark lights, and tense atmospheres. Lines may be thinner and more focused on the depiction of details and textures to enhance mystery and depth. This style corresponds to a smaller custom filter radius to maintain the fineness and clarity of the lines.

[0096] Through the above technical solution, the edge positioning accuracy is improved, the edge missed detection rate is effectively reduced, and the technical effect of making the image comic stylization more significant is achieved.

[0097] In an optional embodiment, saturation enhancement processing is performed on the merged image according to the saturation index in each set of image processing parameters to obtain a target style image corresponding to each set of image processing parameters, including: traversing all pixels of the merged image to obtain an intensity value of each pixel, multiplying the intensity value with the saturation index image processing parameter in each set of image processing parameters to enhance the intensity value of each pixel, and obtaining the target style image corresponding to each set of image processing parameters.

[0098] Optionally, traverse each pixel in the merged image, read its intensity value, and then multiply the intensity value by the saturation index according to the saturation index in each set of image processing parameters to enhance the color saturation of the pixel. This process will be performed for each set of parameters separately to generate multiple target style images of different styles.

[0099] For example, when the merged image is stylized as a fantasy comic, the corresponding saturation index is 2, and the intensity value of a certain pixel point of the merged image is 115. The multiplication of the two results in a new pixel point value of 230.

[0100] Through the above technical solution, on the basis of enhancing contrast and optimizing edge lines, the color saturation of the image is enhanced, making the image color more vivid and more in line with the visual characteristics of the comic style.

[0101] In an optional embodiment, according to the edge thickening algorithm and the custom filter radius in each set of image processing parameters, the edge extracted image is subjected to edge thickening processing to obtain the edge thickened image, which includes: traversing all pixel points of the edge extracted image, taking each pixel point as the center pixel point, using the custom filter radius in each set of image processing parameters as the radius, defining a neighborhood range, searching for all pixel points within the neighborhood range, and if there is a pixel point with a pixel value of 1 among all the pixel points, setting the value of the center pixel point to 1 to thicken the edge of the edge extracted image to obtain the edge thickened image.

[0102] Optionally, traverse all pixels in the edge extraction image. For each pixel, treat it as the center pixel, and define a circular neighborhood with the custom filter radius in each set of image processing parameters as the radius. For each defined neighborhood range, check the pixels therein. If it is found that there is at least one pixel with a pixel value of 1, it means that there is an edge in the neighborhood. At this time, set the value of the current center pixel to 1, that is, mark it as an edge point, thereby visually thickening the edge. This process is performed for each pixel in the edge extraction image, and finally obtains an edge-thickened image with thickened lines.

[0103] For example, if the pixel value of a center pixel is 0 and the custom filter radius is 5, and there is a pixel with a pixel value of 1 in this neighborhood, then the pixel value of the center pixel is set to 1.

[0104] Through the above technical solution, the edge lines in the image can be effectively identified and thickened, ensuring the naturalness and artistic sense of the edge thickening, avoiding the situation where the edges are too harsh or unnatural, and making the image after style transfer closer to the visual style of the comics.

[0105] In an optional embodiment, after obtaining multiple target style images based on the target style images corresponding to each set of image processing parameters, the method also includes: displaying the multiple target style images on a target interface; if it is detected on the target interface that the user selects a target style image from the multiple target style images, using the target style image as the user's avatar image in the target application.

[0106] Optionally, in the user interface design of the mobile banking application, a display area (corresponding to the above-mentioned target interface) can be created to present the multiple target style images generated above in this area. The image display can be in the form of thumbnails, and the user can view different style options by scrolling or touching. An event listener is embedded in the target interface, such as a click or touch listener. When it is detected that the user has selected a target style image, the user's selection is recorded. Then, through the user configuration or database update function of the application, the target style image selected by the user is set as the user's avatar image in the mobile banking application.

[0107] Through the above technical solutions, multiple comic-style avatar options increase the diversity of personalized choices, making mobile banking applications more in line with user needs and improving user satisfaction and security.

[0108] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0109] Example 2

[0110] The embodiment of the present application also provides an image processing device based on style transfer. It should be noted that the image processing device based on style transfer in the embodiment of the present application can be used to execute the image processing method based on style transfer provided in the embodiment of the present application. The image processing device based on style transfer provided in the embodiment of the present application is introduced below.

[0111] According to an embodiment of the present application, a device for implementing the above-mentioned image processing method based on style transfer is also provided, such as Figure 6 As shown, the device includes: a first acquisition unit 601, a second acquisition unit 602, and a processing unit 603.

[0112] Specifically, the first acquisition unit 601 is used to acquire the original image uploaded by the user; the second acquisition unit 602 is used to acquire multiple sets of image processing parameters, wherein the multiple sets of image processing parameters are obtained by training a pre-trained convolutional neural network model on an image data set containing multiple comic styles; the processing unit 603 is used to perform style transfer processing on the original image according to the multiple sets of image processing parameters and the image processing algorithm to obtain multiple target style images.

[0113] The image processing device based on style transfer provided in the embodiment of the present application obtains the original image uploaded by the user through the first acquisition unit 601; the second acquisition unit 602 obtains multiple groups of image processing parameters, wherein the multiple groups of image processing parameters are obtained by training and processing an image data set containing multiple comic styles through a pre-trained convolutional neural network model; the processing unit 603 performs style transfer processing on the original image according to the multiple groups of image processing parameters and the image processing algorithm to obtain multiple target style images, which solves the problem that the style transfer of the image is usually achieved by adding filters with corresponding processing effects to the image in the related art, resulting in poor effect of image style transfer, thereby achieving the effect of diversified image migration style.

[0114] Optionally, in the image processing device based on style transfer provided in the embodiment of the present application, the second acquisition unit 602 also includes: a model optimization unit, which is used to generate multiple sets of image processing parameters, optimize the convolutional neural network model according to the image data set, and obtain the optimized convolutional neural network model; a feature extraction unit, which uses the optimized convolutional neural network model to extract the feature matrix of each image from the image data set to obtain a feature matrix set; a division unit, which divides the feature matrix set into a training matrix set and a verification matrix set; a training unit, which trains the parameter generation network according to the training matrix set to obtain a trained parameter generation network; a prediction unit, which inputs the verification matrix set into the trained parameter generation network to perform prediction processing, and outputs multiple sets of image processing parameters.

[0115] Optionally, in the image processing device based on style transfer provided in the embodiment of the present application, the processing unit 603 also includes: a homomorphic filtering subunit, which performs contrast enhancement processing on the original image according to the homomorphic filtering algorithm and the filter kernel in each group of image processing parameters to obtain a homomorphic filtered image; a bilateral filtering subunit, which performs smoothing processing on the homomorphic filtered image according to the bilateral filtering algorithm and the spatial domain standard deviation in each group of image processing parameters and the grayscale domain standard deviation in each group of image processing parameters to obtain a bilateral filtered image; an edge processing subunit, which performs edge processing on the bilateral filtered image according to the edge processing algorithm and the edge detection threshold in each group of image processing parameters and the custom filter radius in each group of image processing parameters to obtain an edge-thickened image; a merging subunit, which merges the homomorphic filtered image with the edge-thickened image to obtain a merged image; a saturation enhancement subunit, which performs saturation enhancement processing on the merged image according to the saturation index in each group of image processing parameters to obtain a target style image corresponding to each group of image processing parameters; and an image generation subunit, which obtains multiple target style images based on the target style image corresponding to each group of image processing parameters.

[0116] Optionally, in the image processing device based on style transfer provided in the embodiment of the present application, the edge processing subunit also includes: a grayscale conversion module; performing grayscale conversion processing on the bilateral filtered image according to the grayscale processing function to obtain a grayscale image; an edge detection module, performing edge detection on the grayscale image according to the edge detection function and the edge detection threshold in each set of image processing parameters to obtain an edge detection image; an edge extraction module, processing the edge detection image according to the edge extraction method to obtain an edge extracted image; an edge thickening module, performing edge thickening processing on the edge extracted image according to the edge thickening algorithm and the custom filter radius in each set of image processing parameters to obtain an edge thickened image.

[0117] Optionally, in the image processing device based on style transfer provided in an embodiment of the present application, the saturation enhancement subunit also includes: an acquisition module, which traverses all pixels of the merged image to obtain the intensity value of each pixel; an image generation module, which multiplies the intensity value with the saturation index image processing parameter in each set of image processing parameters to enhance the intensity value of each pixel and obtain a target style image corresponding to each set of image processing parameters.

[0118] Optionally, in the image processing device based on style transfer provided in the embodiment of the present application, the edge thickening module also includes: a search range demarcation submodule, which traverses all pixel points of the edge extraction image, takes each pixel point as the center pixel point, and uses the custom filter radius in each set of image processing parameters as the radius to demarcate the neighborhood range; an edge thickening submodule, which searches for all pixel points in the neighborhood range, and if there is a pixel point with a pixel value of 1 among all the pixel points, the value of the center pixel point is set to 1 to thicken the edge of the edge extraction image to obtain an edge thickened image.

[0119] Optionally, in the image processing device based on style transfer provided in the embodiment of the present application, the device also includes: an output unit, which displays the multiple target style images on the target interface after obtaining multiple target style images corresponding to each set of image processing parameters; and an avatar setting unit, which uses the target style image as the avatar image of the user in the target application if it is detected on the target interface that the user selects a target style image from the multiple target style images.

[0120] It should be noted that the first acquisition unit 601, the second acquisition unit 602, and the processing unit 603 described above correspond to steps 201 to S203 in Embodiment 1, and the three units and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in Embodiment 1 described above. It should be noted that the above modules or units may be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n), and the above modules may also be part of a device and may be run in the computer terminal 10 provided in Embodiment 1.

[0121] Example 3

[0122] The embodiment of the present application may provide a computer terminal, which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal or an electronic device.

[0123] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0124] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the image processing method based on style transfer: obtaining the original image uploaded by the user; obtaining multiple sets of image processing parameters, wherein the multiple sets of image processing parameters are obtained by training an image data set containing multiple comic styles through a pre-trained convolutional neural network model; performing style transfer processing on the original image according to the multiple sets of image processing parameters and the image processing algorithm to obtain multiple target style images.

[0125] Optionally, the computer terminal can execute the program code of the following steps in the image processing method based on style transfer: multiple sets of image processing parameters are obtained by the following steps: optimizing the convolutional neural network model according to the image data set to obtain the optimized convolutional neural network model; extracting the feature matrix of each image from the image data set using the optimized convolutional neural network model to obtain a feature matrix set; dividing the feature matrix set into a training matrix set and a verification matrix set; training the parameter generation network according to the training matrix set to obtain a trained parameter generation network; inputting the verification matrix set into the trained parameter generation network to perform prediction processing, and outputting multiple sets of image processing parameters.

[0126] Optionally, the computer terminal can execute the following program codes in the image processing method based on style transfer: the image processing algorithm includes at least: a homomorphic filtering algorithm, a bilateral filtering algorithm and an edge processing algorithm, each group of image processing parameters in the multiple groups of image processing parameters includes at least: a filter kernel, a spatial domain standard deviation, a gray domain standard deviation, an edge detection threshold, a custom filter radius, and a saturation index; style transfer processing is performed on the original image according to the multiple groups of image processing parameters and the image processing algorithm to obtain multiple target style images, including: according to the homomorphic filtering algorithm and the filter kernel in each group of image processing parameters, contrast enhancement processing is performed on the original image to obtain a homomorphic filtered image; according to the bilateral filtering algorithm The homomorphic filtered image is smoothed with the spatial domain standard deviation in each set of image processing parameters and the grayscale domain standard deviation in each set of image processing parameters to obtain a bilateral filtered image; the edge processing is performed on the bilateral filtered image according to the edge processing algorithm and the edge detection threshold in each set of image processing parameters and the custom filter radius in each set of image processing parameters to obtain an edge-thickened image; the homomorphic filtered image is merged with the edge-thickened image to obtain a merged image; the merged image is saturated enhanced according to the saturation index in each set of image processing parameters to obtain a target style image corresponding to each set of image processing parameters; multiple target style images are obtained based on the target style image corresponding to each set of image processing parameters.

[0127] Optionally, the computer terminal may execute the program code of the following steps in the image processing method based on style transfer: performing edge processing on the bilateral filtered image according to the edge processing algorithm and the edge detection threshold in each set of image processing parameters and the custom filter radius in each set of image processing parameters to obtain an edge-thickened image, including: performing grayscale conversion processing on the bilateral filtered image according to the grayscale processing function to obtain a grayscale image; performing edge detection on the grayscale image according to the edge detection function and the edge detection threshold in each set of image processing parameters to obtain an edge-detected image; processing the edge detected image according to the edge extraction method to obtain an edge-extracted image; performing edge thickening processing on the edge extracted image according to the edge thickening algorithm and the custom filter radius in each set of image processing parameters to obtain an edge-thickened image.

[0128] Optionally, the computer terminal may execute the program code of the following steps in the image processing method based on style transfer: performing saturation enhancement processing on the merged image according to the saturation index in each set of image processing parameters to obtain a target style image corresponding to each set of image processing parameters, including: traversing all pixels of the merged image to obtain an intensity value of each pixel, multiplying the intensity value by the saturation index image processing parameter in each set of image processing parameters to enhance the intensity value of each pixel, and obtaining a target style image corresponding to each set of image processing parameters.

[0129] Optionally, the computer terminal may execute the program code of the following steps in the image processing method based on style transfer: performing edge thickening processing on the edge extracted image according to the edge thickening algorithm and the custom filter radius in each set of image processing parameters to obtain the edge thickened image, including: traversing all pixel points of the edge extracted image, taking each pixel point as the center pixel point of a circle, using the custom filter radius in each set of image processing parameters as the radius, defining a neighborhood range, searching for all pixel points within the neighborhood range, and if there is a pixel point with a pixel value of 1 among all the pixel points, setting the value of the center pixel point to 1 to thicken the edge of the edge extracted image to obtain an edge thickened image.

[0130] Optionally, the computer terminal may execute the program code of the following steps in the image processing method based on style transfer: after obtaining multiple target style images according to the target style image corresponding to each set of image processing parameters, the method further comprises: displaying the multiple target style images on a target interface; if it is detected on the target interface that a user selects a target style image from the multiple target style images, the target style image is used as the user's avatar image in the target application.

[0131] Optionally, Figure 7 is a structural block diagram of an electronic device according to an embodiment of the present application. Figure 7As shown, the electronic device may include: one or more ( Figure 7 (only one is shown) processor 702, memory 704, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0132] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image processing method and device based on style transfer in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image processing method based on style transfer. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0133] The processor can call the information and application programs stored in the memory through the transmission device to execute the above steps in the above image processing method based on style transfer.

[0134] By using the embodiment of the present application, an image processing solution based on style transfer is provided. In the embodiment of the present application, the original image uploaded by the user is obtained; multiple sets of image processing parameters are obtained by training a pre-trained convolutional neural network model on an image data set containing multiple comic styles; the original image is style-transferred according to the multiple sets of image processing parameters and the image processing algorithm to obtain multiple target style images, which solves the problem that the style transfer of the image is usually achieved by adding filters with corresponding processing effects to the image in the related art, resulting in poor effect of image style transfer. In the present application, since the multiple sets of image processing parameters are obtained by training a pre-trained convolutional neural network model on an image data set containing multiple comic styles, the multiple sets of image processing parameters and the image processing algorithm are used to perform style transfer on the original image, and multiple target style images can be obtained at the same time, thereby improving the effect of image style transfer.

[0135] It can be understood by those skilled in the art that Figure 7 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Figure 7 The structure of the electronic device is not limited. Figure 7 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 7 Different configurations are shown.

[0136] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0137] Example 4

[0138] The embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the image processing method based on style transfer provided in the first embodiment.

[0139] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0140] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps:

[0141] Obtain the original image uploaded by the user; obtain multiple sets of image processing parameters, where the multiple sets of image processing parameters are obtained by training a pre-trained convolutional neural network model on an image data set containing multiple comic styles; perform style transfer processing on the original image according to the multiple sets of image processing parameters and image processing algorithms to obtain multiple target style images.

[0142] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: multiple sets of image processing parameters are obtained by the following steps: optimizing the convolutional neural network model according to the image data set to obtain an optimized convolutional neural network model; extracting the feature matrix of each image from the image data set using the optimized convolutional neural network model to obtain a feature matrix set; dividing the feature matrix set into a training matrix set and a verification matrix set; training the parameter generation network according to the training matrix set to obtain a trained parameter generation network; inputting the verification matrix set into the trained parameter generation network to perform prediction processing, and outputting multiple sets of image processing parameters.

[0143] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: the image processing algorithm includes at least: a homomorphic filtering algorithm, a bilateral filtering algorithm and an edge processing algorithm, each group of image processing parameters in the multiple groups of image processing parameters includes at least: a filter kernel, a spatial domain standard deviation, a grayscale domain standard deviation, an edge detection threshold, a custom filter radius, and a saturation index; style migration is performed on the original image according to the multiple groups of image processing parameters and the image processing algorithm to obtain multiple target style images, including: according to the homomorphic filtering algorithm and the filter kernel in each group of image processing parameters, contrast enhancement is performed on the original image to obtain a homomorphic filtered image; according to the bilateral filtering algorithm and each group of image processing parameters, contrast enhancement is performed on the original image to obtain a homomorphic filtered image. The homomorphic filtered image is smoothed according to the spatial domain standard deviation in the group image processing parameters and the grayscale domain standard deviation in each group of image processing parameters to obtain a bilateral filtered image; the edge processing is performed on the bilateral filtered image according to the edge processing algorithm and the edge detection threshold in each group of image processing parameters and the custom filter radius in each group of image processing parameters to obtain an edge-thickened image; the homomorphic filtered image and the edge-thickened image are merged to obtain a merged image; according to the saturation index in each group of image processing parameters, the merged image is saturated enhanced to obtain a target style image corresponding to each group of image processing parameters; multiple target style images are obtained based on the target style image corresponding to each group of image processing parameters.

[0144] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: performing edge processing on the bilateral filtered image according to an edge processing algorithm and an edge detection threshold in each set of image processing parameters and a custom filter radius in each set of image processing parameters to obtain an edge-thickened image, including: performing grayscale conversion processing on the bilateral filtered image according to a grayscale processing function to obtain a grayscale image; performing edge detection on the grayscale image according to an edge detection function and an edge detection threshold in each set of image processing parameters to obtain an edge detected image; processing the edge detected image according to an edge extraction method to obtain an edge extracted image; performing edge thickening processing on the edge extracted image according to an edge thickening algorithm and a custom filter radius in each set of image processing parameters to obtain an edge thickened image.

[0145] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: performing saturation enhancement processing on the merged image according to the saturation index in each set of image processing parameters to obtain a target style image corresponding to each set of image processing parameters, including: traversing all pixels of the merged image to obtain an intensity value of each pixel, multiplying the intensity value with the saturation index image processing parameter in each set of image processing parameters to enhance the intensity value of each pixel, and obtaining a target style image corresponding to each set of image processing parameters.

[0146] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: performing edge thickening processing on the edge extracted image according to the edge thickening algorithm and the custom filter radius in each set of image processing parameters to obtain the edge thickened image, including: traversing all pixel points of the edge extracted image, taking each pixel point as the center pixel point, using the custom filter radius in each set of image processing parameters as the radius, defining a neighborhood range, searching for all pixel points within the neighborhood range, and if there is a pixel point with a pixel value of 1 among all the pixel points, setting the value of the center pixel point to 1 to thicken the edge of the edge extracted image to obtain the edge thickened image.

[0147] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: after obtaining multiple target style images based on the target style image corresponding to each set of image processing parameters, the method also includes: displaying the multiple target style images on the target interface; if it is detected on the target interface that the user selects a target style image from the multiple target style images, using the target style image as the user's avatar image in the target application.

[0148] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing the steps of the image processing method based on style transfer.

[0149] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0150] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0151] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0152] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0153] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0154] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0155] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image processing method based on style transfer, characterized in that: include: Get the original image uploaded by the user; Acquire multiple sets of image processing parameters, wherein the multiple sets of image processing parameters are obtained by training a pre-trained convolutional neural network model on an image data set containing multiple comic styles; The original image is subjected to style transfer processing according to the multiple sets of image processing parameters and image processing algorithms to obtain multiple target style images.

2. The method according to claim 1, characterized in that: The multiple sets of image processing parameters are obtained by the following steps: Optimizing the convolutional neural network model according to the image data set to obtain an optimized convolutional neural network model; Extracting a feature matrix of each image from the image data set using the optimized convolutional neural network model to obtain a feature matrix set; Dividing the feature matrix set into a training matrix set and a verification matrix set; Training the parameter generation network according to the training matrix set to obtain a trained parameter generation network; The verification matrix set is input into the trained parameter generation network to perform prediction processing and output multiple sets of image processing parameters.

3. The method according to claim 1, characterized in that The image processing algorithm at least includes: a homomorphic filtering algorithm, a bilateral filtering algorithm and an edge processing algorithm. Each set of image processing parameters in the multiple sets of image processing parameters at least includes: a filter kernel, a spatial domain standard deviation, a grayscale domain standard deviation, an edge detection threshold, a custom filter radius, and a saturation index. The original image is subjected to style migration processing according to the multiple sets of image processing parameters and the image processing algorithm to obtain multiple target style images including: According to the homomorphic filtering algorithm and the filter kernel in each set of image processing parameters, the original image is subjected to contrast enhancement processing to obtain a homomorphic filtered image; Smoothing the homomorphic filtered image according to the bilateral filtering algorithm and the spatial domain standard deviation in each set of image processing parameters and the grayscale domain standard deviation in each set of image processing parameters to obtain a bilateral filtered image; According to the edge processing algorithm, the edge detection threshold in each set of image processing parameters, and the custom filter radius in each set of image processing parameters, edge processing is performed on the bilateral filtered image to obtain an edge-thickened image; Merging the homomorphic filtered image with the edge-thickened image to obtain a merged image; According to the saturation index in each set of image processing parameters, the merged image is subjected to saturation enhancement processing to obtain a target style image corresponding to each set of image processing parameters; A plurality of target style images are obtained according to the target style image corresponding to each set of image processing parameters.

4. The method according to claim 3, characterized in that According to the edge processing algorithm, the edge detection threshold in each set of image processing parameters, and the custom filter radius in each set of image processing parameters, edge processing is performed on the bilateral filtered image to obtain an edge-thickened image, including: Performing grayscale conversion processing on the bilateral filtered image according to a grayscale processing function to obtain a grayscale image; Performing edge detection on the grayscale image according to an edge detection function and an edge detection threshold in each set of image processing parameters to obtain an edge detection image; Processing the edge detection image according to an edge extraction method to obtain an edge extraction image; According to the edge thickening algorithm and the user-defined filter radius in each group of image processing parameters, edge thickening processing is performed on the edge extracted image to obtain an edge thickening image.

5. The method according to claim 3, characterized in that: According to the saturation index in each set of image processing parameters, the merged image is subjected to saturation enhancement processing to obtain the target style image corresponding to each set of image processing parameters, including: All pixels of the merged image are traversed to obtain an intensity value of each pixel, and the intensity value is multiplied by a saturation index image processing parameter in each set of image processing parameters to enhance the intensity value of each pixel, thereby obtaining a target style image corresponding to each set of image processing parameters.

6. The method according to claim 4, characterized in that According to the edge thickening algorithm and the custom filter radius in each set of image processing parameters, the edge extracted image is subjected to edge thickening processing to obtain the edge thickening image, which includes: All pixel points of the edge extraction image are traversed and processed, each pixel point is used as a center pixel point, and a neighborhood range is defined with a custom filter radius in each set of image processing parameters as a radius, and all pixel points in the neighborhood range are searched. If there is a pixel point with a pixel value of 1 among all the pixel points, the value of the center pixel point is set to 1 to thicken the edge of the edge extraction image to obtain an edge-thickened image.

7. The method according to claim 3, characterized in that After obtaining a plurality of target style images according to the target style images corresponding to each set of image processing parameters, the method further includes: Displaying the multiple target style images on a target interface; If it is detected on the target interface that the user selects a target style image from a plurality of target style images, the target style image is used as an avatar image of the user in the target application.

8. An image processing device based on style transfer, characterized in that: include: A first acquisition unit, used to acquire the original image uploaded by the user; A second acquisition unit is configured to acquire multiple sets of image processing parameters, wherein the multiple sets of image processing parameters are obtained by training a pre-trained convolutional neural network model on an image data set containing multiple comic styles; The processing unit performs style transfer processing on the original image according to multiple sets of image processing parameters and image processing algorithms to obtain multiple target style images.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the computer-readable storage medium is located is controlled to execute the image processing method based on style transfer according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory storing an executable program; A processor, used to run the program, wherein the program, when running, executes the image processing method based on style transfer described in any one of claims 1 to 7.

11. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the image processing method based on style transfer described in any one of claims 1 to 7 are implemented.

Images