Interface display method and device, storage medium and electronic equipment

Through the visual geometry group VGG network model and feature enhancement technology, the problem of insufficient content retention in image style conversion is solved, and the interface after style conversion is realized that both retaining the original content and having the target style, improving the quality of image style conversion and user experience .

CN119088482APending Publication Date: 2024-12-06BEIJING XIAOMI MOBILE SOFTWARE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310636725.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In the process of image style conversion, the prior art tends to ignore the content, resulting in poor quality of the image after the style conversion, and it is difficult to take into account the retention of both content and style.

Method used

The VGG network model based on the visual geometry group is used to extract the image feature, and the target image is generated through the method of enhancing lines and contours, color and brightness enhancement, to ensure that the converted interface content retains both the original content and the target style.

Benefits of technology

It improves the quality of image style conversion, ensures that the converted interface not only retains the original content characteristics, but also conforms to the target style, improving the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088482A_ABST
    Figure CN119088482A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses an interface display method and device, a storage medium and electronic equipment, and the interface display method comprises the steps: responding to an interface style switching instruction, and carrying out conversion processing on content representation and style representation of an interface before switching relative to a target style interface; and carrying out image data processing on the interface content after conversion processing, and carrying out interface display in a switched style.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to an interface display method, device, storage medium and electronic device. Background Art

[0002] With the development of artificial intelligence technology, image processing methods are becoming more and more abundant. Image style transfer can change the style of an image while retaining the main content of the image. For example, retain the main content of the collected image and convert the style of the image into an image with a certain artistic style. Summary of the invention

[0003] The present application provides an interface display method, device, storage medium and electronic device.

[0004] In a first aspect, an embodiment of the present application provides an interface display method, the method comprising:

[0005] In response to the interface style switching instruction, the content representation and style representation of the interface before switching are converted relative to the target style interface;

[0006] The interface content after the conversion is processed by image data processing, and the interface is displayed in the switched style.

[0007] Optionally, in response to the interface style switching instruction, converting the content representation and style representation of the interface before switching relative to the target style interface includes:

[0008] In response to the style switching instruction, the content representation of the interface before switching and the target style interface are sent to the server;

[0009] Receive the converted interface content sent by the server.

[0010] Optionally, in response to the interface style switching instruction, converting the content representation and style representation of the interface before switching relative to the target style interface includes:

[0011] In response to the interface style switching instruction, a local conversion of the style of the content representation of the interface before switching is performed according to the target style interface to obtain the interface content after the conversion processing.

[0012] Optionally, the converting of the content representation and style representation of the interface before switching relative to the target style interface includes:

[0013] Performing line and / or contour enhancement on the first feature map extracted from the interface before switching to obtain a third feature map;

[0014] Performing color and / or brightness enhancement on the second feature map extracted from the target style interface to obtain a fourth feature map;

[0015] Perform image synthesis processing based on the third feature map and the fourth feature map to obtain the interface content after the conversion processing.

[0016] Optionally, performing line and / or contour enhancement on the first feature map extracted from the interface before switching to obtain a third feature map includes:

[0017] Based on the attention mechanism, multi-level line and / or contour extraction is performed on the first feature map to obtain a third feature map.

[0018] Optionally, the performing color and / or brightness enhancement on the second feature map extracted from the target style interface to obtain a fourth feature map includes:

[0019] Based on the interface style of the target style interface, the second feature map extracted from the target style interface is enhanced in color and / or brightness to obtain a fourth feature map.

[0020] Optionally, the method further includes:

[0021] Extracting features of the content representation of the interface before switching based on a visual geometry group (VGG) network model to obtain the first feature map;

[0022] The target style interface is subjected to feature extraction based on a visual geometry group (VGG) network model to obtain the second feature map.

[0023] Optionally, the interface style of the target style interface includes at least: artistic style, Chinese style or cartoon style.

[0024] Optionally, the pre-switching interface includes at least one of the following:

[0025] Advertising pages;

[0026] Application Programming Interface;

[0027] Style transfer page;

[0028] Theme interface.

[0029] In a second aspect, an embodiment of the present application provides an interface display device, the device comprising:

[0030] The conversion module is used to respond to the interface style switching instruction and convert the content representation and style representation of the interface before switching to the target style interface;

[0031] The display module is used to process the converted interface content into image data and display the interface in the switched style.

[0032] Optionally, the conversion module is used to:

[0033] In response to the style switching instruction, the content representation of the interface before switching and the target style interface are sent to the server;

[0034] Receive the converted interface content sent by the server.

[0035] Optionally, the conversion module is used to:

[0036] In response to the interface style switching instruction, a local conversion of the style of the content representation of the interface before switching is performed according to the target style interface to obtain the interface content after the conversion processing.

[0037] Optionally, the conversion module includes:

[0038] A first enhancement unit, configured to perform line and / or contour enhancement on the first feature map extracted from the interface before switching to obtain a third feature map;

[0039] A second enhancement unit, configured to enhance the color and / or brightness of the second feature map extracted from the target style interface to obtain a fourth feature map;

[0040] A synthesis unit is used to perform image synthesis processing based on the third feature map and the fourth feature map to obtain the interface content after the conversion processing.

[0041] Optionally, the first enhancement unit is used to:

[0042] Based on the attention mechanism, multi-level line and / or contour extraction is performed on the first feature map to obtain a third feature map.

[0043] Optionally, the second enhancement unit is used to:

[0044] Based on the interface style of the target style interface, the second feature map extracted from the target style interface is enhanced in color and / or brightness to obtain a fourth feature map.

[0045] Optionally, the device further comprises:

[0046] A first extraction module is used to extract features of the content representation of the interface before switching based on a visual geometry group (VGG) network model to obtain the first feature map;

[0047] The second extraction module is used to extract features of the target style interface based on the visual geometry group (VGG) network model to obtain the second feature map.

[0048] Optionally, the interface style of the target style interface includes at least: artistic style, Chinese style or cartoon style.

[0049] Optionally, the pre-switching interface includes at least one of the following:

[0050] Advertising pages;

[0051] Application program interface;

[0052] Style transfer page;

[0053] Theme interface.

[0054] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0055] A display screen, used to display the pre-switching interface or the target style interface described above;

[0056] The processor is used to execute the interface display method described above.

[0057] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0058] In an embodiment of the present application, when an interface style switching instruction is detected, the style representation of the interface before switching can be converted based on the target style interface, so that the style interface after switching can retain the content of the interface before switching and also have the interface style of the target style interface. Through the interface display method of an embodiment of the present application, the style of the electronic device display interface can be freely converted on the basis of retaining the content representation of the interface before switching, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0060] Figure 1 is a flowchart of an interface display method according to an exemplary embodiment;

[0061] Figure 2 is a schematic diagram of a first interface of a style conversion application according to an exemplary embodiment;

[0062] Figure 3 is a schematic diagram of a first interface of another style conversion application according to an exemplary embodiment;

[0063] Figure 4 is a schematic diagram of a second interface of a style conversion application according to an exemplary embodiment;

[0064] Figure 5 is a schematic diagram of a third interface of a style conversion application according to an exemplary embodiment;

[0065] Figure 6 is a schematic diagram showing interface content after conversion according to an exemplary embodiment;

[0066] Figure 7 is a schematic diagram of an advertisement page according to an exemplary embodiment;

[0067] Figure 8 is a flowchart of an interface display method according to another exemplary embodiment;

[0068] Fig. 9 is a structural schematic diagram of a style transfer model according to an exemplary embodiment;

[0069] Fig.10 is a conversion effect diagram showing a style conversion of a previous interface based on a target style interface according to an exemplary embodiment;

[0070] Fig.11 is a flowchart of an interface display method according to another exemplary embodiment;

[0071] Fig.12 is a loss value of a target image generated by a different model based on a pre-switching interface and a target style interface according to another exemplary embodiment;

[0072] Fig.13 is a flowchart of an interface display method according to another exemplary embodiment;

[0073] Fig.14 is a flowchart of an interface display method according to another exemplary embodiment;

[0074] Fig.15 is a structural schematic diagram of an interface display device according to an exemplary embodiment;

[0075] Fig.16 It is a schematic structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0076] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices consistent with some aspects of the present application as detailed in the appended claims.

[0077] In the process of style conversion of some images, more attention may be paid to style and less attention to content, resulting in that the image quality after style conversion does not meet expectations. In view of this, in some cases, a first image to be style converted and a second image with a target style are obtained; a first feature map is obtained based on the first image, and a second feature map is obtained based on the second image; lines and / or contours of the first feature map are enhanced to obtain a third feature map, and color and / or brightness of the second feature map are enhanced to obtain a fourth feature map; based on the third feature map and the fourth feature map, a target image is generated, wherein the target image has the content of the first image and the style of the second image, so that both content and style are taken into account in the image conversion process, thereby improving the quality of image style conversion.

[0078] For example, Figure 1 is a flowchart of an interface display method provided in an embodiment of the present application, refer to Figure 1 , the method comprising:

[0079] S101: responding to the interface style switching instruction, converting the content representation and style representation of the interface before switching to the target style interface;

[0080] S102: Processing the converted interface content into image data, and displaying the interface in the switched style.

[0081] The interface display method of the embodiment of the present application is executed by an electronic device with a display screen.

[0082] Exemplarily, the above-mentioned electronic device may be a smart phone, a tablet computer, a laptop computer, a learning machine or other electronic device.

[0083] For example, when the electronic device displays the current application interface, the first icon may also be displayed on the display interface of the electronic device. If a trigger operation on the first icon is detected, it is determined that a style switching instruction is detected. At this time, the current display interface may be changed based on the interface style of the target style interface. That is, the style representation of the interface before the style switching is converted, and the interface content after the conversion is displayed on the electronic device. The interface content after the conversion simultaneously maintains the content representation of the interface before the switching and the interface style of the target style interface.

[0084] The target style interface mentioned above may be a style image pre-stored in the electronic device and having different styles.

[0085] The interface before switching in the embodiment of the present application may be a type of the first image in the embodiment of the present application. The target style interface here is a type of the second image in the embodiment of the present application. The interface content after conversion processing may be the target image in the embodiment of the present application.

[0086] In one embodiment, a style conversion application is also installed in the electronic device. When a style switching instruction is detected, a target style interface can be determined from the style conversion application based on the style switching instruction, and the style of the content representation of the interface before the switching can be converted based on the target style interface determined from the style conversion application.

[0087] For example, reference Figure 2 or Figure 3 When a style switching instruction is detected, the application jumps to the home page of the style conversion application, that is, the first interface. A plurality of style identifiers are displayed in the first interface of the style conversion application.

[0088] Based on this, when a user's first trigger operation on a first style identifier is detected, a second interface can be displayed on the electronic device, wherein the second interface includes at least one third image having the first style. When the style conversion application displays the second interface, a second trigger operation is detected, and when the second trigger operation is detected, the third image indicated by the second trigger operation is used as the target style interface;

[0089] For example, reference Figure 2 and Figure 3 , the first interface of the style transfer application can be Figure 2 or Figure 3 The application interface shown in the figure. Figure 2 The icon with the words "Chinese style" is the style logo and / or style name of Chinese style. Figure 3 The icon with the word "Abstract" in it is a style logo of abstract style. Figure 3 The "anime style" icon is the style logo of anime style.

[0090] Based on this, when the user needs to switch the interface style of the previous interface to an interface with Chinese style, the user can click the icon with the word "Chinese style" on the first interface. Accordingly, the electronic device will detect the user's triggering operation on the icon with the word "Chinese style" and display the following on the electronic device: Figure 4 The second interface shown in FIG. 1 shows a plurality of images with Chinese style. At this time, if a user trigger operation on a crane image with Chinese style is detected, the crane image with Chinese style can be used as the target style interface.

[0091] In an embodiment, a sample image corresponding to the style indicated by the style identifier may also be displayed near the style identifier, wherein the sample image may be a thumbnail.

[0092] In addition, an icon with the word "more" is also provided on the first interface of the style conversion application. When a trigger operation of the icon with the word "more" is detected by the user, more style identifiers can be displayed on the electronic device.

[0093] In one embodiment, the electronic device may count the number of times the user triggers each style identifier at a preset interval, and display a plurality of top-ranked style identifiers in the first interface.

[0094] For example, the top three style identifiers are displayed on the first interface.

[0095] In addition, since a plurality of third images of different styles are pre-stored in the style conversion application, the electronic device can also count the number of times the user selects each third image at a preset time interval, and display the top-ranked third images on the first interface to facilitate the user's selection.

[0096] Based on this, the user can directly select the target style interface from multiple third images displayed in the first interface, or trigger the style identifier on the first interface to obtain more third images when the style of the third image displayed on the first interface does not meet the user's needs.

[0097] In some embodiments, an import control key may also be displayed on the application interface of the style conversion application, and the import control key is used to import the style image stored in the electronic device into the style conversion application for display.

[0098] In the embodiment of the present application, considering that the styles of the style images prepared in the style conversion application are limited and may not meet the user's style conversion needs, an import control key is also provided in the style conversion application in the embodiment of the present application. Through the import control key, the style image desired by the user can be imported into the style conversion application for display, so as to meet the user's style conversion needs and further improve the user experience.

[0099] In an embodiment of the present application, a style conversion application is installed on an electronic device to allow a user to select a second image of a target style from third images of various styles, and use the second image of the target style as a target style interface. This allows free conversion of the interface style before switching to the style to be converted, thereby improving the user experience.

[0100] When the target style interface is obtained, the electronic device can also simultaneously display the interface before switching and the target style interface on the display screen.

[0101] For example, reference Figure 5 , Figure 5 The first image in the figure is the interface before switching. Figure 5 The second image in the target style interface is detected. When the electronic device detects that the user triggers the "migrate this image" icon, the electronic device can perform style conversion on the first image based on the second image to obtain Figure 6 The target image shown is the interface content after conversion.

[0102] In one embodiment, after obtaining the converted interface content, the electronic device may also store the converted interface content so that the user can view the stored converted interface content at any time.

[0103] Exemplarily, when the electronic device displays the interface content after the conversion process, the fifth icon may be displayed on the electronic device, and based on the detected triggering operation of the user on the fifth icon, the obtained interface content after the conversion process may be stored.

[0104] In one embodiment, the style conversion application installed on the electronic device also includes a user login interface. When the user needs to view the stored converted interface content, the user can enter the user name and verification information in the user login interface. Based on this, the electronic device can verify the user name and verification information entered by the user, and when the verification is passed, the converted interface content stored in the electronic device is displayed, so that the user's privacy can be protected and the user experience can be further improved.

[0105] In one embodiment, the method further comprises:

[0106] When displaying the interface content after the conversion process, displaying the third icon and the fourth icon on the electronic device;

[0107] When a trigger operation on the third icon is detected, a fourth image may be acquired through a camera or from an album, and the acquired fourth image is used as a new pre-switching interface;

[0108] When a trigger operation on the fourth icon is detected, a first interface of the style conversion application is displayed, wherein the first interface includes a plurality of style identifiers.

[0109] In an embodiment of the present application, when the electronic device displays the interface content after the conversion processing, it can also display the third image and the fourth icon on the electronic device. At this time, if the user's trigger operation on the third icon is detected, the image acquisition component can be started or the jump to the album application can be made, so that the user can collect the fourth image that needs to be style converted through the image acquisition component, or select the fourth image to be processed again from the album application, and use the fourth image as the new interface before switching.

[0110] If a trigger operation of the user on the fourth icon is detected, the first interface may be returned to display, so that the user may select the seventh image from the first interface and use the seventh image as a new target style interface.

[0111] Exemplarily, the third icon may be Figure 6 The fourth icon may be the icon of "Move Another Photo" shown in Figure 6 The "More Styles" icon is shown in .

[0112] In an embodiment of the present application, when an interface style switching instruction is detected, the style representation of the interface before switching can be converted based on the target style interface, so that the style interface after switching can retain the content of the interface before switching and also have the interface style of the target style interface. Through the interface display method of an embodiment of the present application, the style of the electronic device display interface can be freely converted on the basis of retaining the content representation of the interface before switching, thereby improving the user experience.

[0113] In one embodiment, the interface style of the target style interface at least includes: artistic style, Chinese style or cartoon style.

[0114] Among them, artistic style can at least include the style of works by Monet, Van Gogh or Picasso;

[0115] Chinoiserie can be the style of Chinese ink painting;

[0116] Anime style can be Japanese style or Disney style

[0117] In one embodiment, the above-mentioned pre-switching interface includes at least one of the following:

[0118] Advertising pages;

[0119] Application Programming Interface;

[0120] Style transfer page;

[0121] Theme interface.

[0122] For example, the advertisement page of the embodiment of the present application may be Figure 7 The ad page shown

[0123] In one embodiment, in response to the interface style switching instruction, the content representation and style representation of the interface before switching are converted relative to the target style interface, including:

[0124] In response to the interface style switching instruction, the content representation and style representation of the interface before switching are converted relative to the target style interface, including:

[0125] In response to the interface style switching instruction, the style of the content representation of the interface before switching is locally converted according to the target style interface to obtain the interface content after conversion.

[0126] Exemplarily, after determining the target style interface in the electronic device style conversion application, it is determined whether to perform style conversion of the interface before switching locally on the electronic device based on the computing power of the electronic device. When it is determined that the electronic device has strong computing power, the content representation of the interface before switching can be converted locally according to the target style interface to obtain the converted interface content, and the converted interface content can be displayed on the electronic device.

[0127] In one embodiment, in response to the interface style switching instruction, the content representation and style representation of the interface before switching are converted relative to the target style interface, including:

[0128] In response to the style switching instruction, the content representation of the interface before switching and the target style interface are sent to the server;

[0129] Receive the converted interface content sent by the server.

[0130] When it is determined that the computing power of the electronic device is weak, the content representation of the interface before switching and the target style interface can be sent to the server, so that the server can perform style conversion on the interface before switching based on the target style interface, and send the converted interface content to the electronic device. Since the entire conversion process is executed on the server side, the occupation of electronic computing resources can be reduced.

[0131] The computing capability of the electronic device may be determined based on the memory of the electronic device and the model of the CPU (Central Processing Unit).

[0132] In another embodiment, after obtaining the pre-switching interface and the target style interface, the current load rate of the electronic device may be determined, and based on the current load rate of the electronic device, it may be determined whether to send the pre-switching interface and the target style interface to the electronic device.

[0133] Exemplarily, when the current load rate of the electronic device is greater than a preset threshold, the previous interface and the target style interface are sent to the server for processing; when the current load rate of the electronic device is less than or equal to the preset threshold, the content representation of the previous interface can be locally converted in style according to the target style interface.

[0134] In one embodiment, after determining the target style interface, the previous interface and the target style interface may also be displayed on the electronic device, and when a user trigger operation on a second icon on the electronic device is detected, the previous interface and the target interface may be sent to the server, or the content representation of the previous interface may be locally converted in style according to the target style interface.

[0135] For example, the third interface displayed on the electronic device may be Figure 5 The interface shown, Figure 5 The icon for "Migrate this image" is the second icon.

[0136] In an embodiment of the present application, after determining the pre-switching interface and the target style interface based on the user's selection, a third interface can also be displayed on the electronic device, which allows the user to further confirm whether the selected target style interface is the style they need, further improving the user experience.

[0137] refer to Figure 8 In one embodiment, the content representation and style representation of the interface before switching are converted relative to the target style interface, including:

[0138] S801: Perform line and / or contour enhancement on the first feature map extracted from the interface before switching to obtain a third feature map;

[0139] S802: Performing color and / or brightness enhancement on the second feature map extracted from the target style interface to obtain a fourth feature map;

[0140] S803: Perform image synthesis processing based on the third feature map and the fourth feature map to obtain interface content after conversion processing.

[0141] refer to Fig. 9 In an embodiment of the present application, one or more style conversion models capable of converting the style of an image may be pre-stored in the electronic device, wherein the style conversion model includes a feature extraction module, a feature enhancement module and a style conversion module.

[0142] Among them, the feature extraction module can adopt a convolutional neural network model that can extract image features, for example, a LeNet-5 model, an Alexet Net model or a VGG (Visual Geometry Group) network model.

[0143] refer to Fig. 9 The feature enhancement module adopts a residual network model and a hybrid attention model (Convolutional Block Attention Module, CBAM), where the residual network model includes multiple series-connected residual blocks.

[0144] Exemplarily, the residual network model includes 8-12 residual blocks connected in series. For example, the residual network model includes 9 residual blocks connected in series, each residual block includes two convolutional layers ( Fig. 9 conv layer in ) and an activation layer ( Fig. 9 The Relu layer in .

[0145] The style conversion module may adopt the StyleProNet model or other network models capable of converting image styles, which is not limited in this application.

[0146] After the style transfer model is constructed, the style transfer model is trained using the following training methods:

[0147] A plurality of training sets are obtained, wherein each training set includes a first training image that needs to be style-converted and a second training image with the same style.

[0148] During the training process of the model, multiple first training images and second training images in the same training set can be input into the style transfer model. At this time, the style transfer model can perform feature extraction on the first training image based on the feature extraction module to obtain the seventh feature map, and perform feature extraction on the second training image to obtain the eighth feature map.

[0149] Afterwards, the lines and / or contours in the seventh feature map are enhanced based on the feature enhancement module to obtain a ninth feature map, and the color and / or brightness in the eighth feature map are enhanced by the feature enhancement module to obtain a tenth feature map.

[0150] After obtaining the ninth feature map and the tenth feature map, the ninth feature map and the tenth feature map are input into a style conversion module, the style of the ninth feature map is converted based on the style features in the tenth feature map to obtain a fifth image after style conversion, and the fifth image is upsampled to obtain an output image, wherein the output image has content features of the first training image and style features of the second training image.

[0151] During the training process of the style transfer model, a first loss value can be calculated based on the output image and the first training image, a second loss value can be calculated based on the output image and the second training image, and a total loss value can be calculated based on the first loss value and the second loss value. The model parameters of the style transfer model are adjusted based on the total loss value until a preset number of training times is reached or the model converges.

[0152] The preset number of training times may be 160,000-180,000 times, for example 160,000 times.

[0153] Exemplarily, when the feature extraction module uses a VGG network, the perceptual loss of the output image and the first training image can be calculated layer by layer after the output image and the first training image pass through the VGG network to determine the first loss value. After the fifth image and the second training image pass through the VGG network, the perceptual loss of the output image and the first training image is calculated layer by layer to obtain the second loss value

[0154] Exemplarily, the first loss value may be calculated by the following formula (1).

[0155] Loss_Perceptual=||E(c)-E(^c)||2……………………(1)

[0156] Among them, Loss_Perceptual is the first loss value, E(c) is the first training image, and E(^c) is the output image.

[0157] By calculating the L2 norm of the difference between the first training image and the output image, a first loss value can be obtained.

[0158] Exemplarily, the second loss value may be calculated by the following formula (2).

[0159]

[0160] Among them, Loss_style is the second loss value, E i (s) is the feature map of the second training image output by the i-th layer of the VGG network, E i (^c) is the feature map of the output image of the i-th layer of the VGG network, μ is the mean value, and σ is the standard deviation.

[0161] In addition, the divergence loss between the first training image and the output image can also be calculated by the following formula (3).

[0162] L_KL=KL[E(c)||E(^c)]………………(3)

[0163] After obtaining the first loss value, the second loss value and the divergence loss, the total loss value can be calculated by the following formula (4).

[0164] L=Loss_Perceptual+λLoss_style+kK_KL…………(4)

[0165] Among them, λ can be 10 and k is 5.

[0166] After completing the training of the style transfer model using a training set, the model parameters of the style transfer model are saved, and the model parameters are marked according to the image style of the second training image in the training set.

[0167] For example, when the second training images in the training set are all training images with anime style, when the model parameters are saved, the model parameters can be marked as model parameters corresponding to the anime style.

[0168] After completing the training of the style transfer model using multiple training sets with different styles, multiple sets of model parameters can be obtained, each set of model parameters corresponding to one style.

[0169] In another embodiment, multiple training sets with different styles may be merged into one training set, and the style conversion model may be trained using the merged training set. That is, the style conversion model may be trained using second training images with various styles, and the model parameters may be saved after the training is completed. That is, only one set of style parameters may be retained, and the conversion of various styles of the interface before switching may be realized using the one style conversion model.

[0170] Among them, the first training image in the above training set can adopt the MS-COCO (Microsoft Common Objects in Context) data set, or it can be an image captured by the user through an image acquisition device, which is not limited in this embodiment of the present application.

[0171] The second training image in the above training set may adopt the WikiArt dataset.

[0172] During the training process of the style transfer model, the sizes of all obtained training images can be adjusted to a first size, and then a first training image and a second training image with a second size are randomly cropped from the image with the first size, wherein the second size is smaller than the first size.

[0173] For example, the first size may be 512*512, and the second size may be 218*218.

[0174] During the training of the style transfer model, the Adam optimizer is used, the initial learning rate is 1e-4, the learning decay rate is 5e-5, the loss weights λ and k are set to 10 and 2.5 respectively, and the batch size of the model is set to 8.

[0175] After the model training is completed, you can switch to the previous interface through the trained model ( Fig. 9 The first image in the ) and the target style interface ( Fig. 9The second image in the image) is processed, wherein the interface before switching can be an image downloaded by the electronic device from the Internet, or an image captured by an image acquisition component on the electronic device, or various application interfaces displayed on the electronic device, or an image acquired by the electronic device by other means, which is not limited in the embodiments of the present application.

[0176] The target style interface mentioned above may be an image that can reflect a certain style.

[0177] For example, the target style interface may be an image with Chinese style, cartoon style, abstract style, or a specific artistic style.

[0178] For example, the target style interface may be a Chinese-style ink painting, Van Gogh's sunflowers, or a Disney-style cartoon image.

[0179] After obtaining the pre-switching interface and the target style interface, feature extraction can be performed on the pre-switching interface and the target style interface through the feature extraction model in the above-mentioned trained style conversion model to obtain a first feature map extracted from the pre-switching interface and a second feature map extracted from the target style interface.

[0180] Afterwards, the first feature map and the second feature map are input into the feature enhancement module, and the lines and / or contours in the first feature map are enhanced by the feature enhancement module to obtain a third feature map, and the color and / or brightness of the second feature map are enhanced by the feature enhancement module to obtain a fourth feature map.

[0181] Exemplarily, compared with the first feature map, the third feature map has enhanced line and / or contour related features, while weakened color related features. For example, color related features may include: brightness and / or chromaticity features, etc.

[0182] As another example, compared with the second feature map, the fourth feature map has color-related features enhanced while line and / or contour features further weakened.

[0183] In this way, the target image is generated according to the third feature map and the fourth feature map, and the generated target image is used as the interface content after the conversion process. Since the target image will inherit the line and / or contour features of the interface before the switch to a greater extent, and inherit the color corresponding style of the target style interface, the interface content after the conversion process that better meets the user's expectations is obtained.

[0184] After obtaining the third feature map and the fourth feature map, the third feature map and the fourth feature map are input into the style conversion module, so that the style conversion module performs style conversion on the third feature map based on the style features in the fourth feature map to obtain a sixth image having the same style as the target interface. Thereafter, the sixth image is upsampled to obtain the target image, that is, the interface content after conversion.

[0185] Exemplarily, the process of generating the converted interface content based on the pre-switching interface and the target style interface through the style conversion model can also be expressed by the following formula (5), as follows:

[0186]

[0187] Among them, I3 is the interface content after conversion, I1 is the interface before switching, I2 is the target style interface, RRDB is the residual block, CBAM is the hybrid attention model, SP is the style conversion module, and UP is the upsampling operation.

[0188] For example, reference Fig.10 , Fig.10 The figure shows the interface before switching ( Fig.10 The first image in the ) and the target style interface ( Fig.10 The second image in the figure), through the style conversion model of the embodiment of the present application, the interface before switching is converted into the interface content after conversion processing having the same style as the target style interface ( Fig.10 The target image in the image) is converted.

[0189] In one embodiment, after obtaining the sixth image, the sixth image may be upsampled and then the upsampled sixth image may be merged with the interface before switching to obtain the interface content after conversion.

[0190] In the embodiment of the present application, after the sixth image is obtained, the sixth image is merged with the interface before switching, so that the obtained interface content after conversion includes more content features of the interface before switching, further improving the conversion effect.

[0191] In an embodiment of the present application, the lines and / or contours of the first feature map extracted from the interface before switching are enhanced, and the color and / or brightness of the second feature map extracted from the target style interface are enhanced. Since the lines and / or contours can indicate which objects and the shapes of the objects are included in the interface before switching, that is, indicating the content features of the interface before switching, by enhancing the lines and / or contours in the first feature map, more content features in the interface before switching can be retained in the third feature map obtained. Color and brightness can indicate the style features of the target style interface. By enhancing the color and / or brightness in the second feature map, more style features in the target style interface can be included in the fourth feature map obtained after enhancement, so that the interface content generated after conversion based on the third feature map and the fourth feature map can retain more content features in the interface before switching, and more style features in the target style interface, thereby improving the style conversion effect.

[0192] In one embodiment, performing line and / or contour enhancement on the first feature map extracted from the interface before switching to obtain a third feature map includes:

[0193] Based on the attention mechanism, multi-level line and / or contour extraction is performed on the first feature map to obtain a third feature map.

[0194] Exemplarily, based on the attention mechanism, multiple residual modules in the residual network model can be used to perform multi-level feature extraction on the lines and / or contours in the first feature map to obtain a third feature map.

[0195] Among them, the hybrid attention module can execute global attention mechanism and local attention mechanism to accurately enhance the first feature map and the second feature map.

[0196] For example, based on the residual network model, the lines and / or contours of the first feature map are enhanced to obtain a fifth feature map;

[0197] The lines and / or contours of the fifth feature map are enhanced based on the hybrid attention model to obtain a third feature map.

[0198] Among them, based on the attention mechanism, the lines and / or contours of the first feature map are enhanced through the residual network model, and the color and / or brightness of the second feature map are enhanced. The implementation process can refer to the relevant technology, and the embodiments of the present application will not be repeated here.

[0199] In one embodiment, color and / or brightness enhancement is performed on the second feature map extracted from the target style interface to obtain a fourth feature map, including:

[0200] Based on the interface style of the target style interface, the second feature map extracted from the target style interface is enhanced in color and / or brightness to obtain a fourth feature map.

[0201] Considering that different styles have different color features, the attention mechanism can be used to determine which colors and / or brightness in the target style interface are enhanced based on the interface style of the target style interface. For example, when the interface style of the target style interface is indeed Chinese style, the black and gray in the second feature map are enhanced based on the attention mechanism.

[0202] Exemplarily, based on the interface style of the target style interface, a first model parameter and a second model parameter corresponding to the interface style of the target style interface may be determined from multiple sets of model parameters stored in the electronic device;

[0203] Based on the first model parameters, the model parameters of the residual network membrane model are updated;

[0204] Based on the second model parameters, the model parameters of the hybrid attention model are updated.

[0205] In the embodiment of the present application, during the training process of the style transfer model, the style transfer model is trained using training sets with different styles, and the model parameters corresponding to each style are saved after the training is completed. Therefore, after determining the target style interface, the first model parameter and the second model parameter corresponding to the interface style of the target style interface can also be determined from multiple sets of model parameters based on the interface style of the target style interface.

[0206] Afterwards, the model parameters of the residual network model are updated based on the first model parameters, and the model parameters of the hybrid attention model are updated based on the second model parameters.

[0207] Exemplarily, assuming that the interface style of the determined target style interface is anime style, at this time, the model parameters corresponding to the anime style can be determined from the stored multiple sets of model parameters, and the model parameters of the residual network model and the model parameters of the hybrid attention model can be updated based on the determined model parameters corresponding to the anime style.

[0208] After updating the model parameters of the residual network model and the hybrid attention model, the first feature map and the second feature map are feature enhanced by the residual network model and the hybrid attention model with updated parameters.

[0209] In an embodiment of the present application, the image style of each third image in the style conversion application can be identified by different identification information. Based on this, when the electronic device uses any third image as the target style interface, the interface style of the target style interface can be determined based on the identification information on the third image serving as the target style interface.

[0210] In addition, the electronic device can also determine the interface style of the target style interface based on the color features in the second feature map extracted from the second image serving as the target style interface, or obtain the interface style of the target style interface through other methods, which is not limited in this embodiment of the present application.

[0211] In the embodiment of the present application, the style conversion model is trained by training sets with different styles, and then the model parameters corresponding to each style can be obtained. When the style conversion is performed on the interface before switching, the interface style of the target style interface is first determined, and based on the determined interface style, the first model parameters and the second model parameters corresponding to the interface style of the target style interface are obtained from the stored multiple sets of model parameters. Since the first model parameters and the second model parameters are obtained by training the model with training images of the same interface style as the target style interface, the color or brightness of the target style interface is enhanced by the feature enhancement module with the first model parameters and the second model parameters, so that the color and / or brightness in the second feature map can be better enhanced, so that more style features of the target style interface are retained in the obtained fourth feature map, further improving the style conversion effect.

[0212] In one embodiment, the method further includes:

[0213] Based on the visual geometry group VGG network model, feature extraction is performed on the content representation of the interface before switching to obtain a first feature map;

[0214] Based on the visual geometry group VGG network model, features of the target style interface are extracted to obtain the second feature map.

[0215] From the above introduction, it can be known that the feature extraction module of the style transfer model can be a VGG network model. Therefore, after obtaining the target style interface, the electronic device can input the pre-switching interface and the target style interface into the VGG network model, and perform feature extraction on the pre-switching interface based on the VGG network model to obtain a first feature map, and perform feature extraction on the target style interface to obtain a second feature map.

[0216] Exemplarily, the above-mentioned VGG network model may be VGG16 or VGG19.

[0217] Among them, the specific implementation process of extracting features of the pre-switching interface and the target style interface through the VGG network model can refer to the relevant technology, and the embodiments of the present application will not be repeated here.

[0218] In an embodiment of the present application, a VGG network model is used to extract features of the pre-switching interface and the target style interface. Since the VGG network model has a smaller convolution kernel and a deeper network depth, each layer of the neural network can further extract more complex features through the output of the previous layer, thereby extracting more correct image features.

[0219] The above-mentioned implementation process of converting the style of the interface before switching based on the target style interface can be performed locally on the electronic device or on the server.

[0220] If the implementation process of converting the style of the interface before switching based on the target style interface is performed on the server, the following process can be performed:

[0221] refer to Fig.11 The embodiment of the present application also provides an interface display method, which is executed by a server, and the method includes:

[0222] S1101: receiving a content representation of a previous interface and a target style interface sent by an electronic device;

[0223] S1102: performing style conversion on the content representation of the interface before switching based on the target style interface to obtain the interface content after conversion;

[0224] S1103: Send the converted interface content to the electronic device for display.

[0225] In the embodiment of the present application, the style conversion model can be stored in a server with strong computing power, and the pre-switching interface and the target style interface sent by each electronic device connected to the server are received. Afterwards, the pre-switching interface is style converted based on the target style interface to obtain the converted interface content, and then the converted interface content is sent to the electronic device for display.

[0226] Since servers usually have strong computing power and large memory, placing the style transfer process on the server side can not only meet users' various style transfer needs, but also reduce the computing and storage resources occupied by electronic devices due to the style transfer model being placed on the electronic device side.

[0227] Among them, the server performs style conversion on the interface before switching based on the target style interface to obtain the implementation process of the interface content after conversion. The implementation process of the server performing style conversion on the interface before switching based on the target style interface to obtain the interface content after conversion can refer to the above-mentioned electronic device performing style conversion on the interface before switching based on the target style interface to obtain the implementation process of the interface content after conversion, and the embodiments of the present application will not be repeated here.

[0228] In one embodiment, the server performs style conversion on the content representation of the interface before switching based on the target style interface to obtain the interface content after conversion, including:

[0229] Extracting features of the content representation of the interface before switching to obtain a first feature map;

[0230] Extract features of the target style interface to obtain a second feature map;

[0231] Performing line and / or contour enhancement on the first feature map to obtain a third feature map, and performing color and / or brightness enhancement on the second feature map to obtain a fourth feature map;

[0232] Based on the third characteristic map and the fourth characteristic map, the interface content after conversion is generated.

[0233] In one embodiment, the server performs line and / or contour enhancement on the first feature map to obtain a third feature map, including:

[0234] Based on the attention mechanism, multi-level line and / or contour extraction is performed on the first feature map to obtain the third feature map

[0235] Exemplarily, the server may enhance the lines and / or contours of the first feature map based on the residual network model to obtain a fifth feature map;

[0236] The lines and / or contours of the fifth feature map are enhanced based on the hybrid attention model to obtain a third feature map.

[0237] In one embodiment, the method further includes:

[0238] Receiving a style identifier sent by an electronic device, where the style identifier is used to indicate an interface style of a target style interface;

[0239] Based on the style identifier, determining a first model parameter and a second model parameter corresponding to the interface style of the target style interface from a plurality of pre-stored sets of model parameters;

[0240] Based on the first model parameters, updating the model parameters of the residual network model;

[0241] Based on the second model parameters, the model parameters of the hybrid attention model are updated.

[0242] The server can also train the style transfer model by using training sets with different styles to obtain model parameters corresponding to each style. When performing style conversion on the interface before switching, the interface style of the target style interface is determined based on the style identifier sent by the received electronic device, and the first model parameters and the second model parameters corresponding to the interface style of the target style interface are obtained from the stored multiple sets of model parameters based on the determined interface style.

[0243] Afterwards, based on the first model parameters, the model parameters of the residual network model are updated, and based on the second model parameters, the model parameters of the hybrid attention model are updated. And the first feature map extracted from the interface before switching is enhanced in lines and / or contours through the parameter network model and the hybrid attention model after parameter update, and the second feature map extracted from the target style interface is enhanced in color and / or brightness.

[0244] Since the first model parameters and the second model parameters are obtained by training the model with training images of the same style as the target style interface, enhancing the color or brightness of the target style interface through a feature enhancement module having the first model parameters and the second model parameters can better enhance the color and / or brightness in the second feature map, thereby retaining more style features of the target style interface in the obtained fourth feature map, further improving the style conversion effect.

[0245] In one embodiment, sending the converted interface content to an electronic device for display includes:

[0246] Receiving a device identification sent by an electronic device;

[0247] The converted interface content is sent to the electronic device indicated by the device identifier for display.

[0248] The server of the embodiment of the present application can be connected to multiple electronic devices. Therefore, when the server receives the pre-switching interface and the target style interface sent by the electronic device connected to the server, it will also receive the device identification sent by the electronic device.

[0249] Based on this, after the electronic device generates the converted interface content based on the pre-switching interface and the target style interface, the generated converted interface content will be sent to the electronic device based on the device identification of the electronic device, so that the electronic device can receive and display the converted interface content.

[0250] The electronic device in the embodiment of the present application is any electronic device among a plurality of electronic devices connected to a server.

[0251] The device identification may be identification information that can uniquely identify the electronic device, such as the physical address or device name of the electronic device.

[0252] refer to Fig.12 , by using the embodiment of the present application, the style conversion of the interface before switching is performed based on the target style interface through different style conversion models, and the first loss value is calculated based on the difference between the interface content after the conversion and the interface before switching, and the second loss value is calculated based on the difference between the target style interface and the interface content after the conversion. The calculated loss value is as follows Fig.12 shown.

[0253] Fig.12 Adding a jump connection in the network means that after upsampling the sixth image, the sixth image and the interface before switching are fused. Fig.12 The LK loss function in is the loss function in the above formulas (1) to (4).

[0254] from Fig.12 It can be seen that the style conversion model of the embodiment of the present application is used to convert the style of the interface before switching based on the target style interface, and the first loss value between the interface content after the conversion and the interface before switching is the smallest. It can be seen that the style conversion model of the embodiment of the present application is used to convert the style of the interface before switching, and the interface content after the conversion can retain more content features.

[0255] refer to Fig.13 and Fig.14 , the embodiment of the present application also provides an interface display method, which is as follows:

[0256] The embodiment of the present application adopts a client / server architecture design, and the client (the electronic device mentioned above) can select the first image to be processed from the album, or capture the first image to be processed through the camera, and select the second image from the style conversion application installed on the client based on the required style. After that, the first image and the second image are packaged, and the packaged first image and the second image are sent to the server.

[0257] The server uses a GPU (graphics processing unit) acceleration mechanism to build a style transfer model, and based on the image style of the second image, performs style transfer on the first image through the style transfer model to obtain a target image with the same image style as the second image. After that, the target image is sent to the corresponding client in the form of a data stream.

[0258] Correspondingly, the client receives and displays the target image, and stores the target image.

[0259] Compared with using the client to perform image processing locally, the interface display method of the embodiment of the present application increases the time required to transmit the image on the network. However, the computing power of the server is much greater than the processing power of the electronic device. Therefore, the actual processing speed of the interface display method of the embodiment of the present application is much greater than the processing speed of the local client.

[0260] The training process of the above style transfer model is as follows:

[0261] The first training image in the training set uses the training data of the MS-COCO dataset, and the second training image uses the training data of the WikiArt dataset. During the training of the model, all training images are resized to 512×512, and each image is randomly cropped to 256×256.

[0262] The model was trained through 160,000 iterations, using the Adam optimizer with an initial learning rate of 1e-4 and a learning rate decay rate of 5e-5. The loss weights λ and k were set to 10 and 2.5, respectively. In addition, the batch size was set to 8 for model training. The feature extractor used in the model is a VGG-19 network pre-trained on ImageNet, and its weights were not updated during training. The feature enhancement module consists of 9 residual blocks and a hybrid attention module, and upsampling uses nearest neighbor interpolation. In the style transfer model of the embodiment of the present application, a normalization layer is not used because it affects the diversity of the synthesized image.

[0263] Image processing process:

[0264] After the model training is completed, the first image taken by the smartphone and the second image with the characteristic style are used as input during the test to generate the test results and view the effect. The first image and the second image are saved in two files respectively for retrieval and processed by the trained model. After about 10 seconds, the target image is saved in the result folder.

[0265] The model structure is as follows Fig. 9 As shown:

[0266] The input is the first image and the second image, and the output is the result of transferring the style features of the second image to the first image.

[0267] The feature extractor uses the VGG19 network to initially extract features from the first image and the second image, and then uses a feature enhancement module to enhance the first feature map extracted from the first image and the second feature map extracted from the second image. The feature enhancement module consists of multiple multi-level residual networks and an attention module (CBAM). The third feature map and the fourth feature map after feature enhancement are input into the style transfer model, and the image style of the first image is converted based on the style features of the second image, and the converted image is upsampled to obtain the target image.

[0268] The entire image processing process can also be expressed by the above formula (5).

[0269] The loss function is an evaluation indicator of the quality of the image style conversion results. It specifically includes the first loss value and the second loss value. The second loss value is to compare the generated target image with the original input second image through VGG layer by layer to calculate a perceptual loss, compare the mean and variance to calculate the 2 norms respectively and then add them; the first loss value is to compare the target image with the original input first image through VGG to calculate a perceptual loss. The smaller the loss function value, the better the image effect. Fig.12 As can be seen from the table, when the style conversion model of the embodiment of the present application is used to convert the style of an image, the total loss value is the smallest.

[0270] refer to Fig.15 , an embodiment of the present application provides an interface display device, the device comprising:

[0271] The conversion module 1501 is used to respond to the interface style switching instruction and convert the content representation and style representation of the interface before switching to the target style interface;

[0272] The display module 1502 is used to process the converted interface content into image data and display the interface in the switched style.

[0273] Optionally, the conversion module 1501 is used to:

[0274] In response to the style switching instruction, the content representation of the interface before switching and the target style interface are sent to the server;

[0275] Receive the converted interface content sent by the server.

[0276] Optionally, the conversion module 1501 is used to:

[0277] In response to the interface style switching instruction, the style of the content representation of the interface before switching is locally converted according to the target style interface to obtain the interface content after conversion.

[0278] Optionally, the conversion module 1501 includes:

[0279] A first enhancement unit is used to perform line and / or contour enhancement on the first feature map extracted from the interface before switching to obtain a third feature map;

[0280] A second enhancement unit, configured to enhance the color and / or brightness of the second feature map extracted from the target style interface to obtain a fourth feature map;

[0281] The synthesis unit is used to perform image synthesis processing based on the third feature map and the fourth feature map to obtain interface content after conversion processing.

[0282] Optionally, the first enhancement unit is used for:

[0283] Based on the attention mechanism, multi-level line and / or contour extraction is performed on the first feature map to obtain a third feature map.

[0284] Optionally, the second enhancement unit is used for:

[0285] Based on the interface style of the target style interface, the second feature map extracted from the target style interface is enhanced in color and / or brightness to obtain a fourth feature map.

[0286] Optionally, the device further comprises:

[0287] A first extraction module is used to extract features of the content representation of the interface before switching based on a visual geometry group (VGG) network model to obtain a first feature map;

[0288] The second extraction module is used to extract features of the target style interface based on the visual geometry group (VGG) network model to obtain a second feature map.

[0289] Optionally, the interface style of the target style interface includes at least: artistic style, Chinese style or cartoon style.

[0290] Optionally, the interface before switching includes at least one of the following:

[0291] Advertising pages;

[0292] Application Programming Interface;

[0293] Style transfer page;

[0294] Theme interface.

[0295] refer to Fig.16 In an embodiment of the present application, an electronic device 1600 is provided, including:

[0296] Memory 1604 for storing processor executable instructions;

[0297] The processor 1620 is connected to the memory 1604;

[0298] Among them, the processor 1620 is configured to execute the interface display method provided by any of the aforementioned technical solutions.

[0299] A block diagram of an electronic device 1600 is shown according to an exemplary embodiment. For example, the electronic device 1600 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc. having a display screen.

[0300] refer to Fig.16 , the electronic device 1600 may include one or more of the following components: a processing component 1602 , a memory 1604 , a power component 1606 , a multimedia component 1608 , an audio component 1610 , an input / output (I / O) interface 1612 , a sensor component 1614 , and a communication component 1616 .

[0301] The processing component 1602 generally controls the overall operation of the electronic device 1600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1602 may include one or more processors 1620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 1602 may include one or more modules to facilitate the interaction between the processing component 1602 and other components. For example, the processing component 1602 may include a multimedia module to facilitate the interaction between the multimedia component 1608 and the processing component 1602.

[0302] The memory 1604 is configured to store various types of data to support operations on the electronic device 1600. Examples of such data include instructions for any application or method operating on the electronic device 1600, contact data, phone book data, messages, pictures, videos, etc. The memory 1604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0303] The power supply component 1606 provides power to the various components of the electronic device 1600. The power supply component 1606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 1600.

[0304] The multimedia component 1608 includes a screen that provides an output interface between the electronic device 1600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1608 includes a front camera and / or a rear camera. When the electronic device 1600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0305] The audio component 1610 is configured to output and / or input audio signals. For example, the audio component 1610 includes a microphone (MIC), and when the electronic device 1600 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1604 or sent via the communication component 1616. In some embodiments, the audio component 1610 also includes a speaker for outputting audio signals.

[0306] I / O interface 1612 provides an interface between processing component 1602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0307] The sensor assembly 1614 includes one or more sensors for providing various aspects of status assessment for the electronic device 1600. For example, the sensor assembly 1614 can detect the open / closed state of the electronic device 1600, the relative positioning of components, such as the display and keypad of the electronic device 1600, and the sensor assembly 1614 can also detect the position change of the electronic device 1600 or a component of the electronic device 1600, the presence or absence of user contact with the electronic device 1600, the orientation or acceleration / deceleration of the electronic device 1600, and the temperature change of the electronic device 1600. The sensor assembly 1614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1614 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0308] The communication component 1616 is configured to facilitate wired or wireless communication between the electronic device 1600 and other devices. The electronic device 1600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0309] In an exemplary embodiment, the electronic device 1600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0310] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1604 including instructions, and the instructions can be executed by a processor 1620 of the electronic device 1600 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0311] An embodiment of the present application provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by a processor of a computer, the computer can execute the interface display method of one or more of the aforementioned technical solutions.

[0312] When a processor executes an instruction, it can perform at least the following steps:

[0313] In response to the interface style switching instruction, the content representation and style representation of the interface before switching are converted relative to the target style interface;

[0314] The converted interface content is processed into image data, and the interface is displayed in the switched style.

[0315] Optionally, in response to the interface style switching instruction, the content representation and style representation of the interface before switching are converted relative to the target style interface, including:

[0316] In response to the style switching instruction, the content representation of the interface before switching and the target style interface are sent to the server;

[0317] Receive the converted interface content sent by the server.

[0318] Optionally, in response to the interface style switching instruction, the content representation and style representation of the interface before switching are converted relative to the target style interface, including:

[0319] In response to the interface style switching instruction, the style of the content representation of the interface before switching is locally converted according to the target style interface to obtain the interface content after conversion.

[0320] Optionally, the content representation and style representation of the interface before switching are converted relative to the target style interface, including:

[0321] Performing line and / or contour enhancement on the first feature map extracted from the interface before switching to obtain a third feature map;

[0322] Performing color and / or brightness enhancement on the second feature map extracted from the target style interface to obtain a fourth feature map;

[0323] The image synthesis process is performed based on the third characteristic map and the fourth characteristic map to obtain the interface content after conversion.

[0324] Optionally, performing line and / or contour enhancement on the first feature map extracted from the interface before switching to obtain a third feature map includes:

[0325] Based on the attention mechanism, multi-level line and / or contour extraction is performed on the first feature map to obtain a third feature map.

[0326] Optionally, color and / or brightness enhancement is performed on the second feature map extracted from the target style interface to obtain a fourth feature map, including:

[0327] Based on the interface style of the target style interface, the second feature map extracted from the target style interface is enhanced in color and / or brightness to obtain a fourth feature map.

[0328] Optionally, the method further comprises:

[0329] Based on the visual geometry group VGG network model, feature extraction is performed on the content representation of the interface before switching to obtain a first feature map;

[0330] Based on the visual geometry group VGG network model, features of the target style interface are extracted to obtain the second feature map.

[0331] Optionally, the interface style of the target style interface includes at least: artistic style, Chinese style or cartoon style.

[0332] Optionally, the interface before switching includes at least one of the following:

[0333] Advertising pages;

[0334] Application Programming Interface;

[0335] Style transfer page;

[0336] Theme interface.

[0337] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0338] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An interface display method, characterized in that: The method comprises: In response to the interface style switching instruction, the content representation and style representation of the interface before switching are converted relative to the target style interface; The interface content after the conversion is processed by image data processing, and the interface is displayed in the switched style.

2. The interface display method according to claim 1, characterized in that: The response interface style switching instruction converts the content representation and style representation of the interface before switching relative to the target style interface, including: In response to the style switching instruction, the content representation of the interface before switching and the target style interface are sent to the server; Receive the converted interface content sent by the server.

3. The interface display method according to claim 1, characterized in that: The response interface style switching instruction converts the content representation and style representation of the interface before switching to the target style interface, including: In response to the interface style switching instruction, a local conversion of the style of the content representation of the interface before switching is performed according to the target style interface to obtain the interface content after the conversion processing.

4. The method according to claim 1, characterized in that: The conversion process of the content representation and style representation of the interface before switching relative to the target style interface includes: Performing line and / or contour enhancement on the first feature map extracted from the interface before switching to obtain a third feature map; Performing color and / or brightness enhancement on the second feature map extracted from the target style interface to obtain a fourth feature map; Perform image synthesis processing based on the third feature map and the fourth feature map to obtain the interface content after the conversion processing.

5. The method according to claim 4, characterized in that The performing line and / or contour enhancement on the first feature map extracted from the interface before switching to obtain a third feature map comprises: Based on the attention mechanism, multi-level line and / or contour extraction is performed on the first feature map to obtain a third feature map.

6. The method according to claim 5, characterized in that The step of performing color and / or brightness enhancement on the second feature map extracted from the target style interface to obtain a fourth feature map includes: Based on the interface style of the target style interface, the second feature map extracted from the target style interface is enhanced in color and / or brightness to obtain a fourth feature map.

7. The method according to claim 6, characterized in that The method further comprises: Extracting features of the content representation of the interface before switching based on a visual geometry group (VGG) network model to obtain the first feature map; The target style interface is subjected to feature extraction based on a visual geometry group (VGG) network model to obtain the second feature map.

8. The method according to any one of claims 1 to 7, characterized in that: The interface style of the target style interface at least includes: artistic style, Chinese style or cartoon style.

9. The method according to claim 1, characterized in that: The pre-switching interface includes at least one of the following: Advertising pages; Application program interface; Style transfer page; Theme interface.

10. An interface display device, characterized in that: The device comprises: The conversion module is used to respond to the interface style switching instruction and convert the content representation and style representation of the interface before switching to the target style interface; The display module is used to process the converted interface content into image data and display the interface in the switched style.

11. The interface display device according to claim 10, characterized in that: The conversion module is used for: In response to the style switching instruction, the content representation of the interface before switching and the target style interface are sent to the server; Receive the converted interface content sent by the server.

12. The interface display device according to claim 10, characterized in that: The conversion module is used for: In response to the interface style switching instruction, a local conversion of the style of the content representation of the interface before switching is performed according to the target style interface to obtain the interface content after the conversion processing.

13. The device according to claim 10, characterized in that The conversion module comprises: A first enhancement unit, configured to perform line and / or contour enhancement on the first feature map extracted from the interface before switching to obtain a third feature map; A second enhancement unit, configured to enhance the color and / or brightness of the second feature map extracted from the target style interface to obtain a fourth feature map; A synthesis unit is used to perform image synthesis processing based on the third feature map and the fourth feature map to obtain the interface content after the conversion processing.

14. The device according to claim 13, characterized in that The first enhancement unit is used for: Based on the attention mechanism, multi-level line and / or contour extraction is performed on the first feature map to obtain a third feature map.

15. The device according to claim 14, characterized in that The second enhancement unit is used for: Based on the interface style of the target style interface, the second feature map extracted from the target style interface is enhanced in color and / or brightness to obtain a fourth feature map.

16. The device according to claim 15, characterized in that The device further comprises: A first extraction module is used to extract features of the content representation of the interface before switching based on a visual geometry group (VGG) network model to obtain the first feature map; The second extraction module is used to extract features of the target style interface based on the visual geometry group (VGG) network model to obtain the second feature map.

17. The device according to any one of claims 10 to 16, characterized in that: The interface style of the target style interface at least includes: artistic style, Chinese style or cartoon style.

18. The device according to claim 10, characterized in that The pre-switching interface includes at least one of the following: Advertising pages; Application program interface; Style transfer page; Theme interface.

19. An electronic device, characterized in that: include: A display screen, used to display the pre-switching interface or the target style interface according to any one of claims 1 to 9; A processor, used to execute the interface display method described in any one of claims 1 to 9 above.