Filter generation method, apparatus, device, medium, and program product

Abstract

The present application provides a filter generation method, device, equipment, medium and program product, relating to the technical field of image processing. The method comprises: receiving the selection of a first image and a second image, wherein the first image is configured to present a preview effect; displaying a plurality of third images and a plurality of first filter information images, wherein the first filter information image is configured to display the color mapping rule of the second image, and the third image is an effect image after applying the color mapping rule to the second image. The first filter information image is automatically generated, solving the problem of long production cycle and low filter generation efficiency of manual filter production, and improving the filter generation efficiency.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a filter generation method, apparatus, device, medium, and program product. Background Technology

[0002] Filters, as a common image processing technique, are widely used in live streaming color grading, video color grading, and post-production color grading in photography. However, the production cycle for filters in these technologies is relatively long, and production efficiency urgently needs improvement. Summary of the Invention

[0003] This invention provides a filter generation method, apparatus, device, medium, and program product that solves the problem of low filter generation efficiency.

[0004] In one scenario, this paper provides a filter generation method, which includes: Receive a selection of a first image and a second image, wherein the first image is configured to present a preview effect; The system displays multiple third images and multiple first filter information images, wherein the first filter information images are configured to display the color mapping rules of the second image, and the third images are the effect images after applying the color mapping rules to the second image.

[0005] In one instance, this paper also provides a filter generation apparatus, which includes: A receiving module is used to receive a selection of a first image and a second image, wherein the first image is configured to present a preview effect; The display module is used to display multiple third images and multiple first filter information images, wherein the first filter information images are configured to display the color mapping rules of the second images, and the third images are the effect images after applying the color mapping rules to the second images.

[0006] In one instance, this document also provides an electronic device comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the filter generation method as described herein.

[0007] In one instance, this document also provides a storage medium containing computer-executable instructions that, when executed by a computer processor, are used to perform the filter generation method as described herein.

[0008] In another scenario, this document also provides a computer program product, including a computer program that, when executed by a processor, implements the filter generation method as described in any of the present documents.

[0009] The above method receives a selection of a first image and a second image. The first image is configured to present a preview effect, displaying multiple third images and multiple first filter information images. The first filter information images are configured to display the color mapping rules of the second image, and the third images are the effect images after applying the color mapping rules to the second image. This method inputs a first image and a second image, outputs multiple first filter information images presenting the color mapping rules, and multiple effect images obtained by applying the color mapping rules to the second image. This enables real-time preview of the filter effect, facilitating the selection of the first filter information image that meets the expectations based on the effect images. The selected first filter information image can be applied to other user images, achieving intelligent and highly reusable filter generation, shortening the filter generation cycle, and solving the problems of long production cycles and low efficiency in manual filter creation, thus improving filter generation efficiency. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments described herein will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0011] Figure 1 This is a schematic diagram of a filter generation system under one specific scenario. Figure 2 This is a flowchart illustrating a filter generation method for one scenario. Figure 3 This is a network structure diagram of an image processing model under one scenario. Figure 4 This is a network structure diagram of an image processing model in another scenario. Figure 5 This is a flowchart illustrating a filter generation method for another scenario. Figure 6 This is a schematic diagram of the first page in one scenario; Figure 7 A flowchart illustrating an optional instance of a filter generation method in one scenario; Figure 8 This is a schematic diagram of a filter generation device in one scenario. Figure 9 This is a schematic diagram of an electronic device used to implement a filter generation method in one scenario. Detailed Implementation

[0012] The embodiments will now be described in more detail with reference to the accompanying drawings. While some embodiments are shown in the drawings, it should be understood that the technical solutions can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the technical solutions herein. It should be understood that the illustrated drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the technical solutions.

[0013] It should be understood that the steps described in the method implementation may be performed in different orders and / or in parallel. Furthermore, the method implementation may include additional steps and / or omit the steps shown. The scope of this document is not limited in this respect.

[0014] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one situation" means "at least one situation"; the term "another situation" means "at least one additional situation"; the term "some situations" means "at least some situations". Definitions of other terms will be given in the following description.

[0015] It should be noted that the concepts of "first" and "second" mentioned are only used to distinguish different devices, modules or units, and are not used to limit the order of the functions performed by these devices, modules or units or their interdependencies.

[0016] It should be noted that the terms "one" and "more" used in this document are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0017] The names of messages or information exchanged between multiple devices in this document are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0018] It is understood that before using the technical solutions disclosed in the various embodiments of this document, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this document in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0019] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as electronic devices, applications, servers, or storage media, that perform the operations described herein, based on the prompt message.

[0020] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0021] It is understood that the above notification and user authorization process is merely illustrative and does not limit the implementation method described in this article. Other methods that comply with relevant laws and regulations may also be applied to the implementation method described in this article.

[0022] It is understood that the data involved in the technical solutions in this article (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0023] In some cases, the provided solution can be applied to Figure 1 The filter generation system shown may include a client 101 and a server 102. The client 101 may include, but is not limited to, browsers, applications (Apps), HyperText Markup Language (HTML) applications, lightweight applications (also known as mini-programs), or cloud applications. The client 101 may be deployed on an electronic device and relies on the operation of that device or certain applications on the device to implement its functions. The electronic device may be, for example, a device with a display screen that supports information browsing, such as a smartphone, tablet, personal computer, or other client terminal. For ease of understanding, Figure 1 The client is primarily represented by a device. Other applications, such as media content editing applications and image processing applications, can also be configured on the electronic device. Server 102 can be one or more servers providing various services. That is, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server; furthermore, it can be a server for a distributed system, a server integrating blockchain technology, a cloud server, or an intelligent cloud computing server or intelligent cloud host deployed with machine learning models, etc.

[0024] The filter generation method described in this paper allows interaction between client 101 and server 102, such as receiving or sending messages. For example, in this paper, server 102 can receive the first image and the second image sent by client 101 based on the information carrier, and send the third image and the first filter information image to client 101 for display on the display interface.

[0025] It should be noted that the filter generation method can be executed by client 101, or by client 101 and server 102, with different functional parts of the corresponding filter generation device deployed on client 101 and server 102 respectively; wherein, the front-end interaction module and local data acquisition module of the device are deployed on client 101, and the back-end data processing module and data storage module are deployed on server 102, and client 101 and server 102 achieve data interaction and functional collaboration through network communication. It should be understood that... Figure 1 The number of clients and servers shown is for illustrative purposes only. Any number of clients and servers can be configured to meet specific implementation requirements.

[0026] Figure 2 This is a flowchart illustrating a filter generation method for one scenario. This method is applicable to image processing scenarios, particularly filter creation scenarios, such as learning the style features of a target image to obtain filter information. This filter generation method can be executed by a filter generation device, which can be implemented in software and / or hardware, optionally through an electronic device such as a mobile terminal, PC, or server. Figure 2 As shown, the filter generation method may specifically include: S210, Receive selection of a first image and a second image, wherein the first image is configured to present a preview effect.

[0027] The first image is a target image selected by the user, possessing preset style characteristics. These preset style characteristics include color and lighting features. For example, a user can upload the first image, and an image processing model can perform structured learning on the overall tone, contrast relationship, brightness structure, and lighting features of the first image to obtain a reusable filter infographic. This avoids the need for designers to repeatedly adjust colors to create filters, where filter effects are highly dependent on the designer's experience, and sometimes a single image or segment can take hours to create, resulting in a long filter file production cycle. Users can be designers of special effects filters or ordinary users. The filter infographic can be a color lookup table (LUT) image, etc. The second image is the image to which the filter effect is to be applied, selected by the user. By performing style transfer processing on the second image using the LUT image, the filter application effect can be intuitively presented, enabling real-time preview of the filter effect and facilitating the selection of filter information that meets expectations.

[0028] In some cases, image processing models can be neural network models used to achieve style feature transfer. Image processing models include a first input module, a second input module, a first style extraction module, a second style extraction module, a color removal module, and a color filling module, etc. Figure 3 Here is a network structure diagram of an image processing model in one scenario, such as... Figure 3 As shown, the image processing model 300 includes a first input module 310, a second input module 360, a first style extraction module 320, a second style extraction module 330, a color removal module 340, and a color filling module 350. The first input module 310 is used to determine the feature vector of the second image. The second input module 360 ​​is used to concatenate the feature vectors of the first and second images along the channel dimension to obtain a concatenated feature vector. Optionally, the input module includes convolutional layers, a moving bottleneck convolution module, and an attention module. The first style extraction module 320 is used to encode the feature vector of the second image into a first style matrix and a first feature map. Each dimension of the first style matrix corresponds to a style attribute of the second image. The feature vectors in the first feature map represent the local color features of the second image. The second style extraction module 330 is used to encode the concatenated feature vector into a second style matrix. Each dimension of the second style matrix corresponds to a style attribute of the concatenated image along the channel dimension of the first and second images. Optionally, the style extraction module includes a style feature extraction submodule and a color feature extraction submodule. The style feature extraction submodule includes pooling layers, fully connected mapping layers, and style normalization layers. The color feature extraction submodule includes feature selection layers, upsampling layers, and channel attention layers. The color removal module 340 acquires the feature vector of the second image and the first feature map, and outputs a grayscale image with color features removed but texture features preserved. Optionally, the color removal module 340 achieves accurate color stripping through a network structure including an encoder and a decoder. The color filling module 350 receives the grayscale image and the second style matrix, and outputs the style-transferred image. Optionally, the color filling module 350 uses a generative adversarial network architecture to achieve style transfer. In the above approach, by inputting the grayscale image and the second style matrix containing the style attribute features of the second image into the color filling module, the semantically compliant features in the second image are preserved while enhancing the realism of the generated image, based on style transfer. The semantic compliance condition refers to objects, etc., that do not appear in the first image in the second image. For example, semantically compliant features include features corresponding to skin regions, etc.

[0029] In another scenario, the image processing model includes an input module, a style extraction module, a color removal module, and a color filling module. Figure 4 Here is a network structure diagram for another type of image processing model, such as... Figure 4As shown, the image processing model 400 includes an input module 410, a style extraction module 420, a color removal module 430, and a color filling module 440. The input module 410 determines the feature vectors of the first image and the second image, respectively. The style extraction module 420 encodes the feature vectors of the second image into a first style matrix and a first feature map. Each dimension of the first style matrix corresponds to a style attribute of the second image. The feature vectors in the first feature map represent the local color features of the second image. The style extraction module 420 encodes the feature vectors of the first image into a second style matrix and a second feature map. Each dimension of the second style matrix corresponds to a style attribute of the first image. The feature vectors in the second feature map represent the local color features of the first image. The color removal module 430 acquires the feature vectors and the first feature map of the second image and outputs a grayscale image with color features removed but texture features preserved. The color filling module 440 receives the grayscale image and the second style matrix and outputs the style-transferred image.

[0030] S220. Display multiple third images and multiple first filter information images, wherein the first filter information images are configured to display the color mapping rules of the second image, and the third images are the effect images after applying the color mapping rules to the second image.

[0031] The first filter information image includes color mapping rules. Optionally, the color mapping rules include mapping rules for visual features, such as color features and lighting features. For example, the first filter information image is a color lookup table image (LUT image). The essence of a LUT is a color mapping rule between the original color and the target color. Each grid point in the LUT image stores a set of input colors and their corresponding output colors; these grid points constitute a complete color mapping rule. The LUT image is used to replace the RGB color values ​​in the second image with corresponding new color values ​​to obtain the third image.

[0032] Each first filter information image is configured to display a set of color mapping rules. Multiple first filter information images correspond to multiple sets of color mapping rules. Optionally, the process of determining multiple first filter information images includes: receiving a first image and a second image; selecting a fourth image similar to the first image from an image set. The image set includes multiple types of images, including at least one of shooting scene type and content subject type. The image set can be viewed as a collection of images containing different shooting scenes and different types of shooting objects. The image set is pre-configured and can be switched in response to user interaction. Optionally, the image set can also be user-defined. This document does not specifically limit this. The selected image set is named the first image set. The similarity between the fourth image and the first image meets the requirements, wherein the similarity includes at least one of color similarity and content semantic similarity. Semantic features represent the meaning of the image content. Semantic features include object semantics, scene semantics, and event semantics, etc. Object semantics is used to represent specific objects in the image, such as a person, a cat, or a cup. Scene semantics is used to represent the environment in which the image is located, such as a living room, a forest, or an office, etc. Event semantics is used to represent actions or events in an image, such as running, dancing, or drinking water. Optionally, the fourth image is an image in the first image set whose similarity to the first image in at least one of color and semantic features exceeds a preset threshold. Alternatively, the fourth image is an image in the first image set whose similarity to the first image in at least one of color and semantic features meets the ranking requirements. For example, N images from the first image set whose similarity to the first image in color and semantic features is greater than the preset threshold are selected as N fourth images. Alternatively, N images from the first image set whose similarity to the first image in color and semantic features is ranked are selected as N fourth images. Here, N is a preset value.

[0033] In some cases, the first filter information image is the result of fusing multiple second filter information images. The second filter information images include color mapping rules used to transfer the style features of the first image to a fourth image, where the fourth image belongs to the first image set. For example, based on the similarity in color and semantic features between images in the first image set and the first image, N fourth images are obtained from the first image set. A single fourth image and the first image are input into an image processing model. The image processing model transfers the style features of the first image to the fourth image, resulting in a style transfer result image. The style features include color features and lighting features, etc. Based on the color mapping relationship between the fourth image and the style transfer result image, color mapping rules are determined, thereby obtaining the second filter information images. Applying the above method to the style transfer of N fourth images yields N second filter information images. Fusing the N second filter information images yields multiple first filter information images. Because the second filter information image obtained by style transfer from a single fourth image not only contains color mapping rules but also inevitably includes content-specific biases such as color distribution, local exposure, skin tone ratio, and scene lighting of the fourth image. These content-specific biases can lead to defects such as excessively strong or weak color casts, unstable skin tones, collapsed shadows, or blown highlights when the second filter information image is applied to another image with different distributions (e.g., different scenes, different skin tones, and different local exposures). By fusing multiple second filter information images to obtain multiple first filter information images, content-specific biases can be greatly reduced, resulting in a more stable filter effect and increased aesthetics, thereby improving the filter's generalization ability.

[0034] Optionally, a first identifier is displayed, which represents a first image set, and the first image set represents a set of images in a selected state; in response to a trigger on the first identifier, a set of candidate identifiers is displayed, the set of candidate identifiers including the first identifier, a second identifier, and a third identifier, wherein the second identifier represents a second image set, and the second image set represents a set of images in a selectable state, and the third identifier is used to trigger the uploading of images that conform to the application scenario of the filter to be generated to create a third image set, and at least some of the images in different image sets are different.

[0035] The first page displays a first identifier, which represents the first image set. The first page can be an interactive page for a filter-making tool. Optionally, the filter-making tool can be integrated as a component into a media content editing application. For example, media content editing applications include video editing applications, etc. The first image set refers to the set of images used to determine the information images for the second filter. Multiple image sets can be pre-configured, with different numbers of images in each set. Optionally, multiple images from various scenes can be obtained from sources such as image libraries or the internet. At least two image sets can be constructed based on the obtained images, with at least two sets having different numbers of images and at least some of the images differing. The first identifier represents the selected image set. A second identifier can be used to represent the unselected image set. A third identifier can be used to represent a custom image set. The custom image set consists of images uploaded by the user. For example, if a user wants to create dog-related effects, they can upload multiple photos of dogs taken in different scenes to form the third image set. Since the first filter information image is obtained by fusing multiple second filter information images, and the second filter information images are configured to display the color mapping rules used to transfer the style features of the first image to the fourth image in the first image set, the color mapping rules displayed by the first filter information image can be affected by switching the image set corresponding to the first identifier.

[0036] In one scenario, in response to a trigger operation on the first page targeting the first identifier, a set of candidate identifiers is displayed. The candidate identifier set includes identifier information from a preset image set and custom labels. In response to a trigger operation on the identifier information of the preset image set, the first identifier is updated using the identifier information of the triggered image set. In response to a trigger operation on the first control on the first page, the third image and the first filter information image are updated. Specifically, at least one of the color features and lighting features of the third image before and after the update is different. The color mapping rules of the first filter information image before and after the update are different.

[0037] Optionally, a download control is displayed, wherein the download control is used to download at least one of the third image and the first filter information image. To facilitate viewing the third image and the first filter information image, they can be downloaded to a preset storage space first. By viewing the third image, a third image whose effect meets expectations can be selected. Then, based on the correspondence between the third image and the first filter information image, a first filter information image that meets expectations is determined, and a filter effect package is obtained by packaging at least one first filter information image that meets expectations.

[0038] The above method receives a selection of a first image and a second image. The first image is configured to present a preview effect, displaying multiple third images and multiple first filter information images. The first filter information images are configured to display the color mapping rules of the second image, and the third images are the effect images after applying the color mapping rules to the second image. This method inputs a first image and a second image, outputs multiple first filter information images presenting the color mapping rules, and multiple effect images obtained by applying the color mapping rules to the second image. This enables real-time preview of the filter effect, facilitating the selection of the first filter information image that meets the expectations based on the effect images. The selected first filter information image can be applied to other user images, achieving intelligent and highly reusable filter generation, shortening the filter generation cycle, and solving the problems of long production cycles and low efficiency in manual filter creation, thus improving filter generation efficiency.

[0039] Figure 5 This is a flowchart illustrating a filter generation method under another scenario. The technical solution in this scenario can be combined with implementation methods in other scenarios. For identical or related parts, descriptions of other scenarios can be used, and will not be repeated here. Figure 5 As shown, the method in this case may specifically include: S510, in response to an image selection operation, displays at least one first image and at least one second image on the first page.

[0040] For example, a first page is displayed, wherein the first page includes a third area and a fourth area. In response to an image selection operation, at least one of the first images is displayed in the third area, and at least one of the second images is displayed in the fourth area.

[0041] The third area refers to a partial area on the first page used to display the first image. The fourth area refers to a partial area on the first page used to display the second image. Optionally, the third and fourth areas can be different areas on the first page. Optionally, the first background text is displayed in the third area. For example, the first background text could be a dragged-in first image or a click to upload the first image. The second background text is displayed in the fourth area. For example, the second background text could be a dragged-in second image or a click to upload the second image.

[0042] Figure 6 This is a diagram illustrating the first page in one scenario. For example... Figure 6As shown, a first page 600 is displayed. The first page 600 includes a third area 610 and a fourth area 620. In response to a click operation within the third area 610, an album page 630 is displayed. In response to a selection operation within the album page 630, a first image 640 is displayed in the third area 610. In response to a click operation within the fourth area 620, the album page 630 is displayed. In response to a selection operation within the album page 630, a second image 650 is displayed in the fourth area 620.

[0043] Optionally, in response to an image upload operation in the third area, a first image is displayed in the third area. In response to an image upload operation in the fourth area, a second image is displayed in the fourth area.

[0044] Optionally, in response to a batch upload operation in the third area, at least two first images are displayed in the second area. In response to a batch upload operation in the fourth area, at least two second images are displayed in the fourth area.

[0045] In one scenario, at least two of the first images differ in at least one of their color and lighting characteristics. Through batch upload operations, at least two first images with different style characteristics can be uploaded to the third region.

[0046] S520, Display the first control.

[0047] S530. Display a first identifier, which is used to represent a first image set, which represents a set of images in the selected state.

[0048] See Figure 6 The first page 600 displays a first identifier 6100, which is used to represent a first image set. Multiple second filter information images are generated based on the first image set and the first image.

[0049] S540, In response to a trigger operation on the first control, display multiple third images and a first filter information image.

[0050] For example, in response to the triggering operation of the first control, the plurality of third images are displayed in a first area of ​​the first page; the plurality of first filter information images are displayed in a second area of ​​the first page, wherein the plurality of first filter information images are arranged in a matrix.

[0051] Here, the first region refers to a local area on the first page used to display the third image. The second region refers to a local area on the first page used to display the first filter information image. Optionally, the first region and the second region can be different regions on the first page. Any third image in the first region has a corresponding first filter information image in the second region.

[0052] In response to a click on the first control, multiple fourth images similar to the first image are selected from the first image set based on color similarity and semantic content similarity. The fourth images and the first image are input into an image processing model to obtain a style transfer image output by the model. Based on the color mapping relationship between the fourth images and the style transfer image, a second filter information image is obtained. The second filter information image includes the color mapping rules between the fourth images and the style transfer image. Multiple second filter information images are fused using a preset algorithm to obtain a first filter information image. It should be noted that different fusion algorithms can produce different first filter information images. Based on multiple second filter information images, different fusion algorithms are used to obtain multiple first filter information images in this text.

[0053] See Figure 6 As shown, a first control 660 is displayed on the first page 600. In response to a trigger operation on the first control 660, multiple third images are displayed in the first area 670. Multiple first filter information images are displayed in the second area 680. Optionally, the multiple first filter information images are presented in an n*m matrix arrangement.

[0054] Optionally, the first filter information image can be a color lookup table image. A color lookup table image includes multiple grid points, and at least some of the grid points in different color lookup tables have different attributes. The attributes include at least one of color features and lighting features.

[0055] In one scenario, multiple third images corresponding to the same second image are presented in a stitched format, and the stitched images corresponding to different second images are presented in a matrix arrangement.

[0056] See Figure 6 As shown, a stitched image 690 of multiple third images corresponding to the same second image is displayed in the first region 670, and the stitched images corresponding to different second images are arranged in a matrix.

[0057] In one scenario, in response to an image triggering operation within the second region, a magnified view of the triggered image and a zoomed-out view of the untriggered image are displayed in the second region.

[0058] See Figure 6 As shown, in response to a click operation on the first filter information image within the second region 680, a magnified view of the clicked first filter information image is displayed in the second region, and multiple zoomed-out views of the unclicked first filter information images are displayed at the bottom of the second region 680. Optionally, the unclicked first filter information images are displayed in a row.

[0059] The above method involves uploading at least one first image and at least one second image to a first page, combining a fourth image similar to the first image in the first image set with the first image to determine multiple second filter information images, and fusing these multiple second filter information images to determine multiple first filter information images. These first filter information images are configured to display color mapping rules. Multiple color mapping rules are applied to at least one second image to obtain multiple third images. The filter effect of the first filter information images is displayed through these third images, enabling real-time preview of the filter effect and facilitating users to select the first filter information image whose filter effect meets their expectations. In this method, multiple second filter information images are determined by using a fourth image similar to the first image in various application scenarios within the first image set, and these multiple second filter information images are then fused to obtain multiple first filter information images, thereby improving the stability and aesthetics of filter generation.

[0060] Figure 7 This is a flowchart illustrating an optional example of a filter generation method under one scenario. The technical solution in this scenario can be combined with implementation methods in other scenarios. For identical or related parts, descriptions of other scenarios can be used, and will not be repeated here. Figure 7 As shown, the method in this case may specifically include: S710, acquire the first image and the second image.

[0061] S720, In response to the triggering operation of the first control, determine the set of images to be migrated based on the similarity between the images in the first image set and the first image.

[0062] The set of images to be transferred is the set of fourth images. The fourth image is an image in the first set whose color similarity and semantic content similarity with the first image exceeds a preset threshold.

[0063] In one scenario, an image in the first image set whose color similarity to the first image exceeds a preset threshold can be identified as the fourth image. Alternatively, an image in the first image set whose semantic similarity to the first image exceeds a preset threshold can be identified as the fourth image. Or, an image in the first image set whose color similarity to the first image exceeds a preset threshold and whose semantic similarity exceeds a preset threshold can be identified as the fourth image. A set of images to be transferred is then constructed based on multiple fourth images.

[0064] In another scenario, the images in the first image set are sorted according to their color similarity to the first image, and multiple fourth images are obtained based on the sorting result. Alternatively, the images in the first image set are sorted according to their semantic similarity to the first image, and multiple fourth images are obtained based on the sorting result. Another approach is to perform a weighted calculation of color and semantic similarity, sort the images in the first image set based on the calculation result, and obtain multiple fourth images based on the sorting result. A set of transfer maps is then constructed based on these multiple fourth images.

[0065] S730. By transferring the style features of the first image to each of the fourth images in the set of images to be transferred, multiple second filter information images are obtained.

[0066] The transfer operation involves inputting a first image and a fourth image into an image processing model. The image processing model then transfers the style features of the first image to the fourth image, resulting in a style-transferred image, as detailed in the above embodiment. Based on the color mapping relationship between the fourth image and the style-transferred image, a second filter information image is obtained.

[0067] S740. Multiple fusion algorithms are used to fuse multiple second filter information images to obtain multiple first filter information images.

[0068] The fusion algorithms include mean averaging, median averaging, K-means clustering, and principal component analysis. For principal component analysis, multiple different principal component retention ratios are pre-set, allowing for the generation of different first-filter information images.

[0069] In one scenario, taking N second-filter information images as an example, these N images have the same resolution, the same color space, and the same color value range. Vectorizing these N second-filter information images yields N vectors, denoted as V1, V2, ..., V... N Based on V1, V2, ..., V N Calculate the mean vector Vmean. Reconstruct the mean vector into a 3D image to obtain the first filter information image.

[0070] In another scenario, the second filter information image includes multiple grid points, each storing a set of input colors and their corresponding output colors. These grid points constitute a complete color mapping rule. Vectorizing N second filter information images yields N vectors, denoted as V1, V2, ..., V... NFor N vectors, calculate the median for each color channel of each grid point, obtaining the median vector for each color channel of each grid point. Reconstruct the median vector for each color channel of each grid point into a 3D image, obtaining the first filter information image.

[0071] In another scenario, the N second filter information images are vectorized to obtain N vectors, represented as V1, V2, ..., V N Perform K-means clustering on N vectors to obtain K clusters. For each vector V within a cluster... i Calculate the mean to obtain the cluster center vector μ. k The cluster center vector μ k The image is then restored to a three-dimensional image format to obtain the first filter information image.

[0072] In another scenario, the N second filter information images are vectorized to obtain N vectors, represented as V1, V2, ..., V N Calculate the mean vector based on N vectors. Then, calculate the mean vector of vector v. i Data-driven processing is performed to obtain centralized processing results. Data-driven processing can be computational v i The difference from the mean vector. Based on the centering process. Calculate the covariance matrix. Perform eigenvalue decomposition on the covariance matrix to obtain the principal component directions u1, u2, ..., sorted by variance from largest to smallest. Based on multiple preset principal component retention ratios, different low-dimensional representations can be obtained. For example, selecting d principal components from principal component directions u1, u2, ... based on the principal component retention ratio yields a low-dimensional representation z. i .

[0073] For the low-dimensional representation z i The vectorized result is obtained through low-rank reconstruction, and the vectorized result is restored to a three-dimensional image form to obtain the first filter information image.

[0074] S750. Apply style transfer processing to the second image using multiple first filter information images to obtain multiple third images, and display the multiple third images and multiple first filter information images.

[0075] The above method determines the set of images to be transferred based on the similarity between the first image set and the first image. Multiple second filter information images are obtained based on the first image and a fourth image in the set of images to be transferred. These multiple second filter information images are then fused using various fusion algorithms to obtain multiple first filter information images. This reduces the content-specific bias of non-color mapping rules in the filter information images, thus achieving stability of the filter effect. A preview of the multiple first filter information images is displayed using multiple third images, enabling real-time preview of the filter effect.

[0076] Figure 8 This is a schematic diagram of a filter generation device in one scenario, such as... Figure 8 As shown, the device includes: The receiving module 810 is used to receive the selection of a first image and a second image, wherein the first image is configured to present a preview effect; The display module 820 is used to display multiple third images and multiple first filter information images, wherein the first filter information images are configured to display the color mapping rules of the second images, and the third images are the effect images after applying the color mapping rules to the second images.

[0077] The aforementioned device receives selections of a first image and a second image. The first image is configured to present a preview effect, displaying multiple third images and multiple first filter information images. The first filter information images are configured to display the color mapping rules of the second image, and the third images are the effect images after applying the color mapping rules to the second image. This method inputs a first image and a second image, outputs multiple first filter information images presenting color mapping rules, and multiple effect images obtained by applying the color mapping rules to the second image. This enables real-time preview of the filter effect, facilitating the selection of the first filter information image that meets expectations based on the effect images. The selected first filter information image can be applied to other user images, achieving intelligent and highly reusable filter generation, shortening the filter generation cycle, and solving the problems of long production cycles and low efficiency in manual filter creation, thus improving filter generation efficiency.

[0078] In one scenario, the first filter information image is a color lookup table image.

[0079] In one scenario, the display module 820 is specifically used for: In response to the triggering operation of the first control, the multiple third images are displayed in the first area of ​​the first page; The first page displays the multiple first filter information images in a second area, wherein the multiple first filter information images are arranged in a matrix.

[0080] In one case, it also includes: Multiple third images corresponding to the same second image are presented in a stitched form, and the stitched images corresponding to different second images are presented in a matrix arrangement.

[0081] In one case, it also includes: In response to an image triggering operation within the second region, a magnified view of the triggered image and a zoomed-out view of the untriggered image are displayed in the second region.

[0082] In one scenario, the receiving module 810 is specifically used for: In response to an image selection operation, at least one of the first image and at least one of the second image are displayed on the first page.

[0083] In one scenario, at least two of the first images differ in at least one of their color features and lighting features.

[0084] In one scenario, displaying at least one of the first images and at least one of the second images on the first page in response to an image selection operation includes: Display a first page, which includes a third area and a fourth area; In response to an image selection operation, at least one of the first images is displayed in the third region, and at least one of the second images is displayed in the fourth region.

[0085] In one case, it also includes: Display a first identifier, which is used to represent a first image set, which represents a set of images in the selected state; In response to a triggering of the first identifier, a set of candidate identifiers is displayed, the set of candidate identifiers including the first identifier, the second identifier, and the third identifier, wherein the second identifier is used to represent a second image set, the second image set represents a set of images in a selectable state, and the third identifier is used to trigger the uploading of images that conform to the application scenario of the filter to be generated to create a third image set, and at least some of the images in different image sets are different.

[0086] In one case, it also includes: The image collection includes multiple types of images, including at least one of the shooting scene type and the content subject type.

[0087] In one scenario, the first filter information image is the result of fusing multiple second filter information images, wherein the second filter information images include color mapping rules used to transfer style features of the first image to a fourth image, and the fourth image belongs to the first image set.

[0088] In one scenario, the fourth image meets the similarity requirement with the first image, wherein the similarity includes at least one of color similarity and content semantic similarity.

[0089] In one case, it also includes: A download control is displayed, wherein the download control is used to download at least one of the third image and the first filter information image.

[0090] The above-described filter generation apparatus can execute the filter generation method provided in any of the embodiments described herein, and has the corresponding functional modules and beneficial effects for executing the filter generation method.

[0091] It is worth noting that the various units and modules included in the above-mentioned filter generation device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments in this article.

[0092] The following is for reference. Figure 9 This document illustrates a schematic diagram of an electronic device (e.g., a terminal device or server) 900 suitable for implementing the above-described methods. The terminal device referred to herein may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital televisions and desktop computers. Figure 9 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments described herein.

[0093] like Figure 9 As shown, the electronic device 900 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the electronic device 900. The processing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0094] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic device 900 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 9 An electronic device 900 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0095] In particular, according to embodiments of this document, the processes described in the above-referenced flowcharts can be implemented as computer software programs. For example, the technical solutions of this document include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by a processing device 901, it performs the functions defined in the methods of the embodiments of this document.

[0096] The names of messages or information exchanged between multiple devices in this document are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0097] The electronic device provided in this embodiment and the filter generation method provided in the above technical solutions belong to the same inventive concept. Technical details not described in detail in this document can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0098] This article provides a computer storage medium storing a computer program that, when executed by a processor, implements the filter generation method provided in the above embodiments.

[0099] It should be noted that the computer-readable medium mentioned above can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM, also known as flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this document, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0100] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include Local Area Networks (LANs), Wide Area Networks (WANs), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0101] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0102] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: Receive a selection of a first image and a second image, wherein the first image is configured to present a preview effect; The system displays multiple third images and multiple first filter information images, wherein the first filter information images are configured to display the color mapping rules of the second image, and the third images are the effect images after applying the color mapping rules to the second image.

[0103] Computer program code for performing the operations described herein can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including local area networks (LANs) or wide area networks (WANs), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0104] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this document. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0105] The modules or units described herein can be implemented in software or hardware. The names of modules or units do not necessarily constitute a limitation on the module or unit itself.

[0106] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include at least one of the following: Field-Programmable Gate Array (FPGA), Application-Specific Integrated Circuit (ASIC), Application-Specific Standard Product (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), etc.

[0107] In the context of this document, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (flash memory), optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0108] The above description is merely a preferred embodiment and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure herein is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed herein that have similar functions.

[0109] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of this document. Certain features described in the context of individual implementations may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0110] Although the subject matter has been described using a programming language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A filter generation method, comprising: Receive a selection of a first image and a second image, wherein the first image is configured to present a preview effect; The system displays multiple third images and multiple first filter information images, wherein the first filter information images are configured to display the color mapping rules of the second image, and the third images are the effect images after applying the color mapping rules to the second image.

2. The method according to claim 1, wherein the first filter information image is a color lookup table image.

3. The method according to claim 1, wherein displaying multiple third images and multiple first filter information images includes: In response to a trigger operation on the first control, the plurality of third images are displayed in a first area of ​​the first page; The first page displays the multiple first filter information images in a second area, wherein the multiple first filter information images are arranged in a matrix.

4. The method according to claim 3, further comprising: Multiple third images corresponding to the same second image are presented in a stitched form, and the stitched images corresponding to different second images are presented in a matrix arrangement.

5. The method according to claim 3, further comprising: In response to an image triggering operation within the second region, a magnified view of the triggered image and a zoomed-out view of the untriggered image are displayed in the second region.

6. The method of claim 1, wherein receiving the selection of the first image and the second image includes In response to an image selection operation, at least one of the first image and at least one of the second image are displayed on the first page.

7. The method according to claim 6, wherein at least one of the color features and lighting features of at least two of the first images is different.

8. The method of claim 6, wherein displaying at least one of the first images and at least one of the second images on the first page in response to an image selection operation comprises: Display a first page, which includes a third area and a fourth area; In response to an image selection operation, at least one of the first images is displayed in the third region, and at least one of the second images is displayed in the fourth region.

9. The method according to claim 1, further comprising: Display a first identifier, which is used to represent a first image set, which represents a set of images in the selected state; In response to a triggering of the first identifier, a set of candidate identifiers is displayed, the set of candidate identifiers including the first identifier, the second identifier, and the third identifier, wherein the second identifier is used to represent a second image set, the second image set represents a set of images in a selectable state, and the third identifier is used to trigger the uploading of images that conform to the application scenario of the filter to be generated to create a third image set, and at least some of the images in different image sets are different.

10. The method of claim 9, further comprising: The image collection includes multiple types of images, including at least one of the shooting scene type and the content subject type.

11. The method according to claim 9, wherein the first filter information image is a fusion result of multiple second filter information images, wherein, The second filter information image includes the color mapping rules used to transfer the style features of the first image to the fourth image, which belongs to the first image set.

12. The method according to claim 11, wherein the similarity between the fourth image and the first image meets the requirements, wherein, The similarity includes at least one of color similarity and content semantic similarity.

13. The method according to claim 1, further comprising: A download control is displayed, wherein the download control is used to download at least one of the third image and the first filter information image.

14. A filter generating apparatus, comprising: A receiving module is used to receive a selection of a first image and a second image, wherein the first image is configured to present a preview effect; The display module is used to display multiple third images and multiple first filter information images, wherein the first filter information images are configured to display the color mapping rules of the second images, and the third images are the effect images after applying the color mapping rules to the second images.

15. An electronic device, the electronic device comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the filter generation method as described in any one of claims 1-13.

16. A storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to perform the filter generation method as described in any one of claims 1-13.

17. A computer program product comprising a computer program that, when executed by a processor, implements the filter generation method as described in any one of claims 1-13.

Images