Training-free content-adaptive style transfer apparatus and method using vector quantization model

Abstract

A computer program stored on a computer-readable storage medium, according to one embodiment of the present disclosure, is disclosed. When executed by a computer device, the computer program causes the following method for style transfer to be performed, and the method may comprise the steps of: using a pre-trained image encoder to acquire, respectively, a content feature from a content image and a style feature from a style image; clustering queries based on the content feature into a plurality of groups, and performing an attention operation between the clustered queries and keys and values based on the style feature so as to generate a primary style-transferred feature in which style is applied for each content area; using an adaptive weight calculated on the basis of channel-wise differences between the primary style-transferred feature and the style feature so as to improve the style of the primary style-transferred feature, thereby generating a secondary style-transferred feature; interpolating the content feature into the secondary style-transferred feature, and aligning a pixel-unit distribution of the secondary style-transferred feature with a pixel-unit distribution of the content feature so as to generate a final style-transferred feature in which content information is preserved; and inputting the final style-transferred feature into a pre-trained decoding block so as to generate a final style-transferred image.

Description

Non-learning type content adaptive style switching device and method using a vector quantization model

[0001] The present invention relates to image style transfer technology, and more specifically, to an apparatus and method for applying an arbitrary image style without a separate learning process using a vector quantization model.

[0002] Image style conversion technology is a field of computer vision that applies visual styles, such as the art style or color tone of a style image, while maintaining the core structure of the content image.

[0003] Early style conversion techniques utilized Convolutional Neural Networks (CNNs) to extract content and style features and generated resulting images through an iterative optimization process. However, this method had the disadvantage of being time-consuming. Subsequently, techniques such as Adaptive Instance Normalization (AdaIN) emerged, enabling real-time style conversion, but they had limitations in applying complex styles.

[0004] Recently, research utilizing generative models, such as Diffusion Models, for style conversion has been actively underway. A representative example is a method that injects styles by modifying the computational process of the self-attention layer of a pre-trained large-scale diffusion model. This technique has the advantage of being able to generate high-quality results.

[0005] However, conventional technologies based on such diffusion models have the following problems.

[0006] First, since the model is very large and includes a vast number of computational parameters, it requires a lot of memory and high computing performance.

[0007] Second, there is a problem that it takes a relatively long time to process a single image because the diffusion and generation processes of adding and removing noise are repeated during style conversion.

[0008] Third, some other vector quantization-based approaches also had the disadvantage of being cumbersome and time-consuming, as they required additional fine-tuning using datasets of specific domains or separate model training.

[0009] Republic of Korea Registered Patent: No. 10-2592348

[0010] The present disclosure is conceived based on the aforementioned background technology and is intended to overcome the limitations of high computational costs and slow inference times inherent in diffusion model-based technologies.

[0011] The present invention aims to provide a training-free style conversion technology that is universally applicable to any style by utilizing a pre-trained vector quantization model as is, without a separate model training or fine-tuning process.

[0012] The present invention aims to provide a technology that reflects the artistic characteristics of style images with high fidelity while preserving the structural information of the content to the maximum extent by proposing new modules that effectively manipulate features within the decoder of a vector quantization model.

[0013] According to one embodiment of the present disclosure for realizing the aforementioned objectives, a computer program stored on a computer-readable storage medium is disclosed. When the computer program is executed on a computer device, it performs the following methods for style conversion, wherein the method comprises: a step of obtaining content features from a content image and style features from a style image, respectively, using a pre-trained image encoder; a step of clustering a query based on the content features into a plurality of groups and performing an attention operation between the clustered query and a key and value based on the style features to generate a first style conversion feature with applied style for each content region; and a step of generating a second style conversion feature by improving the style of the first style conversion feature using an adaptive weight calculated based on the channel-wise difference between the first style conversion feature and the style feature. The method may include: a step of interpolating the content feature to the second style transition feature and aligning the pixel unit distribution of the second style transition feature with the pixel unit distribution of the content feature to generate a final style transition feature in which content information is preserved; and a step of inputting the final style transition feature into a pre-trained decoding block to generate a final style transition image.

[0014] Alternatively, the pre-trained image encoder is a Vector Quantization Encoder, and the step of acquiring the content features and the style features may include the step of acquiring the content features and the style features by quantizing a raw feature map extracted through the Vector Quantization Encoder through a vector-quantization bottleneck.

[0015] Alternatively, the step of generating the primary style transition feature may include the step of calculating a cluster centroid for each of a plurality of groups obtained by clustering the queries of the content feature, and clustering each query of the content feature by assigning it to the group corresponding to the cluster centroid closest in distance.

[0016] Alternatively, the step of generating the first style transition feature may further include the step of generating stylized clustered features through cross-attention operations between the calculated cluster centroid and the key and value generated based on the style feature.

[0017] Alternatively, the step of generating the primary style transition feature may further include the step of generating the primary style transition feature by mapping the group-specific style transition feature corresponding to the group to which each query of the content feature belongs to the image coordinates of each query.

[0018] Alternatively, the number of the above plurality of groups may be 22.

[0019] Alternatively, in the step of generating the second style transition feature, the adaptive weight can be calculated by averaging the channel-wise differences between the first style transition feature and the style feature over a spatial dimension and applying min-max scaling to the average value.

[0020] Alternatively, the step of generating the second style transition feature may include: for the first style transition feature, performing an Adaptive Instance Normalization (AdaIN) operation using channel-specific statistics of the style feature, and generating the second style transition feature by multiplying the result of the operation by the calculated adaptive weight.

[0021] Alternatively, the step of generating the final style transition feature may include linear interpolating the secondary style transition feature and the content feature using a predetermined interpolation weight, and passing the linearly interpolated feature through a residual block.

[0022] Alternatively, the above interpolation weight may be 0.87.

[0023] Alternatively, the step of generating the final style transition feature may include the step of adjusting and aligning the pixel-wise average and standard deviation of the secondary style transition feature to match the pixel-wise average and standard deviation of the content feature.

[0024] Alternatively, the first style transition feature generation, second style transition feature, interpolation, and alignment steps can be performed sequentially in blocks included in the decoder.

[0025] Alternatively, prior to the step of generating the first style transition feature, the method further includes the step of generating an initial style transition feature through an adaptive instance normalization (AdaIN) operation that matches the channel-specific statistics (mean, standard deviation) of the content feature with the channel-specific statistics of the style feature; and the step of generating the first and second style transition features can be performed based on the initial style transition feature.

[0026] Alternatively, a computer device for performing methods for style switching is disclosed. The computer device comprises one or more processors; and memory storing instructions executable on the one or more processors; wherein the one or more processors respectively acquire content features from a content image and style features from a style image using a pre-trained image encoder, cluster queries based on the content features into multiple groups, perform attention operations between the clustered queries and key and value based on the style features to generate first style transition features with applied styles for each content region, generate second style transition features by improving the style of the first style transition features using adaptive weights calculated based on the channel-wise difference between the first style transition features and the style features, interpolate the content features to the second style transition features, and align the pixel-unit distribution of the second style transition features with the pixel-unit distribution of the content features to generate final style transition features in which content information is preserved, and, the final style transition features are pre-trained decoding You can generate the final style transition image by inputting into the block.

[0027] Alternatively, a method for style switching executed on a computer device is disclosed. The method comprises the steps of: acquiring content features from a content image and style features from a style image, respectively, using a pre-trained image encoder; clustering a query based on the content features into a plurality of groups and performing an attention operation between the clustered query and a key and value based on the style features to generate a first style switching feature with style applied for each content region; generating a second style switching feature by improving the style of the first style switching feature using an adaptive weight calculated based on the channel-wise difference between the first style switching feature and the style features; interpolating the content features onto the second style switching feature and aligning the pixel-unit distribution of the second style switching feature with the pixel-unit distribution of the content features to generate a final style switching feature in which content information is preserved. and may include the step of generating a final style transition image by inputting the final style transition feature into a pre-trained decoding block.

[0028] The present disclosure can overcome the limitations of high computational costs and slow inference times associated with diffusion model-based technologies.

[0029] The present invention can provide a training-free style switching technology that is universally applicable to any style by utilizing a pre-trained vector quantization model as is, without a separate model training or fine-tuning process.

[0030] The present invention proposes new modules that effectively manipulate features within the decoder of a vector quantization model, thereby providing a technology that reflects the artistic characteristics of style images with high fidelity while preserving the structural information of the content to the maximum extent.

[0031] FIG. 1 is a drawing showing the overall structure of a style switching system according to one embodiment of the present invention.

[0032] FIG. 2 illustrates a brief and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

[0033] Various embodiments are now described with reference to the drawings. In this specification, various descriptions are provided to provide an understanding of the present disclosure. However, it is evident that these embodiments can be practiced without such specific descriptions.

[0034] As used herein, terms such as “component,” “module,” “system,” etc. refer to computer-related entities, hardware, firmware, software, combinations of software and hardware, or executions of software. For example, a component may be, but is not limited to, a procedure executed on a processor, a processor, an object, an execution thread, a program, and / or a computer. For example, both an application executed on a computer device and the computer device itself may be a component. One or more components may reside within a processor and / or an execution thread. A component may be localized within a single computer. A component may be distributed among two or more computers. Additionally, these components may be executed from various computer-readable media having various data structures stored therein. Components may communicate through local and / or remote processes, for example, according to signals having one or more data packets (e.g., data from a component interacting with another component in a local system or distributed system, and / or data transmitted through signals to other systems and networks such as the Internet).

[0035] Furthermore, the term "or" is intended to mean an implicit "or" rather than an exclusive "or." That is, unless otherwise specified or evident from the context, "X uses A or B" is intended to mean one of the natural implicit substitutions. In other words, if X uses A; if X uses B; or if X uses both A and B, "X uses A or B" may apply to any of these cases. Additionally, the term "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the enumerated related items.

[0036] Additionally, the terms “comprising” and / or “comprising” should be understood to mean that such features and / or components are present. However, the terms “comprising” and / or “comprising” should be understood not to exclude the presence or addition of one or more other features, components and / or groups thereof. Furthermore, unless otherwise specified or clearly evident from the context to indicate a singular form, the singular in this specification and claims should generally be interpreted to mean “one or more.”

[0037] And, the term “at least one of A or B” should be interpreted to mean “a case including only A,” “a case including only B,” or “a combination of A and B.”

[0038] Those skilled in the art should recognize that the various exemplary logical blocks, configurations, modules, circuits, means, logics, and algorithmic steps described in connection with the embodiments disclosed herein may be implemented in electronic hardware, computer software, or a combination of both. To clearly exemplify the interchangeability of hardware and software, various exemplary components, blocks, configurations, means, logics, modules, circuits, and steps have been generally described above in terms of their functionality. Whether such functionality is implemented in hardware or software depends on the specific application and design constraints imposed on the overall system. Skilled technicians may implement the described functionality in various ways for each specific application. However, such decisions regarding implementation should not be construed as going beyond the scope of this disclosure.

[0039] The description of the presented embodiments is provided to enable those skilled in the art to use or practice the present invention. Various modifications to these embodiments will be apparent to those skilled in the art. The general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present invention is not limited to the embodiments presented herein. The present invention should be interpreted in the broadest possible scope consistent with the principles and novel features presented herein.

[0040]

[0041] FIG. 1 is a drawing showing the overall structure of a style switching system according to one embodiment of the present invention.

[0042] Referring to FIG. 1, a style conversion system according to one embodiment of the present invention largely comprises an image encoder (100, 101) and a decoder (200). The image encoder (100, 101) performs the role of extracting features from an input image, and the decoder (200) receives the extracted and converted features and performs the role of generating a final result image.

[0043] More specifically, the user inputs a content image (10), which is the original image to which a style is to be applied, and a style image (20) containing the art style or color tone to be applied into the system. The content image (10) is the target image to which a style is to be applied, and in the present invention, it provides the overall structure and object information of the image to be finally generated. The style image (20) is an image that provides an artistic style, such as the art style, color tone, or texture to be applied.

[0044] The image encoder (100, 101) is a pre-trained model, and in this embodiment, a vector quantization encoder (VQ Encoder) is used. The image encoder (100) receives a content image (10) as input and extracts content features (11), and the image encoder (101) receives a style image (20) as input and extracts style features (21). Although the image encoder (100, 101) is shown as separate blocks in the drawing, it may be a single encoder that shares the same parameters. That is, the same encoder sequentially processes the content image (10) and the style image (20) to extract features. The extracted features are converted into quantized initial content features (121) and quantized initial style features (122) through a quantization process that refers to the codebook (110).

[0045] The decoder (200) is a part where the core style conversion operation of the present invention is performed, and it plays the role of receiving a feature map as input and generating a final style conversion image (22). Unlike a standard decoder, the decoder (200) of this embodiment has a special structure that includes a first decoding block (210) and a second decoding block (220) internally. The first decoding block (210) and the second decoding block (220) each include core components of the present invention, such as a Content Consistency Style Injection (CSI) module (300), an Adaptive Style Improvement (ASR) module (400), and a Content Distribution Alignment (CDA) module (500), and perform the role of sequentially improving and refining the input features. The detailed internal operation of the decoder (200) will be described later.

[0046] A codebook (110) is a set of pre-learned discrete feature vectors used in a vector quantization process. Continuous feature vectors extracted through an image encoder (100, 101) undergo a quantization process in which they are mapped to the feature vector with the shortest Euclidean distance from themselves within the codebook (110). In one embodiment of the present invention, the codebook (110) may include 4,096 feature vectors.

[0047] The quantized initial content features (121) are discrete feature maps generated when the content image (10) undergoes an image encoder (100) and a quantization process. This represents the unique structure and object information of the content image (10) in a latent space and serves as the basis for applying styles thereafter.

[0048] The quantized initial style feature (122) is a discrete feature map generated when the style image (20) undergoes an image encoder (101) and a quantization process. This contains statistical information regarding the style, color tone, texture, etc. of the style image (20) and is used as a source for injecting the style.

[0049] / *48 The quantized initial style application feature (120) is a feature map generated by applying a style in a preliminary manner before entering the core feature improvement step of the present invention. This is generated by combining the quantized initial style feature (122) with the quantized initial content feature (121). Specifically, it is generated by matching the channel-wise average and standard deviation of the initial content feature (121) with the channel-wise average and standard deviation of the initial style feature (122) through an adaptive instance normalization (AdaIN) operation. The initial style application feature (120) generated in this way is used as an initial input value for the first decoding block (210) of the decoder (200).

[0050] The first decoding block (210) is a unit block that performs core style conversion and refinement operations of the decoder (200). In one embodiment of the present invention, the decoder (200) includes N (e.g., 3) of these first decoding blocks (210), and the style conversion feature is repeatedly improved as it passes through these blocks sequentially.

[0051] The first decoding block (210) receives a style transition feature (31) (or, in the case of the first block, a quantized initial style application feature (120)) from the previous block. Additionally, the original content feature (11) and the style feature (21) are referenced together for computation.

[0052] The internal operation of the first decoding block (210) is largely divided into two parallel paths.

[0053] The first is a style injection and enhancement path, which is performed within a Self Attention Block. In this path, the Content Consistency Style Injection (CSI) module (300), which will be described later, operates first to generate a first style transition feature (32) using content features (11) and style features (21). Subsequently, the Adaptive Style Enhancement (ASR) module (400) receives this first style transition feature (32) and enhances the finer details of the style to generate a second style transition feature (33).

[0054] The second is a content information preservation path, which is performed within a residual block. In this path, a residual feature interpolation (RFI) module (600) operates. The RFI module (600) linearly interpolates the style conversion feature (31) and the original content feature (11) input into the first decoding block (210) to obtain an interpolated feature (with enhanced content information) Creates ).

[0055] Finally, the results of the two parallel paths are combined into one through element-wise addition. That is, the secondary style conversion feature (33), which is the final result of the style injection and improvement path, and the interpolated feature, which is the result of the content information preservation path ( ) is added.

[0056] Through this operation, an intermediate style transition feature (34) is generated in which the style is more refined and the content is preserved more clearly. This feature becomes the final output of the first decoding block (210), is passed as the input to the next first decoding block, or is passed to the second decoding block (220) after passing through all N blocks.

[0057] The Content Consistency Style Injection (CSI) module (300) is located within the first decoding block (210) of the decoder (200) and performs the role of effectively injecting the style of the style image (20) while maintaining the structural consistency of the content image (10). This is intended to reduce distortion and produce a natural result by identifying each area of ​​the content and applying a style suitable for that area, rather than simply applying the same style to the entire image. The Content Consistency Style Injection module (300) internally includes a clustering module (310) and an attention module (320).

[0058] The operation process of the content consistency style injection module (300) is as follows.

[0059] First, the clustering module (310) receives content features (11) transmitted from the image encoder (100). The clustering module (310) considers feature vectors corresponding to each location (pixel) of the content features (11) as queries, and analyzes the distribution of these queries to cluster them into multiple groups. For example, queries for pixels corresponding to the sky within the content image may be grouped into one group, and queries for pixels corresponding to trees may be grouped into another group. In one embodiment of the present invention, the number of these groups may be 22. Once clustering is complete, the average of the queries belonging to each group is calculated to determine the cluster centers (Cluster centers, ) is produced. Through this process, the content feature (11), which is a sequence of consecutive pixels, is summarized into several semantically distinct representative regions.

[0060] Next, the attention module (320) performs a cross-attention operation for style injection. At this time, the cluster centers calculated by the clustering module (310) are used as the query for the attention operation. The key and value are generated from the style features (21). That is, the association between each representative area (cluster center) of the content image and the entire style features (21) is calculated to determine which style to apply to each representative area. By performing attention based on semantically grouped area units rather than individual pixels in this way, a consistent style is applied within the same area, and the style is naturally distinguished between areas.

[0061] Finally, the 'group-specific style transition feature' calculated for each cluster center through the attention module (320) is mapped back to the overall coordinates of the image. That is, the style transition feature value assigned to that group is filled identically into the locations of all queries (pixels) that belonged to a specific group during the clustering phase.

[0062] The feature map that is finally output after going through all these processes is the primary style transition feature (32). This feature is a state in which a consistent style is applied to each area while maintaining the structural characteristics of the content.

[0063] The Adaptive Style Improvement (ASR) module (400) serves to reinforce the detailed expression of the style and fine-tune the overall style application intensity for the primary style transition feature (32) that has passed through the Content Consistency Style Injection (CSI) module (300). In the process of the Content Consistency Style Injection module (300) applying a consistent style to each content area, some fine style information may be averaged or lost, but the Adaptive Style Improvement module (400) compensates for this to improve the completeness of the result.

[0064] The operation process of the adaptive style improvement module (400) is largely an adaptive weight ( It can be divided into a step of producing ) and a step of refining the style using it.

[0065] First, to calculate adaptive weights, the adaptive style enhancement module (400) compares the input primary style conversion feature (32) and the original style feature (21) by channel. It calculates the difference between the two features in each channel and averages this difference value over the spatial dimensions (horizontal, vertical) of the image. This serves as a measure indicating how much style information is lacking in each channel. Channels with insufficient style applied will have large values, while channels with sufficient style applied will have small values. Subsequently, min-max scaling is applied to this average value to normalize the adaptive weight (which is normalized to a value between 0 and 1). It finally produces ).

[0066] Next, the style is refined using the calculated adaptive weights. This process is performed in the Adaptive Instance Normalization (AdaIN) operation module (410). The operation module (410) re-injects the style by applying the AdaIN operation to the primary style transition feature (32). Specifically, the statistics of the primary style transition feature (32) are matched with the statistics of the style feature (21). As a result of this operation, the previously calculated adaptive weights ( Multiply ) by channel.

[0067] Through this weight-based operation, channels deemed lacking in style are subjected to strong style reinforcement, while channels already sufficient are maintained or weakly reinforced. By selectively adding information only to the parts requiring style in this way, detailed style expression can be restored and improved without compromising the overall content structure.

[0068] The final output of the adaptive style improvement module (400) is a secondary style transition feature (33) with a refined style applied, which is passed as input to the next stage.

[0069] The Residual Feature Interpolation (RFI) module (600) is performed in the Residual Block within the first decoding block (210) and serves to enhance preservation by reinforcing information of the original content that may be diluted during the style injection process (CSI, ASR). This helps in the stable restoration of the image by utilizing the skip-connection structure of the decoder to progressively integrate the features to which the style is applied with the features of the original content.

[0070] The operation process of the RFI module (600) is as follows.

[0071] First, the RFI module (600) receives style conversion features (31) transmitted from the previous decoding block and original content features (11) transmitted from the image encoder (100). Then, the two feature maps are given a predetermined interpolation weight ( Linear interpolation is performed using ). This interpolation weight is a hyperparameter that determines which information between styled features and original content features is reflected more, and in one embodiment of the present invention, it can be set to 0.87.

[0072] Features combined through linear interpolation pass through a residual block containing convolution operations to finally interpolate features ( ) is generated.

[0073] The interpolated features generated in this way ( ) is a secondary style transition feature (33, which has gone through the adaptive style improvement module (400), It is combined through ) and element-wise addition. Through this process, the features with finely applied styles and the features with reinforced structure of the original content are combined, resulting in an intermediate style transition feature (34, which preserves both style and content information well. ) is generated. This feature becomes the final output of the first decoding block (210) and is passed to the next second decoding block (220).

[0074] The Content Distribution Alignment (CDA) module (500) is located in the second decoding block (220) of the decoder (200) and serves to finally preserve and enhance content information for intermediate style transition features (34) to which styles have been applied while passing through the previous steps (CSI, ASR). The main purpose is to correct distortions in the fine structure or texture of the content that may occur during the style injection process, thereby maintaining the clarity and unique texture of the original content.

[0075] The content distribution alignment module (500) operates through a pixel-wise alignment method. Unlike the AdaIN method, which adjusted statistics based on channel units, this is a method that aligns statistics based on each pixel position.

[0076] The specific operation process is as follows.

[0077] First, the content distribution alignment module (500) calculates the mean and standard deviation of feature values ​​across all channels for each pixel location of the input intermediate style transition feature (34). Likewise, it calculates the mean and standard deviation across channels for each pixel location of the original content feature (11).

[0078] Next, each pixel vector of the intermediate style transition feature (34) is normalized using its pixel-unit average and standard deviation. Then, the normalized pixel vector is scaled and shifted using the pixel-unit average and standard deviation of the original content feature (11) located at the same position as the corresponding pixel.

[0079] Through this pixel-level alignment process, the statistical distribution of each pixel in the styled feature map is forced to resemble the distribution of the corresponding pixel in the original content's feature map. This has the effect of maintaining the relationships between adjacent pixels and the gradients of their values ​​similar to the original, enabling the effective preservation of the content's detailed texture and structural integrity even during the styling process.

[0080] The feature map output through the content distribution alignment module (500) becomes the final style transition feature (35), which is then passed to the decoding block for generating the final image in a state where the style and content are combined in an optimal balance.

[0081] The final style transition feature (35) is a feature map in which all style injection and content preservation processes are completed, and is input into a pre-trained decoding block to be restored to the final image.

[0082] Here, the decoding block may refer to the standard decoder structure portion of the original pre-trained model, excluding the core feature transformation modules (CSI, ASR, CDA, etc.) of the present invention. This decoding block receives a low-resolution feature map as input and progressively increases spatial resolution and reduces channel dimensions through a series of upsampling and convolution operations.

[0083] Through this image restoration process, the final style transition feature (35) that was expressed in the latent space is converted into a high-resolution RGB image, i.e., a final style transition image (22), which is finally shown to the user. This final style transition image (22) is the result of optimally combining the structure of the original content image (10) and the art style of the original style image (20).

[0084] Preferred embodiments of the present invention will be described in detail below with reference to mathematical formulas.

[0085] The present invention operates based on a pre-trained vector quantization (VQ) autoencoder model. A typical VQ autoencoder consists of an encoder (E), a quantization bottleneck (Q), and a decoder (D) as shown in Equation 1 below.

[0086]

[0087]

[0088] Here, the image (I) is converted into a raw feature map (f) through an encoder (E), and this feature map is converted into a quantized feature map (f) through a quantization bottleneck (Q), and then restored back to the image (I′) by a decoder (D).

[0089] The quantization process is as shown in Equation 2 below, for each vector of the raw feature map ( ) is the closest vector in the codebook (z) It is done by replacing with ).

[0090]

[0091]

[0092] The present invention performs style switching by replacing the decoder (D) in the above structure with a decoder (D*) of a new structure, which can be expressed as Equation 3 below.

[0093]

[0094] Here is the final style transition image (22) Each represents a quantized initial content feature (121), an initial style feature (122), and an initial style application feature (120).

[0095] Apply initial style

[0096] Initial style application features ( ) is generated through the Adaptive Instance Normalization (AdaIN) operation as shown in Equation 4 below.

[0097]

[0098] This is an early content feature ( Channel-specific statistics of ) , ) initial style characteristics( ) statistics( , It is the process of aligning with ).

[0099] Content Consistency Style Injection (CSI) Module (300)

[0100] The CSI module uses content features as shown in Equation 5 below for attention calculations ( Query from ) ) style features( Key from ) ) and Value( Creates ).

[0101]

[0102] Afterwards, query through the following mathematical formula 6 ( Cluster the ) into K clusters and the center of each cluster ( Calculate )

[0103]

[0104] Calculated cluster center ( Using ) as a query, perform cross-attention as shown in Equation 7 below, and group-specific style transition features( Creates ).

[0105]

[0106] Finally, based on the logic of mathematical formula 8, the group-specific style transition features corresponding to each pixel location are mapped to the primary style transition features ( Complete )

[0107]

[0108] Adaptive Style Improvement (ASR) Module (400)

[0109] The ASR module has a first-order style transition feature through the following mathematical formula 9 ( ) and original style features( Adaptive weights based on the channel-specific differences of ) Produces ).

[0110]

[0111] Calculated weights ( ) is multiplied by an operation result similar to AdaIN as in mathematical formula 10, and finally the secondary style transition feature ( Creates ).

[0112]

[0113] Content Improvement Module (RFI, CDA)

[0114] Residual Feature Interpolation (RFI) is style transition features as shown in Equation 11 ( ) and original content characteristics( ) is interpolated using interpolation weights (α).

[0115]

[0116] RFI results ( ) is the result of ASR( Combined with ) and mathematical formula 12, the intermediate style transition feature( forms ).

[0117]

[0118] Content Distribution Alignment (CDA) is an intermediate style transition feature as shown in Equation 13 ( Pixel-unit statistical distribution of ) original content features( Matching with the distribution of ) to final style conversion features ( Creates ).

[0119]

[0120]

[0121] The main components of the present invention, their combination relationships, and operating principles have been explained through the aforementioned Fig. 1. Below, the components of the present invention will be explained in detail once again based on the contents of each claim.

[0122] The style conversion method according to the present invention first includes the step of obtaining content features (11) from a content image (10) and style features (21) from a style image (20), respectively, using a pre-trained image encoder (100, 101).

[0123] Specifically, the pre-trained image encoder may be a Vector Quantization Encoder. In this case, the process of acquiring features consists of a step in which the Vector Quantization Encoder processes the image to extract a raw feature map, and a step in which quantization is performed on the extracted raw feature map.

[0124] First, a vector quantization encoder (100, 101) processes each input image (10, 20) to extract a raw feature map in a continuous space. Then, the extracted raw feature map is passed to a vector-quantization bottleneck. The vector-quantization bottleneck performs quantization by replacing each feature vector of the raw feature map with the most similar (e.g., closest in Euclidean distance) code vector within a predefined codebook (110).

[0125] Through this quantization process, quantized content features (121) and style features (122) having discrete values ​​are finally obtained.

[0126]

[0127] The present invention further includes a step of generating initial style transition features (120) prior to performing a full-scale feature improvement step. This step is a process of combining acquired content features (121) and style features (122) to create a preliminary styled feature map that serves as the basis for a subsequent processing step.

[0128] Specifically, at this stage, an Adaptive Instance Normalization (AdaIN) operation is performed. The AdaIN operation serves to match the channel-specific statistics (mean, standard deviation) of the content feature (121) with the channel-specific statistics of the style feature (122). This is done by normalizing the content feature (121) to its own statistics and then scaling and biasing it again to the statistics of the style feature (122).

[0129] The initial style transition feature (120) generated in this way is in a state where the overall color and texture of the style are applied while maintaining the structure of the content. This initial style transition feature (120) is used as an input value that serves as the basis for sequential feature improvement processes within the decoder (200), such as the subsequent first style transition feature generation step (CSI) and second style transition feature generation step (ASR).

[0130] The present invention clusters a query based on a content feature (11) into a plurality of groups to generate a primary style transition feature (32), and performs an attention operation between the clustered query and a key and value based on a style feature (21). The purpose of this step is to distinguish semantic regions of a content image and to generate a natural result by applying a consistent style within each region.

[0131] The specific process of this stage is as follows.

[0132] First, feature vectors corresponding to each location of the content feature (11) are considered as queries, and these queries are clustered into multiple groups. In one embodiment of the present invention, the number of multiple groups may be 22. The clustering process is performed by first calculating a cluster centroid representing each group, and then assigning each query of the content feature (11) to the group corresponding to the cluster centroid closest in distance. Through this process, regions with similar characteristics within the content image are identified and grouped.

[0133] Second, a cross-attention operation is performed to inject styles into the grouped content areas. In this operation, the previously calculated cluster centers are used as a query, and the key and value generated from the style features (21) are used. By calculating the association between each cluster center (representative of the content area) and the overall style features, stylized clustered features are generated to determine which style to apply to each area.

[0134] Third, the generated group-specific style transition features are extended into an image-wide feature map. That is, the group-specific style transition features corresponding to the group to which each query of the content features (11) belongs are mapped to the original image coordinates of the corresponding query. Through this process, the first style transition features (32) with a consistent style applied to each content area are finally completed.

[0135] The present invention includes a step of generating a second style transition feature (33) to reinforce the detailed expression of the style and fine-tune the overall application intensity for the first style transition feature (32) that has undergone the previous step. This is a process of restoring fine style information that may be partially lost during the process of applying the style to each content area to improve the completeness of the result.

[0136] Specifically, this step consists of the process of calculating adaptive weights and the process of refining styles using the calculated weights.

[0137] First, to calculate adaptive weights, the primary style conversion feature (32) and the original style feature (21) are compared channel by channel. The difference between the two features is calculated for each channel, and this difference value is averaged over the spatial dimension of the image. This average value serves as a measure of how much style information is lacking in each channel. Subsequently, min-max scaling is applied to this average value to obtain adaptive weights normalized to values ​​between 0 and 1 ( It finally produces ).

[0138] Next, the style is refined using the calculated adaptive weights. In this process, an adaptive instance normalization (AdaIN) operation is performed on the primary style transition feature (32) using the channel-specific statistics (mean, standard deviation) of the original style feature (21). Then, the result of the above AdaIN operation is multiplied by the previously calculated adaptive weights for each channel. Through this weight-based operation, strong style reinforcement is performed on channels that are judged to be lacking in style, while channels that are already sufficient are maintained as they are.

[0139] Through this process, a second style transition feature (33) with a refined style is finally generated.

[0140] The present invention includes a step of generating a final style transition feature (35) by preserving and enhancing the structure and texture of the original content for a secondary style transition feature (33) to which a style is applied. This step is a process that minimizes the loss of content information that may occur during the previous style injection process and increases the completeness of the result, and can be largely composed of a content feature interpolation process and a pixel-unit distribution alignment process.

[0141] First, in the content feature interpolation process, the overall content structure of the secondary style transition feature (33) is reinforced. Specifically, the secondary style transition feature (33) and the original content feature (11) are linearly interpolated using a predetermined interpolation weight. This interpolation weight determines which of the two features' information is reflected more, and in one embodiment of the present invention, it may be set to 0.87. The linearly interpolated feature passes through a residual block and is converted into an interpolated feature with reinforced content information. This interpolated feature is combined with the secondary style transition feature (33) to form an intermediate style transition feature (34).

[0142] Next, the pixel-wise distribution alignment process preserves the fine texture and structure of the intermediate style transition feature (34). This process is performed by aligning the pixel-wise average and standard deviation of the intermediate style transition feature (34) to match the pixel-wise average and standard deviation of the original content feature (11). That is, the statistical distribution of each pixel of the style-applied feature map is made similar to the distribution of the corresponding pixel of the original content, thereby preserving the integrity of the local texture.

[0143] Through these two processes, a final style transition feature (35) in which style and content are in exquisite harmony is completed.

[0144] The above-described first style transition feature generation, second style transition feature generation, interpolation, and alignment steps are performed sequentially in a plurality of decoder blocks included in the decoder (200). This structure is intended to generate high-quality results by progressively and iteratively improving the style transition features.

[0145] In one embodiment of the present invention, the decoder blocks may be divided into two groups according to their function.

[0146] The first group consists of first decoding blocks (210), which are primarily responsible for style injection and enhancement, and overall content preservation. This group may consist of N (e.g., 3) blocks. When an initial style application feature (120) is input into the first first decoding block (210), the aforementioned C SI, ASR, and RFI operations are performed within the block to output an intermediate style conversion feature (34). This output value is then passed as input to the next first decoding block (210), and this process is repeated N times.

[0147] The second group consists of second decoding blocks (220) that serve to preserve fine content texture and structure. This group may also consist of N (e.g., 3) blocks. When an intermediate style transition feature (34) that has passed through all of the first decoding block groups is input into the first second decoding block (220), the aforementioned content distribution alignment (CDA) operation is performed within the block. This process is also repeated N times.

[0148] In this way, as multiple decoding blocks are passed sequentially, style and content information is refined, and finally, a final style conversion feature (35) is produced.

[0149] The final style transition feature (35), having undergone all the aforementioned feature improvement processes, is finally restored as an image. To this end, the present invention includes the step of inputting the final style transition feature (35) into a pre-trained decoding block to generate a final style transition image (22).

[0150] Here, the above-mentioned decoding block refers to the standard image restoration structure part of the original pre-trained model, excluding the core feature transformation modules (CSI, ASR, CDA, etc.) of the present invention.

[0151] This decoding block receives the final style conversion feature (35) that was expressed in the latent space and gradually increases the spatial resolution through a series of upsampling and convolution operations. Through this process, the low-resolution feature map is converted into a high-resolution RGB image, i.e., the final style conversion image (22), which is finally displayed to the user. This final style conversion image (22) is the result of optimally combining the structure of the original content image (10) and the art style of the original style image (20).

[0152] The present invention, having the configuration described above, solves various problems of the prior art and provides the following excellent effects.

[0153] First, computational efficiency and inference speed are significantly improved. The present invention operates based on a pre-trained lightweight vector quantization (VQ) model rather than a diffusion model. Therefore, since the diffusion and generation processes requiring numerous iterative operations are omitted, it is possible to operate with much less memory and computational resources compared to conventional generation model-based technologies, and the inference time required to process a single image can be drastically reduced.

[0154] Second, it offers excellent ease of use and versatility. This invention is implemented in a training-free manner, requiring no separate model training or fine-tuning for specific styles. Consequently, users can freely select any content images and style images without any restrictions to immediately obtain the desired results. This provides high versatility that is not dependent on specific datasets or domains.

[0155] Third, it achieves an optimal balance between content preservation and style application. This invention performs high-quality style conversion through the organic combination of independently proposed modules. The 'Content Consistency Style Injection (CSI)' module applies styles while respecting the semantic domain of the content, while the 'Residual Feature Interpolation (RFI)' and 'Content Distribution Alignment (CDA)' modules actively preserve the overall structure and fine textures of the original content. At the same time, the 'Adaptive Style Enhancement (ASR)' module restores and strengthens the fine details of the style that might otherwise be lost. Through this, it is possible to generate natural and aesthetically superior results that deeply reflect the artistic characteristics of the style while minimizing content distortion.

[0156] Fourth, technical excellence has been proven through objective indicators. In quantitative evaluation metrics such as ArtFID, which comprehensively evaluates content retention (LPIPS) and style similarity (FID), the present invention has achieved state-of-the-art performance that surpasses both existing traditional style conversion technologies and the latest generative model-based technologies. This means that the effectiveness of the present invention is not limited to subjective judgment but is also proven by objective data.

[0157] FIG. 2 illustrates a brief and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

[0158] Although the present disclosure has been described as generally being implementable by a computer device, those skilled in the art will understand that the present disclosure may be implemented in combination with computer-executable instructions and / or other program modules that can be executed on one or more computers and / or as a combination of hardware and software.

[0159] Generally, a program module includes routines, programs, components, data structures, etc., that perform a specific task or implement a specific abstract data type. Furthermore, those skilled in the art will be well aware that the method of the present disclosure may be implemented in other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computer devices, microprocessor-based or programmable consumer electronics, etc. (each of which may be connected to and operated with one or more associated devices).

[0160] The embodiments described in the present disclosure may also be implemented in a distributed computing environment in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0161] Computers typically include various computer-readable media. Any medium accessible by a computer may be a computer-readable medium, and such computer-readable media include volatile and non-volatile media, transitory and non-transitory media, and removable and non-removable media. By example, but not by limitation, computer-readable media may include computer-readable storage media and computer-readable transmission media. Computer-readable storage media include volatile and non-volatile media, transitory and non-transitory media, and removable and non-removable media implemented by any method or technique for storing information such as computer-readable commands, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, DVD (digital video disk) or other optical disk storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other media that can be accessed by a computer and used to store desired information.

[0162] Computer-readable transmission media typically include all information transmission media that implement computer-readable instructions, data structures, program modules, or other data, etc., on a modulated data signal, such as a carrier wave or other transport mechanism. The term modulated data signal means a signal in which one or more of the characteristics of the signal are set or modified to encode information within the signal. By example, not limiting, computer-readable transmission media include wired media such as wired networks or direct-wired connections, and wireless media such as acoustic, RF, infrared, and other wireless media. Any combination of the media described above is also included within the scope of computer-readable transmission media.

[0163] An exemplary environment (1100) for implementing various aspects of the present disclosure, including a computer (1102), is shown, wherein the computer (1102) includes a processing unit (1104), system memory (1106), and a system bus (1108). The system bus (1108) connects system components, including system memory (1106) (but not limited thereto), to the processing unit (1104). The processing unit (1104) may be any processor among various commercial processors. Dual processor and other multiprocessor architectures may also be used as the processing unit (1104).

[0164] The system bus (1108) may be any of several types of bus structures that can be additionally interconnected to a local bus using any of the memory bus, peripheral bus, and various commercial bus architectures. System memory (1106) includes read-only memory (ROM) (1110) and random access memory (RAM) (1112). The Basic Input / Output System (BIOS) is stored in non-volatile memory (1110), such as ROM, EP ROM, EEPROM, etc., and this BIOS includes basic routines that help transfer information between components within the computer (1102) at times such as during startup. The RAM (1112) may also include high-speed RAM, such as static RAM, for caching data.

[0165] The computer (1102) also includes an internal hard disk drive (HDD) (1114) (e.g., EIDE, SATA)—this internal hard disk drive (1114) may also be configured for external use within a suitable chassis (not shown)—a magnetic floppy disk drive (FDD) (1116) (e.g., for reading from or writing to a removable diskette (1118)), and an optical disk drive (1120) (e.g., for reading from a CD-ROM disk (1122) or reading from or writing to other high-capacity optical media such as a DVD). The hard disk drive (1114), the magnetic disk drive (1116), and the optical disk drive (1120) may each be connected to the system bus (1108) by a hard disk drive interface (1124), a magnetic disk drive interface (1126), and an optical drive interface (1128). The interface (1124) for implementing an external drive includes at least one or both of the USB (Universal Serial Bus) and IEEE 1394 interface technologies.

[0166] These drives and associated computer-readable media provide non-volatile storage of data, data structures, computer-executable instructions, etc. In the case of a computer (1102), the drives and media correspond to storing any data in a suitable digital format. Although the description of computer-readable media above refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those skilled in the art will know that other types of computer-readable media, such as zip drives, magnetic cassettes, flash memory cards, cartridges, etc., may also be used in exemplary operating environments and that any of these media may contain computer-executable instructions for performing the methods of the present disclosure.

[0167] A number of program modules, including an operating system (1130), one or more application programs (1132), other program modules (1134), and program data (1136), may be stored in the drive and RAM (1112). All or part of the operating system, application, module and / or data may also be cached in RAM (1112). It will be well known that the present disclosure may be implemented in various commercially available operating systems or combinations of operating systems.

[0168] The user can input commands and information into the computer (1102) through one or more wired / wireless input devices, such as a pointing device like a keyboard (1138) and a mouse (1140). Other input devices (not shown) may include a microphone, an IR remote control, a joystick, a game pad, a stylus pen, a touch screen, etc. These and other input devices are often connected to the processing device (1104) via an input device interface (1142) connected to the system bus (1108), but may also be connected via other interfaces such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, etc.

[0169] A monitor (1144) or other type of display device is also connected to the system bus (1108) via an interface such as a video adapter (1146). In addition to the monitor (1144), the computer generally includes other peripheral output devices (not shown), such as speakers, a printer, etc.

[0170] The computer (1102) may operate in a networked environment using a logical connection to one or more remote computers, such as remote computer(s) (1148), via wired and / or wireless communication. The remote computer(s) (1148) may be a workstation, a computing device computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other conventional network node, and generally include many or all of the components described for the computer (1102), but for brevity, only the memory storage device (1150) is illustrated. The illustrated logical connection includes a wired / wireless connection to a local area network (LAN) (1152) and / or a larger network, e.g., a wide area network (WAN) (1154). These LAN and WAN networking environments are common in offices and companies, facilitating enterprise-wide computer networks such as intranets, and all of them can be connected to global computer networks, for example, the Internet.

[0171] When used in a LAN networking environment, the computer (1102) is connected to a local network (1152) via a wired and / or wireless communication network interface or adapter (1156). The adapter (1156) may facilitate wired or wireless communication to the LAN (1152), and the LAN (1152) may also include a wireless access point installed therein to communicate with the wireless adapter (1156). When used in a WAN networking environment, the computer (1102) may include a modem (1158), be connected to a communication computing device on the WAN (1154), or have other means to establish communication through the WAN (1154), such as through the Internet. The modem (1158), which may be an internal or external and a wired or wireless device, is connected to the system bus (1108) via a serial port interface (1142). In a networked environment, the program modules described for the computer (1102) or parts thereof may be stored in a remote memory / storage device (1150). It will be well known that the illustrated network connection is exemplary and that other means of establishing a communication link between computers may be used.

[0172] The computer (1102) operates to communicate with any wireless device or object that is deployed and operated by wireless communication, for example, a printer, scanner, desktop and / or portable computer, PDA (portable data assistant), communication satellite, any equipment or place associated with a wireless detectable tag, and a telephone. This includes at least Wi-Fi and Bluetooth wireless technologies. Accordingly, the communication may be a predefined structure as in a conventional network or simply ad hoc communication between at least two devices.

[0173] Wi-Fi (Wireless Fidelity) enables connectivity to the Internet and other sources without wires. Wi-Fi is a wireless technology, similar to cellular communication, that allows devices, such as computers, to transmit and receive data indoors and outdoors—that is, anywhere within the coverage area of ​​a base station. Wi-Fi networks use a wireless technology called IEEE 802.11 (a, b, g, etc.) to provide secure, reliable, and high-speed wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in unlicensed 2.4 and 5 GHz wireless bands at data rates, for example, 11 Mbps (802.11a) or 54 Mbps (802.11b), or in products that include both bands (dual band).

[0174] Those skilled in the art of the present disclosure will understand that information and signals may be represented using any various different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0175] Those skilled in the art will understand that the various exemplary logic blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented by electronic hardware, various forms of programs or design code (referred to herein as software for convenience), or a combination of all such. To clearly illustrate this interoperability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in relation to their functions. Whether such functions are implemented as hardware or software depends on the design constraints imposed on the specific application and the entire system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementation decisions should not be interpreted as being outside the scope of this disclosure.

[0176] The various embodiments presented herein may be implemented as methods, devices, or articles of manufacture using standard programming and / or engineering techniques. The term "article of manufacture" includes a computer program, a carrier, or a medium accessible from any computer-readable storage device. For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Additionally, the various storage media presented herein include one or more devices and / or other machine-readable media for storing information.

[0177] It should be understood that the specific order or hierarchy of the steps in the presented processes is an example of exemplary approaches. It should be understood that the specific order or hierarchy of the steps in the processes may be rearranged within the scope of this disclosure based on design priorities. The appended method claims provide elements of various steps in a sample order, but do not imply being limited to the specific order or hierarchy presented.

[0178] Description of the presented embodiments is provided so that a person skilled in the art may use or practice the present disclosure. Various modifications to these embodiments will be apparent to a person skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments presented herein, but should be interpreted in the broadest possible scope consistent with the principles and novel features presented herein.

[0179]

[0180] As described above, the relevant details have been described in the best mode for carrying out the invention.

[0181] The present invention can be used in technical fields where image style conversion technology is utilized.

Claims

1. A computer program stored on a computer-readable storage medium, wherein the computer program performs the following methods for style switching when executed on a computer device, and the method comprises: A step of obtaining content features from a content image and style features from a style image, respectively, using a pre-trained image encoder; A step of clustering a query based on the above content features into multiple groups, and performing an attention operation between the clustered query and a key and value based on the above style features to generate a primary style transition feature with a style applied for each content area; A step of generating a second style transition feature by improving the style of the first style transition feature using an adaptive weight calculated based on the channel-specific difference between the first style transition feature and the style feature; A step of interpolating the content feature to the second style transition feature and aligning the pixel-unit distribution of the second style transition feature with the pixel-unit distribution of the content feature to generate a final style transition feature in which content information is preserved; and A step of generating a final style transition image by inputting the above final style transition feature into a pre-trained decoding block; including, A computer program stored on a computer-readable storage medium.

2. In Paragraph 1, The above-mentioned pre-trained image encoder is a Vector Quantization Encoder, and The step of acquiring the above-mentioned content features and the above-mentioned style features is, A step of obtaining the content features and style features by quantizing the raw feature map extracted through the vector quantization encoder through a vector-quantization bottleneck; including, A computer program stored on a computer-readable storage medium.

3. In Paragraph 1, The step of generating the above-mentioned first style transition feature is, A step of calculating a cluster centroid for each of a plurality of groups obtained by clustering the queries of the above-mentioned content features, and clustering by assigning each query of the above-mentioned content features to the group corresponding to the cluster centroid closest in distance; including, A computer program stored on a computer-readable storage medium.

4. In Paragraph 3, The step of generating the above-mentioned first style transition feature is, The method further comprises the step of generating stylized clustered features through a cross-attention operation between the cluster centroid calculated above and the key and value generated based on the style features above. A computer program stored on a computer-readable storage medium.

5. In Paragraph 4, The step of generating the above-mentioned first style transition feature is, A step of generating the first style transition feature by mapping the group-specific style transition feature corresponding to the group to which each query of the above content feature belongs to the image coordinates of each query; including, A computer program stored on a computer-readable storage medium.

6. In Paragraph 1, The number of the above plurality of groups is 22, A computer program stored on a computer-readable storage medium.

7. In Paragraph 1, In the step of generating the above-mentioned second style transition feature, calculating the above-mentioned adaptive weight is, Calculated by averaging the channel-specific differences between the above-mentioned first-order style transition feature and the above-mentioned style feature over the spatial dimension, and applying min-max scaling to the said average value, A computer program stored on a computer-readable storage medium.

8. In Paragraph 7, The step of generating the above secondary style transition feature is, A step of generating the second style transition feature by performing an Adaptive Instance Normalization (AdaIN) operation on the first style transition feature using channel-specific statistics of the style feature, and multiplying the result of the operation by the calculated adaptive weight; including, A computer program stored on a computer-readable storage medium.

9. In Paragraph 1, The step of generating the above-mentioned final style transition feature is, A step of linearly interpolating the above-mentioned secondary style transition feature and the above-mentioned content feature using a predetermined interpolation weight, and passing the linearly interpolated feature through a residual block; including, A computer program stored on a computer-readable storage medium.

10. In Paragraph 9, The above interpolation weights are, 0.87 people, A computer program stored on a computer-readable storage medium.

11. In Paragraph 1, The step of generating the above-mentioned final style transition feature is, A step of adjusting and aligning the pixel-wise average and standard deviation of the above-mentioned secondary style transition features to match the pixel-wise average and standard deviation of the above-mentioned content features; including, A computer program stored on a computer-readable storage medium.

12. In Paragraph 1, The above first style transition feature generation, second style transition feature, interpolation, and alignment steps are performed sequentially in blocks included in the decoder, A computer program stored on a computer-readable storage medium.

13. In Paragraph 1, Prior to the step of generating the above primary style transition feature, The method further includes the step of generating initial style transition features through an adaptive instance normalization (AdaIN) operation that matches the channel-specific statistics (mean, standard deviation) of the above-mentioned content features with the channel-specific statistics of the above-mentioned style features; The step of generating the above first and second style transition features is performed based on the above initial style transition features, A computer program stored on a computer-readable storage medium.

14. As a computer device that performs methods for style switching, One or more processors; and Memory for storing instructions executable on one or more of the above processors; Includes, The above one or more processors, Using a pre-trained image encoder, content features are obtained from content images and style features are obtained from style images, respectively, and A query based on the above content features is clustered into multiple groups, and an attention operation is performed between the clustered query and the key and value based on the above style features to generate a primary style transition feature with applied styles for each content area, and A second style transition feature is generated by improving the style of the first style transition feature using adaptive weights calculated based on the channel-specific difference between the first style transition feature and the style feature, and The content feature is interpolated to the second style transition feature, and the pixel-unit distribution of the second style transition feature is aligned with the pixel-unit distribution of the content feature to generate a final style transition feature in which content information is preserved, and, Generating a final style transition image by inputting the above final style transition features into a pre-trained decoding block, Computer device.

15. A method for style switching executed on a computer device, A step of obtaining content features from a content image and style features from a style image, respectively, using a pre-trained image encoder; A step of clustering a query based on the above content features into multiple groups, and performing an attention operation between the clustered query and a key and value based on the above style features to generate a primary style transition feature with a style applied for each content area; A step of generating a second style transition feature by improving the style of the first style transition feature using an adaptive weight calculated based on the channel-specific difference between the first style transition feature and the style feature; A step of interpolating the content feature to the second style transition feature and aligning the pixel-unit distribution of the second style transition feature with the pixel-unit distribution of the content feature to generate a final style transition feature in which content information is preserved; and A step of generating a final style transition image by inputting the above final style transition feature into a pre-trained decoding block; including, method.

Images