Visual style conversion method and related device

Abstract

The invention provides a visual style conversion method and a related device, and the method comprises the steps: obtaining prompt data matched with style description data after the style description data inputted by a user is obtained, and obtaining metadata of a to-be-processed UI component and image data of the to-be-processed UI component; inputting the style description data, the image data of the to-be-processed UI component, the metadata of the to-be-processed UI component and the prompt data into a visual style conversion model, and obtaining configuration data of the to-be-processed UI component output by the visual style conversion model, the configuration data of the to-be-processed UI component is used for obtaining the UI component matched with the style indicated by the style description data on the premise of not changing the control logic in the basic configuration of the to-be-processed UI component, so that style consistency is achieved, the manufacturing efficiency is improved, the manufacturing period is shortened, and the manufacturing cost is reduced; and non-art professionals can complete visual style conversion on the UI component after inputting the style description data, so that the difficulty is reduced.

Description

Technical Field

[0001] This application relates to the field of data processing methods, and in particular to a visual style transfer method and related apparatus. Background Technology

[0002] Currently, the user interface (UI) has become a key component for enhancing user immersion and interactive experience, and UI visual style changes require manual replacement. Specifically, a UI comprises multiple UI prefabs, each containing at least one UI component. When a UI prefab includes multiple UI components, these components are arranged in a hierarchical structure to represent the priority and dependencies among the components. When manually replacing the UI visual style, artists use image editing software such as Photoshop to manually draw the UI components. After completing the drawing, the UI component resource files are imported into the UI resource file stored on the electronic device. The electronic device then uses the UI resource file to render the UI to display it on its screen.

[0003] However, artists often struggle to maintain consistency in the overall visual style of a UI when designing UI components, which can negatively impact the overall visual effect. Manually drawing UI component images results in long production cycles and high costs, making it difficult to meet the demands of rapid iteration. Furthermore, artists need to rely on image editing software to create UI component images, which presents a steep learning curve and makes it difficult for non-art professionals to use efficiently, further increasing the complexity. Summary of the Invention

[0004] In view of the above problems, this application provides a visual style conversion method and related apparatus to achieve style consistency, improve production efficiency, shorten production cycle, reduce production cost, and reduce difficulty. The specific solution is as follows:

[0005] The first aspect of this application provides a visual style transfer method, including:

[0006] Obtain style description data input by the user, the style description data being used to indicate the style corresponding to the UI component to be processed after style conversion;

[0007] Obtain prompt data that matches the style description data, the prompt data being used to optimize the style transition effect of the UI component to be processed;

[0008] Obtain the metadata of the UI component to be processed and the image data of the UI component to be processed. The metadata of the UI component to be processed is structured data used to describe the basic configuration of the UI component to be processed.

[0009] The style description data, the image data of the UI component to be processed, the metadata of the UI component to be processed, and the prompt data are input into the visual style conversion model to obtain the configuration data of the UI component to be processed output by the visual style conversion model. The configuration data of the UI component to be processed is used to obtain a UI component that matches the style indicated by the style description data without changing the control logic in the basic configuration of the UI component to be processed.

[0010] In one possible implementation, the method further includes: if configuration data of a key UI component is obtained before obtaining configuration data of the UI component to be processed, then the configuration data of the key UI component is input into the visual style conversion model.

[0011] In one possible implementation, the method further includes: determining the processing order of the UI components to be processed based on their priority relationship and their dependency relationship;

[0012] According to the processing order of each UI component to be processed, multiple image generation tasks are used to process multiple UI components to be processed in parallel. Each image generation task is used to call the visual style transfer model to process the UI component to be processed.

[0013] In one possible implementation, the method further includes at least one of the following:

[0014] If the image generation task fails due to a temporary error, a retry time for the image generation task is determined, and the image generation task is re-executed within the retry time. The temporary error is used to indicate that there is a possibility that the image generation task will succeed after re-execution.

[0015] If the image generation task fails due to a permanent error, the image generation task is terminated. The permanent error is used to indicate that the image generation task needs to be repaired before execution.

[0016] If the UI component generated based on the configuration data of the UI component to be processed does not meet the preset quality requirements, a random seed is assigned to the UI component to be processed, triggering the image generation task of the UI component to be processed. The image generation task of the UI component to be processed inputs the random seed into the visual style conversion model to call the visual style conversion model to regenerate the configuration data of the UI component to be processed.

[0017] In one possible implementation, the method further includes:

[0018] For the UI prefab in the UI currently displayed on the electronic device, obtain the rendering data of the UI component to be updated in the UI prefab, wherein the rendering data of the UI component is obtained according to the configuration data of the UI component;

[0019] Based on the rendering data of the UI components to be updated in the UI prefab, the UI prefab is drawn in the UI; after all UI prefabs in the UI are drawn, the electronic device displays a UI that matches the style indicated by the style description data.

[0020] In one possible implementation, obtaining the rendering data of the UI component to be updated in the UI prefab includes:

[0021] Based on the resource identifier of the UI component to be updated, determine whether the configuration data corresponding to the resource identifier is stored in the cache of the electronic device. The resource identifier is stored in the resource mapping table and can be found in the UI prefab through the identifier of the UI component to be updated.

[0022] If the cache stores configuration data corresponding to the resource identifier, the rendering data of the UI component to be updated is updated using the configuration data corresponding to the resource identifier;

[0023] If the configuration data corresponding to the resource identifier is not stored in the cache, the configuration data corresponding to the resource identifier is obtained from the disk of the electronic device;

[0024] The configuration data corresponding to the resource identifier obtained from the disk is stored in the cache, and the rendering data of the UI component to be updated is updated using the configuration data corresponding to the resource identifier.

[0025] In one possible implementation, the method further includes: after obtaining the configuration data of the UI component to be processed output by the visual style transfer model, fine-tuning the configuration data of the UI component to be processed according to context data related to the UI component to be processed, wherein the context data includes the metadata of the UI component to be processed, the historical configuration data of the UI component to be processed, and the configuration data of UI components adjacent to the UI component to be processed.

[0026] A second aspect of this application provides a visual style transfer system, comprising:

[0027] The first acquisition module is used to acquire style description data input by the user, wherein the style description data is used to indicate the style corresponding to the UI component to be processed after the style conversion is completed;

[0028] The second acquisition module is used to acquire prompt data that matches the style description data, and the prompt data is used to optimize the style conversion effect of the UI component to be processed.

[0029] The third acquisition module is used to acquire the metadata of the UI component to be processed and the image data of the UI component to be processed. The metadata of the UI component to be processed is structured data used to describe the basic configuration of the UI component to be processed.

[0030] The conversion module is used to input the style description data, the image data of the UI component to be processed, the metadata of the UI component to be processed, and the prompt data into the visual style conversion model, and obtain the configuration data of the UI component to be processed output by the visual style conversion model. The configuration data of the UI component to be processed is used to obtain a UI component that matches the style indicated by the style description data without changing the control logic in the basic configuration of the UI component to be processed.

[0031] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the visual style transfer method of the first aspect or any implementation thereof.

[0032] A fourth aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein the memory is used to store a computer program;

[0033] The processor is used to execute the computer program so that the electronic device can implement the visual style transfer method of the first aspect or any implementation thereof.

[0034] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs that, when executed by an electronic device, enable the electronic device to perform the visual style conversion method described in the first aspect or any implementation thereof.

[0035] By employing the above technical solutions, the visual style conversion method and related apparatus provided in this application, after obtaining the style description data input by the user, obtain prompt data matching the style description data, as well as the metadata and image data of the UI component to be processed. The metadata of the UI component to be processed is structured data used to describe the basic configuration of the UI component to be processed. The style description data, the image data of the UI component to be processed, the metadata of the UI component to be processed, and the prompt data are input into the visual style conversion model to obtain the configuration data of the UI component to be processed output by the visual style conversion model. The configuration data of the UI component to be processed is used to obtain a UI component that matches the style indicated by the style description data without changing the control logic in the basic configuration of the UI component to be processed. This achieves automatic generation of UI components that match the style indicated by the style description data, thereby achieving style consistency, improving production efficiency, shortening the production cycle, and reducing production costs. Furthermore, even non-art professionals can use the above method to complete the visual style conversion of UI components after inputting style description data, reducing the difficulty. Attached Figure Description

[0036] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0037] Figure 1 A flowchart of a visual style transfer method provided in this application;

[0038] Figure 2 A schematic diagram of the PrefabUILayout layout resources and PrefabUILayoutSkin skin resources provided in this application;

[0039] Figure 3 A schematic diagram illustrating the processing through different AI processing management methods provided in this application;

[0040] Figure 4 Another flowchart of a visual style transfer method provided in this application;

[0041] Figure 5 The flowchart provided in this application illustrates the batch processing of various UI components to be processed.

[0042] Figure 6 A diagram illustrating resource management provided for this application;

[0043] Figure 7 Another flowchart of a visual style transfer method provided in this application;

[0044] Figure 8 A schematic diagram illustrating resource loading and optimized rendering provided in this application;

[0045] Figure 9 A schematic diagram illustrating seamless engine integration provided for this application;

[0046] Figure 10 Architecture diagram of the MCP tool provided in this application;

[0047] Figure 11 A schematic diagram illustrating the parsing of a PSD file to obtain a UI prefab, provided for this application;

[0048] Figure 12 A diagram illustrating the multilingual support provided for this application;

[0049] Figure 13 A schematic diagram of the visual style transfer system provided in this application;

[0050] Figure 14 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0051] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0052] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0053] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements but may include other elements not explicitly listed or inherent to such processes, methods, systems, products, or apparatus.

[0054] Please see Figure 1 It illustrates an optional flow of a visual style transfer method provided in an embodiment of this application, which may include the following steps:

[0055] S101. Obtain the style description data input by the user. The style description data is used to indicate the style of the UI component to be processed after the style conversion. The style description data can be keywords describing the style, such as any one of the keywords entered by the user, such as "tech feel," "traditional Chinese style," or "festival," which represents the style currently desired by the user.

[0056] Upon receiving style description data input by the user, a style conversion is triggered for the UI components in the UI prefab. UI components that require style conversion can be designated as pending UI components, and the style conversion process can begin for any of these pending UI components.

[0057] S102. Obtain the hint data that matches the style description data. The hint data is used to optimize the style transition effect of the UI component to be processed.

[0058] In this embodiment, the style description data may only record the style. However, the style conversion effect may be different under the same style due to different details. Therefore, after obtaining the style description data, prompt data that matches the style description data can also be obtained. The prompt data can be used to supplement the style corresponding to the style description data, such as adding detailed description data under the style.

[0059] For example, when a user enters "tech feel", adding related words such as "glowing lines", "holographic", "clean interface", and "minimalist" can produce a better style transition effect. These related words are suggestions that match the style description data.

[0060] In one possible implementation, this embodiment provides a preset style tone prompt word library, which records prompt data for at least one style. After obtaining style description data input by the user, prompt data for the style corresponding to that style description data is retrieved from the preset style tone prompt word library. If prompt data for the style corresponding to that style description data is not found in the preset style tone prompt word library, prompt data for a style similar to that style can be retrieved.

[0061] In one possible implementation, the cue data can be shared negative cue words used to optimize style transition effects. These shared negative cue words record related words for effects that are prohibited during style transitions, thus preventing the corresponding effects from occurring during the transition. Examples of shared negative cue words include, but are not limited to, text, watermark, signature, and blurry. Because the functions (purposes) of UI prefabs differ, the shared negative cue words used may vary. Therefore, this embodiment can select matching shared negative cue words based on the function of the UI prefab, which will not be detailed here.

[0062] In practical applications, related words can be selected from a preset style tone cue word library and / or shared negative cue words. These related words allow for setting a unified macro-artistic tone for different UI components, eliminating the hassle of users memorizing and manually inputting complex related words, thus greatly improving usability. When including related words in the cue data, different weights can be applied to the related words; for example, related words associated with the core style design have the highest weight to emphasize the core style design and ensure that the generated results meet expectations.

[0063] One possible implementation incorporates a built-in structured prompt generator for the UI domain. Style description data is input into the prompt generator, which then outputs prompt data matching the style description data. The prompt generator can be trained using image data of historical UI components under various styles, enabling it to generate prompt data for any style. For example, during training, a preset style-based prompt library and / or related words from shared negative prompts can be used as training data. The prompt generator eliminates the need for users to memorize and manually input prompt data, greatly improving usability.

[0064] S103. Obtain the metadata and image data of the UI component to be processed. The metadata of the UI component to be processed is structured data used to describe the basic configuration of the UI component to be processed.

[0065] In this embodiment, after triggering a style transition for a UI component in the UI prefab, the metadata and image data of the UI component to be processed are obtained. The metadata of the UI component to be processed may include its attribute data and control logic. The control logic indicates the function and control rules of the UI component. For example, the metadata of the UI component to be processed may include aspect ratio (a form of size), component type (whether it is a button, icon, or background), and 3x3 grid slice information. This embodiment does not limit the metadata of the UI component to be processed.

[0066] A UI component to be processed is a UI component at a specific level within a UI prefab. It can be understood that a UI prefab can include multiple UI components, arranged hierarchically to indicate their priority and dependencies. For any UI prefab, a UI layout resource file (PrefabUILayout layout resource) manages UI structure information, preview images, and a list of UI component IDs (Identifiers). The UI structure information includes the hierarchical structure of the UI prefab and the metadata of each UI component within it. The preview image is an image of the UI prefab, showcasing its display effect. After changing the style of a UI component within a UI prefab, the UI prefab allows for real-time preview of its visual effect. The UI component ID list records the identifier (i.e., ID) of each UI component within the UI prefab. In addition to the UI layout resource file, this embodiment also provides a UI skin resource file (PrefabUILayoutSkin). The PrefabUILayoutSkin manages the image mapping relationships of UI components, supporting the switching and management of multiple skin schemes. Specifically, the PrefabUILayoutSkin manages the configuration data of any UI component in the UI prefab under different styles. This configuration data includes the image data of the UI component under different styles, the rendering type and size matching the image data, etc. The PrefabUILayoutSkin can be categorized by style type to manage the configuration data of UI components under each style. Therefore, this embodiment can obtain metadata from the PrefabUILayout layout resource and image data from the PrefabUILayoutSkin resource based on the ID of the UI component to be processed in the UI prefab.

[0067] like Figure 2As shown, the PrefabUILayout layout resource records UI structure information, preview images, and a list of UI component IDs. The PrefabUILayoutSkin skin resource records configuration data for three styles: Skin A is the default style, Skin B is a dark style, and Skin C is a holiday style. Each style records the GUID (Globally Unique Identifier) ​​of each UI component and its ID in the UI prefab. The GUID of the UI component serves as its resource identifier, used to find the configuration data of the UI component in at least one style. When updating a UI prefab, the ID of the UI component in the UI prefab is used to find its GUID in the PrefabUILayoutSkin skin resource. Then, the configuration data of that UI component in a particular style is found using the GUID. This configuration data is then used to update the UI prefab, such as redrawing the image of the UI prefab (the UI component's metadata may also be referenced during the drawing of the UI prefab, which will not be detailed here). Images of UI prefabs can be stored in the preview image of the UI prefab, allowing the preview image to include images of the UI prefab under different styles. This embodiment, through resource separation design, allows users to manage UI layout and skin resources independently, improving editing flexibility and maintainability.

[0068] Furthermore, this embodiment provides a visual editor with a drag-and-drop editing interface. Users can drag and drop preview images of UI prefabs into the drag-and-drop editing interface to preview the display effect of the UI prefabs in real time. Users can also drag and adjust the preview images of UI prefabs in the drag-and-drop editing interface to adjust parameters, etc. This drag-and-drop editing interface supports common operations such as real-time preview and parameter adjustment, allowing users to precisely control key parameters (such as configuration data) of UI prefabs in any style.

[0069] One point to note here is that different UI prefabs may use the same UI components, and the IDs of the same UI components may differ in different UI prefabs. When retrieving the configuration data of a UI component under a certain style, you can first look up the corresponding GUID in the PrefabUILayoutSkin skin resource based on the ID of the UI component in the UI prefab. Then, based on the GUID, determine whether the configuration data for the current style exists in the PrefabUILayoutSkin skin resource. If it does, there is no need to retrieve it again. Therefore, binding the configuration data of the UI component with the GUID of the UI component in the PrefabUILayoutSkin skin resource can reduce the number of processing steps and improve execution efficiency.

[0070] S104. Input the style description data, the image data of the UI component to be processed, the metadata and prompt data of the UI component to be processed into the visual style conversion model, and obtain the configuration data of the UI component to be processed output by the visual style conversion model. The configuration data of the UI component to be processed is used to obtain a UI component that matches the style indicated by the style description data without changing the control logic in the basic configuration of the UI component to be processed.

[0071] The visual style transfer model is trained based on historical style description data, metadata and image data of historical UI components under historical styles, and historical cue data. This allows the model to determine the style transfer effect based on the style description and cue data. Based on the style transfer effect and referring to the UI component's metadata, it generates configuration data for the UI component. The purpose of referring to the UI component's metadata is to avoid changing the control logic in the UI component's basic configuration (such as the UI component's structure and function), so as to obtain a UI component that matches the style indicated by the style description data using the configuration data. The visual style transfer model can be trained using, but is not limited to, the LoRA (Low-Rank Adaptation) fine-tuning principle. In this embodiment, multiple LoRA models (visual style transfer models) are supported, with more than 100 models.

[0072] The configuration data of a UI component may include the image data of the UI component, the rendering type of the UI component under the style indicated by the style description data, and the attributes of the UI component under that style. The attributes of the UI component under that style may include the size of the UI component, so that the style effect displayed by the UI component matches the style indicated by the style description data.

[0073] One point to note here is that the metadata of UI components can be converted into keywords and input into the visual style transfer model. For example, the metadata of a UI component includes aspect ratio, component type (whether it is a button, icon, or background), 9-sliced ​​panel information, etc. This structured data (such as circular icon, 16:9 aspect ratio background, 9-sliced ​​panel) is automatically and preferentially converted into keywords, providing strong structural and usage constraints when the visual style transfer model generates the configuration data of the UI component.

[0074] In some examples, the visual style transfer model can integrate the functionality of the Prompt smart generator. For example, style description data can be input into the visual style transfer model, which can then obtain cue data related to the style description data. The visual style transfer model can then output the configuration data of the UI component to be processed based on the style description data, the image data of the UI component to be processed, the metadata of the UI component to be processed, and the cue data. The specific process will not be described further.

[0075] In some examples, the visual style transformation of UI components is controlled through multiple processing pipelines. For instance, this embodiment configures three different AI (Artificial Intelligence) processing pipelines, such as... Figure 3 As shown, the three different AI processing pipelines can include: the image-to-image pipeline, the style transfer pipeline, and the intelligent background removal pipeline. The intelligent background removal pipeline can utilize advanced vision-language models to intelligently remove backgrounds and optimize edges for UI components, improving their adaptability. The style transfer pipeline can utilize visual style transfer models to convert UI components into specific art styles, such as cartoon, realistic, or traditional Chinese styles. The image-to-image pipeline can utilize image-to-image models to perform style transfer based on the original image of the UI component, maintaining its basic structure and functionality while applying a new visual style.

[0076] The core advantage of this multi-pipeline architecture lies in its flexible programmability, allowing multiple AI processing pipelines to be combined on demand to form complex workflows. For example, the processing task of a UI component can be set as follows: first, the "intelligent background removal pipeline" precisely separates the main body of the UI component (i.e., removes the background of the UI component); the output is then sent to the "style transfer pipeline" to apply the target art style; finally, the "image-to-image pipeline" fine-tunes the configuration data of the UI component based on the context data related to the UI component, such as fine-tuning the details and lighting of the UI component. This mechanism of processing different AI capabilities in series or in parallel is the key difference from traditional single-processing pipelines.

[0077] Combination Figure 3 and Figure 1 The process involves receiving style description data input by the user and the original image of the UI component to be processed, which can be obtained from the PrefabUILayout layout resource. The user then selects the desired AI processing pipeline from three different options. Figure 3 The specific functions of the three AI processing pipelines are as follows:

[0078] The intelligent matting pipeline can call the GPT40 matting model, input the original image of the UI component to be processed into the GPT40 matting model, and the GPT40 matting model intelligently removes the background from the original image of the UI component to be processed, so as to obtain the image data and metadata of the UI component to be processed.

[0079] The style transfer pipeline can call the visual style transfer model, inputting style description data, image data of the UI component to be processed, metadata and tooltip data of the UI component to be processed into the visual style transfer model, and obtaining the configuration data of the UI component to be processed output by the visual style transfer model. The configuration data of the UI component to be processed can be used to generate an image that matches the style indicated by the style description data.

[0080] The graph-based pipeline can call a graph-based model, inputting the original image and style description data of the UI component to be processed into the graph-based model. The graph-based model then outputs the target image of the UI component, which is an image that matches the style indicated by the style description data. For example, by inputting the metadata of the UI component to be processed and the configuration data of the UI component output from the visual style transformation model into the graph-based model, an image of the UI component that matches the style indicated by the style description data can be obtained without changing the control logic in the basic configuration of the UI component to be processed.

[0081] In some examples, after obtaining the configuration data of the UI component to be processed output by the visual style transfer model, the visual style transfer method provided in this embodiment fine-tunes the configuration data of the UI component to be processed based on the context data related to the UI component to be processed. The context data includes the metadata of the UI component to be processed, the historical configuration data of the UI component to be processed, and the configuration data of the UI components adjacent to the UI component to be processed.

[0082] For example, the graph-generated graph model can fine-tune the details and lighting of the UI component to be processed based on the context data related to the UI component to be processed, so as to ensure that the generated image of the UI component meets the style requirements and has overall consistency.

[0083] In some examples, the visual style conversion method provided in this embodiment may further include: if the configuration data of a key UI component is obtained before obtaining the configuration data of the UI component to be processed, then the configuration data of the key UI component is input into the visual style conversion model.

[0084] The key UI components can be one or more, such as a main button and / or a general panel. During the visual style transformation of UI components, configuration data for the key UI components can be generated first, and the image data in the configuration data can be used as "style anchors". When processing other UI components, the "style anchors" are used as reference inputs for image prompts or control networks, thereby forcing the image data of other UI components to align with the "style anchors" to suppress style drift and ensure overall style consistency. For example, other UI components can be forced to align with the "style anchors" in at least one aspect such as color, lighting, texture, and brushstrokes.

[0085] As can be seen from the above technical solution, after obtaining the style description data input by the user, the solution obtains prompt data matching the style description data, as well as the metadata and image data of the UI component to be processed. The metadata of the UI component to be processed is structured data used to describe the basic configuration of the UI component to be processed. The style description data, the image data of the UI component to be processed, the metadata of the UI component to be processed, and the prompt data are input into the visual style conversion model. The configuration data of the UI component to be processed output by the visual style conversion model is obtained. The configuration data of the UI component to be processed is used to obtain a UI component that matches the style indicated by the style description data without changing the control logic in the basic configuration of the UI component to be processed. This achieves automatic generation of UI components that match the style indicated by the style description data, thereby achieving style consistency, improving production efficiency, shortening the production cycle, and reducing production costs. Furthermore, even non-art professionals can use the above method to complete the visual style conversion of UI components after inputting style description data, reducing the difficulty.

[0086] Experiments were conducted on mainstream image formats such as PNG, JPG, and TGA. The experiments proved that the generation time of a single UI component is less than 30 seconds, the style consistency reaches more than 95%, the production cycle is shortened from several days to several hours, the efficiency is improved by more than 80%, the labor cost is reduced by more than 60%, and the image generation of UI components with resolutions of 4K and below is supported.

[0087] Please see Figure 4 This illustrates another optional process for a visual style transfer method provided in an embodiment of this application, which may include the following steps:

[0088] S201. Obtain the style description data input by the user. The style description data is used to indicate the style corresponding to the UI component to be processed after the style conversion is completed.

[0089] S202. Obtain hint data that matches the style description data. The hint data is used to optimize the style transition effect of the UI component to be processed.

[0090] S203. Obtain the metadata and image data of the UI component to be processed. The metadata of the UI component to be processed is structured data used to describe the basic configuration of the UI component to be processed.

[0091] In this embodiment, steps S201 to S203 are the same as steps S101 to S103, and will not be described again here.

[0092] S204. Determine the processing order of each UI component based on its priority and dependency relationships.

[0093] S205. According to the processing order of each UI component to be processed, multiple image generation tasks are used to process multiple UI components to be processed in parallel. Each image generation task is used to call the visual style transfer model to process the UI component to be processed.

[0094] S206. The image generation task inputs style description data, image data of the UI component to be processed, metadata and prompt data of the UI component to be processed into the visual style conversion model, and obtains the configuration data of the UI component to be processed output by the visual style conversion model. The configuration data of the UI component to be processed is used to obtain a UI component that matches the style indicated by the style description data without changing the control logic in the basic configuration of the UI component to be processed.

[0095] In this embodiment, the UI components to be processed in a UI prefab have priority and dependency relationships. UI components with higher priority that are depended upon by other UI components are processed earlier than those components. This embodiment can construct a processing queue, collect all UI components to be processed in the queue, and then determine the processing order of each UI component based on its priority and dependency relationships. This processing order allows for batch processing of the UI components in the processing queue.

[0096] The batch processing process can involve launching multiple image generation tasks, with each task processing one UI component to be processed. Specifically, each image generation task can call a visual style transfer model. The image generation task inputs style description data, image data of the UI component to be processed, metadata of the UI component to be processed, and prompt data into the visual style transfer model to obtain the configuration data of the UI component to be processed output by the visual style transfer model.

[0097] For example, SpriteProcessQueue enables batch processing of UI components, supporting priority sorting during batch processing. The batch processing progress can be displayed in real time, and pause, resume, and cancel operations are supported, providing a good user experience. Each image generation task has clear state management, including pending, processing, completed, and failed states, ensuring precise control over visual style transfer. Image generation tasks can be sorted by dependency or priority to form an efficient batch processing workflow.

[0098] In some examples, the visual style transfer method provided in this embodiment also supports error recovery. When the image generation task fails, it can automatically retry or skip problematic UI components to be processed, ensuring the stability of batch processing. The error recovery methods include, but are not limited to, at least one of the following:

[0099] If the image generation task fails due to a temporary error, a retry time for the image generation task is determined, and the image generation task is re-executed within the retry time. The temporary error is used to indicate that there is a possibility of success after re-execution.

[0100] If the image generation task fails due to a permanent error, the image generation task will be terminated. A permanent error indicates that the image generation task needs to be repaired before it can be executed.

[0101] If the UI component generated based on the configuration data of the UI component to be processed does not meet the preset quality requirements, a random seed is assigned to the UI component to be processed, triggering the image generation task of the UI component to be processed. The image generation task of the UI component to be processed inputs the random seed into the visual style transfer model to call the visual style transfer model to regenerate the configuration data of the UI component to be processed.

[0102] Temporary errors can include network timeouts and momentary server overload. Since there's a possibility of success after re-execution, when a task fails due to a temporary error, the image generation task can enter a retry queue using an exponential backoff algorithm. The exponential backoff algorithm determines the retry time for the image generation task, and within that time, the failed task is executed again. The exponential backoff algorithm gradually increases the retry interval, avoiding continuous service overload caused by a surge of requests within a short period.

[0103] Permanent errors can include content security policy and input parameter errors. If a task fails due to a permanent error, the image generation task cannot be successfully retried. Therefore, if an image generation task fails due to a permanent error, the task should be terminated. After terminating the image generation task, it can be marked as a failure and a detailed reason can be recorded to avoid unnecessary retries, thereby conserving API (Application Programming Interface) call quotas and computing resources.

[0104] The reason why UI components generated based on the configuration data of the UI component to be processed do not meet the preset quality requirements is to perform rapid automated quality checks on the UI components. Configuration data and preview images of the UI components are only stored when the preset quality requirements are met. One approach is to incorporate a lightweight image quality assessment model. After generating a symbolic style image using the UI component's metadata and configuration data, this image undergoes rapid automated quality checks, such as detecting blank images, solid color blocks, or severely mismatched sizes. If the quality check fails, indicating that the UI component does not meet the preset quality requirements, a new image generation task is initiated using a new random seed parameter, aiming to obtain a better random result without changing the core cue words (i.e., the input to the visual style transfer model).

[0105] Figure 5 The present application illustrates a process for batch processing UI components to be processed according to an embodiment of this application. This process may include: constructing a processing queue and collecting all UI components to be processed in the queue; determining the processing order of each UI component to be processed based on its priority and dependency relationships; initiating multiple image generation tasks to batch process the UI components to be processed; monitoring the task execution status, continuing to monitor the task execution status if the task is in progress; and determining whether the task has succeeded or failed if the task has ended. If the task has succeeded, the progress display is updated; if the task has failed, the failed UI components to be processed are retried or skipped.

[0106] After processing a portion of the UI components, the generated results are verified. Verification primarily determines whether the UI components meet preset quality requirements. If they do not, a new image generation task is initiated using a new random seed parameter. If they do meet the requirements, resource references are updated. Updating resource references means updating the UI component identifiers and mapping the updated identifiers to their GUIDs in the PrefabUILayoutSkin skin resource. A processing report is generated after batch processing, which can record failed image generation tasks, etc., without specific limitations.

[0107] In some examples, the visual style transfer method provided in this embodiment can monitor API response latency and success rate in real time when batch processing UI components. When an increase in API response latency or consecutive failures are detected, the number of image generation tasks in the batch can be reduced; conversely, the number of image generation tasks in the batch can be increased to achieve a dynamic balance between maximizing processing efficiency and maintaining system stability. Experiments have shown that it supports processing 500 UI components simultaneously.

[0108] As can be seen from the above technical solution, this embodiment can determine the processing order of each UI component to be processed based on their priority and dependency relationships. Following this processing order, multiple image generation tasks process multiple UI components in parallel. Each image generation task calls a visual style transfer model to process the UI component, achieving batch processing of the UI components. Compared to manual replacement one by one, this significantly improves work efficiency. Experiments have shown that batch processing efficiency is improved by more than 10 times.

[0109] In some examples, the visual style conversion method provided in this embodiment can employ a resource caching mechanism and a GUID referencing system to store UI components. The resource caching mechanism can store the latest version resource data (configuration data and metadata) of frequently used UI components in a cache. Resource data for all versions of a UI component can be stored on disk, with different versions identified by version identifiers. Frequently used UI components can be determined based on their usage frequency, enabling fast access to these components through the resource caching mechanism and avoiding duplicate loading. The GUID referencing system is used to map UI components to their stored resource data, thereby reducing memory usage.

[0110] like Figure 6 As shown, resource inputs can include: UI prefabs, raw image resources, and AI-generated images. UI structure information and a list of UI component IDs are obtained from the UI prefabs, and this information is recorded in the PrefabUILayout layout resource. Configuration data (such as image data) for UI components is obtained from the raw image resources and AI-generated images. This data can be recorded in the PrefabUILayoutSkin skin resource. The PrefabUILayoutUniqueSpriteUI element is used to associate the PrefabUILayout layout resource and the PrefabUILayoutSkin skin resource to find the resource data of the same UI component from both resources.

[0111] exist Figure 6In this system, the GUID referencing mechanism records the mapping between the identifier of a UI component in a UI prefab and its GUID in the PrefabUILayoutSkin skin resource. This mapping is used to retrieve the resource data of the UI component from the PrefabUILayoutSkin skin resource. Furthermore, GUIDs are also used to control storage in the resource caching mechanism, version control, and history. The resource caching mechanism stores the latest versions of frequently used UI components in the cache. Version control records any version of a UI component, and the history records historical versions of UI components. When outputting resources, the resource reference mapping is output from version control. For example, different versions of resource data are output based on the UI component's GUID. The resource caching mechanism outputs the latest version of the UI component's resource data, and the updated UI prefab is output using the GUID referencing system.

[0112] By employing the aforementioned resource caching and reference management optimization techniques, stable operation under high load can be ensured while reducing memory usage. Experiments have shown that this reduces memory usage by 50%.

[0113] Please see Figure 7 This illustrates another optional process of a visual style transfer method provided in an embodiment of this application, which may include the following steps:

[0114] S101. Obtain the style description data input by the user. The style description data is used to indicate the style corresponding to the UI component to be processed after the style conversion is completed.

[0115] S102. Obtain the hint data that matches the style description data. The hint data is used to optimize the style transition effect of the UI component to be processed.

[0116] S103. Obtain the metadata and image data of the UI component to be processed. The metadata of the UI component to be processed is structured data used to describe the basic configuration of the UI component to be processed.

[0117] S104. Input the style description data, the image data of the UI component to be processed, the metadata and prompt data of the UI component to be processed into the visual style conversion model, and obtain the configuration data of the UI component to be processed output by the visual style conversion model. The configuration data of the UI component to be processed is used to obtain a UI component that matches the style indicated by the style description data without changing the control logic in the basic configuration of the UI component to be processed.

[0118] S105. For the UI prefab in the UI currently displayed on the electronic device, obtain the rendering data of the UI component to be updated in the UI prefab. The rendering data of the UI component is obtained according to the configuration data of the UI component.

[0119] One possible approach is to use the UI component's configuration data as its rendering data to update the component's resources and attributes. For example, if the configuration data includes image data, text content, and some metadata, then it's not enough to just use the image data to render the UI component; the component's metadata also needs to be updated, and the displayed text content replaced. For instance, if the configuration data records a rendering type of "Sliced," and the UI component's known rendering type is "Simple," then the UI component's rendering type needs to be switched from "Simple" to "Sliced" to ensure correct scaling across different resolutions.

[0120] S106. Based on the rendering data of the UI components to be updated in the UI prefab, draw the UI prefab in the UI; after all UI prefabs in the UI are drawn, the electronic device displays the UI that matches the style indicated by the style description data.

[0121] After obtaining the rendering data of all UI components to be updated, a rendering command is sent to the rendering pipeline of the electronic device's rendering engine (such as the Unity engine) to force a UI refresh, thereby presenting the latest visual effects to the user in real time and realizing a real-time preview of the visual style transformation of UI components.

[0122] In some examples, the process of obtaining rendering data for the UI component to be updated includes:

[0123] 1) Based on the resource identifier of the UI component to be updated, determine whether the configuration data corresponding to the resource identifier is stored in the electronic device's cache. The resource identifier is stored in a resource mapping table and can be found through the identifier of the UI component to be updated in the UI prefab. The resource identifier of the UI component to be updated can be the GUID of the UI component in the PrefabUILayoutSkin skin resource. A UI component in the UI prefab is pre-bound with an identifier that points to the UI component in the UI prefab. The identifier of the UI component in the UI prefab can be recorded in the PrefabUILayoutSkin skin resource and corresponds to the GUID in the PrefabUILayoutSkin skin resource. Therefore, based on the identifier of the UI component in the UI prefab, the corresponding GUID can be found in the PrefabUILayoutSkin skin resource, and then the configuration data of the UI component can be found using the GUID.

[0124] 2) If the cache stores the configuration data corresponding to the resource identifier, update the rendering data of the UI component to be updated using the configuration data corresponding to the resource identifier.

[0125] 3) If the configuration data corresponding to the resource identifier is not stored in the cache, retrieve the configuration data corresponding to the resource identifier from the disk of the electronic device.

[0126] 4) Store the configuration data corresponding to the resource identifier obtained from the disk into the cache, and use the configuration data corresponding to the resource identifier to update the rendering data of the UI component to be updated. After storing the configuration data into the cache, return a reference to the configuration data, that is, the address of the configuration data in the cache.

[0127] The process is as follows Figure 8 As shown, after a request to update a UI component is initiated (e.g., a user-triggered UI update), the skin resource is searched using the UI component's GUID. The skin resource is the configuration data of the UI component. Specifically, the skin resource is searched to see if it exists in the cache. If it does, it is quickly loaded from the cache; otherwise, it is loaded from disk and stored in the cache. After obtaining the loaded skin resource, it is applied to the UI component. Then, a layered update mechanism is triggered to redraw only the affected UI layers within the UI prefab. The affected UI layers are the layers of the UI component whose skin resource is being updated. After redrawing the affected UI layers, the UI prefab is redrawn, enabling real-time previewing of the UI prefab within the UI.

[0128] One point to note here is that different UI prefabs may share a single UI component. Therefore, after obtaining the rendering data of the shared UI component, multiple UI prefabs can be redrawn simultaneously using the rendering data of the shared UI component to improve execution efficiency.

[0129] In this embodiment, the rendering engine of the electronic device can integrate the ComfyAI UI system. The ComfyAI UI system provides an API interface through PrefabUILayoutEditor, such as a call interface for the ApplySkin method. The ApplySkin method allows for real-time preview of the visual style transformation of UI components, providing immediate visual feedback. The processing procedure of the ApplySkin method is described above. Figure 8 , Figure 7 Steps S105 and S106 are not described in detail here.

[0130] Furthermore, the ComfyAI UI system provides APIs for calling the LoadSkin and ReleaseSkin methods. The LoadSkin method loads skin resources, and the ReleaseSkin method releases them. This modular management of skin resources, achieved through these methods, allows different projects to manage their own skin resources. This modular design and extensible architecture (such as extensible functional interfaces) enable the ComfyAI UI system to adapt to game projects of different sizes and types, supporting continuous functional evolution. Figure 9 The diagram shown illustrates seamless engine integration, in which... Figure 9 The interfaces in the API interface layer can be encapsulated into the MCP (Model Context Protocol) tool, which can then be used to call these interfaces to support remote operations. Remote AI operations can include adjusting parameters of visual style transfer models, etc., using the ReleaseSkin method. The ReleaseSkin method can also be used to adjust parameters of models called by other processing pipelines, such as adjusting parameters of image-generated models and GPT40 matting models.

[0131] The architecture of MCP tools is as follows Figure 10 As shown, the MCP tool can include an MCP protocol layer, a ComfyAI UI tool layer, an AI service layer, a resource management layer, and third-party tool integration. The MCP protocol layer is responsible for communication, and the ComfyAI UI tool layer is responsible for providing interfaces for each service. For example, the ComfyAIMCPToolEntryUISkinUI tool entry is a remote operation interface, which provides remote operation support to support remote visual style conversion triggering operations. Remote AI operation supports remote AI image generation operations through the MCP protocol, that is, generating configuration data (such as image data / image resources) for UI components. The tool interface standardization is used to provide standardized tool interfaces to support third-party tool integration. The ComfyAIMCPResourceUISkin resource management tool is a resource management interface, which supports skin resource operations to achieve unified management of skin resources.

[0132] The AI ​​service layer is responsible for providing various services. For example, the ComfyAI node system provides the calling interfaces for the ApplySkin, LoadSkin, and ReleaseSkin methods; the AI ​​image generation service provides configuration data generation services for UI components; the LoRA model service provides model parameter adjustment services; and the parameter configuration service provides version and other parameter configuration services for UI component configuration data. The resource management layer provides UI resource management, skin resource management, image resource management, and version control. UI resource management corresponds to PrefabUILayout layout resources, skin resource management corresponds to PrefabUILayoutSkin skin resources, and image resource management corresponds to data generated by the processing pipeline. Third-party tool integration refers to the externally visible tools of the MCP tool.

[0133] In some examples, the process of obtaining the UI structure information and UI component ID list from the PrefabUILayout layout resource is as follows: Figure 11 As shown, the following steps may be included:

[0134] S301. Parse the PSD file (Photoshop Document) to obtain layer information. The PSD file is a UI design file created according to user requirements. This embodiment can use at least two dynamically switchable parsing kernels to parse the PSD file. For example, the at least two dynamically switchable parsing kernels include: 1. A parsing kernel based on an open-source Python library, characterized by high cost-effectiveness and the ability to provide complete layer information extraction; 2. A high-performance parsing kernel based on a commercial-grade library, which has higher efficiency and stability when processing complex or large PSD files. This design allows developers to choose according to the needs of different stages of the project (such as the flexibility of the prototype verification stage and the stability of the final release stage), achieving a balance between cost and performance.

[0135] S302. Integrate the layer nesting relationships in the layer information to obtain the layer nesting relationship of a UI prefab. The layer nesting relationship is used to indicate the parent layer and child layer. A layer can be a UI component in the UI prefab. Therefore, the layer nesting relationship of the UI prefab can indicate the hierarchical structure of the UI prefab.

[0136] S303. Perform coordinate transformation on each layer in the layer information. The coordinate transformation is to convert the coordinates of the layer in the PSD coordinate system to the coordinate system used by the rendering engine in the electronic device. For example, if the rendering engine uses the Unity UI coordinate system, the coordinate transformation can ensure the accuracy of the position and size of the UI components, and the transformation error is controlled within 1 pixel.

[0137] The coordinate transformation process includes: obtaining the absolute coordinates of the layer in the PSD coordinate system. The absolute coordinates of the layer in the PSD coordinate system can be the normalized position of the layer's center point relative to the bounding box of its parent layer. The normalized position is a two-dimensional vector with a value between 0 and 1. Based on the layer's normalized position, the anchor point of the layer in the Unity UI coordinate system is determined. For example, a layer with a normalized position of (0.95, 0.9) has its anchor point set to the top right corner in the Unity UI coordinate system. After determining the anchor point of the layer in the Unity UI coordinate system, the pixel offset from the layer's center point to its anchor point is calculated. This pixel offset is used as the layer's AnchoredPosition (UI anchor point coordinates) in the Unity UI coordinate system.

[0138] S304. Identify the type of each layer in the layer information, and obtain the layer identifier based on the layer type. The layer identifier serves as the identifier for the UI component. Identifying the type of each layer in the layer information includes: pre-configuring regular expressions, matching the layer name against the regular expression, and if the layer name matches a regular expression, then the type corresponding to the regular expression is determined as the layer type. For example, if a layer is named Btn_Confirm_Normal, and this layer name matches a regular expression whose corresponding type is "button," then the layer type is Btn_Confirm_Normal (a button in its normal state), and the layer type is determined as the layer identifier.

[0139] If the layer name does not match a regular expression, and the layer is a layer group, the layer type is determined based on the layer hierarchy. For example, a layer hierarchy indicating that it contains a background shape layer and a text layer determines the layer type as a button or label. The layer is named according to a pre-configured regular expression that matches its type, resulting in a layer name. This layer name serves as the layer's identifier. The regular expression matching the layer's type can be a regular expression whose corresponding type is close to the layer's type.

[0140] If the layer name does not match the regular expression, and the layer is a single layer, pixel content analysis is performed on the layer to determine its type. For example, edge detection and color histogram analysis are performed on the layer to determine whether it is an icon or a background image, thereby maximizing recognition coverage and providing accurate input classification for subsequent processing. After determining the layer type, the layer is named according to the regular expressions in the pre-configured regular expressions that match the layer type, resulting in a layer name. This layer name serves as the layer's identifier. The regular expression that matches the layer type can be a regular expression whose corresponding type is close to the layer type.

[0141] S305. Extract metadata and image data for each layer from the layer information, such as layer opacity, blending mode, and effects. A layer can be considered a UI component; therefore, the layer's metadata and image data are the same as the UI component's metadata and image data. The coordinates in the metadata can use the layer's AnchoredPosition in the Unity UI coordinate system, fundamentally solving the problem of UI component misalignment caused by absolute coordinate transformations at different screen resolutions, and achieving pixel-level accurate reconstruction from PSD files to the rendering engine.

[0142] S306. Store the layer nesting relationship of the UI prefab, the identifiers of each UI component in the UI prefab, the metadata of each UI component in the UI prefab, and the preview image of the UI prefab in the PrefabUILayout layout resource. Specifically, the identifiers of each UI component in the UI prefab are recorded in the UI component ID list, and the layer nesting relationship and the metadata of each UI component in the UI prefab are recorded in the UI structure information.

[0143] from Figure 11 As can be seen, this embodiment can use a dynamically switchable parsing kernel to parse PSD files, supports more than 99% PSD file format compatibility, improves the efficiency of extracting layer information from PSD files by more than 20 times, and can automatically generate information corresponding to UI prefabs (such as hierarchical structure, UI component identifiers, etc.) based on layer information, greatly improving the conversion efficiency from design to development.

[0144] In some examples, the visual style transfer method provided in this embodiment can support multilingual functionality, such as... Figure 12As shown, resources in multiple languages ​​are provided, and localized configuration management is achieved through AudioGlobalConfiguration, which supports language settings, resource path configuration, and dynamic language addition. The TranslationUtil path conversion tool enables one-click switching between different language versions of resources, and the SyncLocaleResources resource synchronization function ensures resource synchronization across different language versions within the editor.

[0145] The above describes a visual style conversion method provided by the embodiments of this application. The following will describe a system for performing the above visual style conversion method.

[0146] Please see Figure 13 , Figure 13 This is a schematic diagram of the structure of a visual style transfer system provided in an embodiment of this application. Figure 13 As shown, the visual style conversion system may include: a first acquisition module 10, a second acquisition module 20, a third acquisition module 30, and a conversion module 40.

[0147] The first acquisition module 10 is used to acquire style description data input by the user. The style description data is used to indicate the style corresponding to the UI component to be processed after the style conversion is completed.

[0148] The second acquisition module 20 is used to acquire prompt data that matches the style description data. The prompt data is used to optimize the style conversion effect of the UI component to be processed.

[0149] The third acquisition module 30 is used to acquire the metadata and image data of the UI component to be processed. The metadata of the UI component to be processed is structured data used to describe the basic configuration of the UI component to be processed.

[0150] The conversion module 40 is used to input style description data, image data of the UI component to be processed, metadata and prompt data of the UI component to be processed into the visual style conversion model, and obtain the configuration data of the UI component to be processed output by the visual style conversion model. The configuration data of the UI component to be processed is used to obtain a UI component that matches the style indicated by the style description data without changing the control logic in the basic configuration of the UI component to be processed.

[0151] In one possible implementation, if the configuration data of the key UI component is obtained before obtaining the configuration data of the UI component to be processed, the conversion module 40 inputs the configuration data of the key UI component into the visual style conversion model.

[0152] In one possible implementation, the visual style transfer system may further include a sequence determination module and a parallel processing module. The sequence determination module is used to determine the processing order of each UI component to be processed based on its priority relationship and its dependency relationship.

[0153] The parallel processing module is used to process multiple UI components in parallel through multiple image generation tasks according to the processing order of each UI component. Each image generation task is used to call the visual style transfer model to process the UI component to be processed.

[0154] In one possible implementation, the parallel processing module is also used to perform at least one of the following:

[0155] If the image generation task fails due to a temporary error, a retry time is determined, and the task is re-executed within that time. Temporary errors indicate a possibility of success after re-execution. If the image generation task fails due to a permanent error, it is terminated. Permanent errors indicate that the task needs to be repaired before execution. If the UI component generated based on the configuration data of the UI component to be processed does not meet the preset quality requirements, a random seed is assigned to the UI component to be processed, triggering the image generation task. The image generation task of the UI component to be processed inputs the random seed into the visual style transfer model to regenerate the configuration data of the UI component to be processed.

[0156] In one possible implementation, the visual style transfer system further includes: a rendering processing module, used to obtain rendering data of UI components to be updated in the UI prefabs currently displayed on the electronic device, the rendering data of the UI components being obtained according to the configuration data of the UI components; drawing the UI prefabs in the UI according to the rendering data of the UI components to be updated in the UI prefabs; and after all UI prefabs in the UI have been drawn, the electronic device displays a UI that matches the style indicated by the style description data.

[0157] In one possible implementation, the rendering processing module obtains the rendering data of the UI component to be updated from the UI prefab, including: determining whether the configuration data corresponding to the resource identifier is stored in the cache of the electronic device based on the resource identifier of the UI component to be updated. The resource identifier is stored in a resource mapping table and can be found by looking up the identifier of the UI component to be updated in the UI prefab; if the configuration data corresponding to the resource identifier is stored in the cache, updating the rendering data of the UI component to be updated using the configuration data corresponding to the resource identifier; if the configuration data corresponding to the resource identifier is not stored in the cache, retrieving the configuration data corresponding to the resource identifier from the disk of the electronic device; storing the configuration data corresponding to the resource identifier retrieved from the disk into the cache, and updating the rendering data of the UI component to be updated using the configuration data corresponding to the resource identifier.

[0158] In one possible implementation, the transformation module is used to fine-tune the configuration data of the UI component to be processed based on the context data related to the UI component to be processed after obtaining the configuration data of the UI component to be processed output by the visual style transformation model. The context data includes the metadata of the UI component to be processed, the historical configuration data of the UI component to be processed, and the configuration data of the UI components adjacent to the UI component to be processed.

[0159] In this embodiment, please refer to the above method embodiment for the description of each module in the visual style transfer system, and it will not be repeated here.

[0160] This application also provides an electronic device in its embodiments. (See reference...) Figure 14 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 14 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0161] like Figure 14As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. When the electronic device is powered on, the RAM 403 also stores various programs and data required for the operation of the electronic device. The processing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404. The processing unit 401 can be considered as a processor in the electronic device, and the ROM 402, RAM 403, and storage device 408 can be considered as memory in the electronic device. The memory is used to store computer programs; the processor is used to execute the computer programs to implement any of the visual style transfer methods provided in the embodiments of this application.

[0162] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, memory cards, hard drives, etc.; and communication devices 409. Communication device 409 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 14 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0163] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the visual style conversion methods provided in this application.

[0164] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the visual style transfer methods provided in this application.

[0165] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0166] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0167] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0168] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A visual style transfer method, characterized in that, include: Obtain style description data input by the user, the style description data being used to indicate the style corresponding to the UI component to be processed after style conversion; Obtain prompt data that matches the style description data, the prompt data being used to optimize the style transition effect of the UI component to be processed; Obtain the metadata of the UI component to be processed and the image data of the UI component to be processed. The metadata of the UI component to be processed is structured data used to describe the basic configuration of the UI component to be processed. The style description data, the image data of the UI component to be processed, the metadata of the UI component to be processed, and the prompt data are input into the visual style conversion model to obtain the configuration data of the UI component to be processed output by the visual style conversion model. The configuration data of the UI component to be processed is used to obtain a UI component that matches the style indicated by the style description data without changing the control logic in the basic configuration of the UI component to be processed.

2. The method according to claim 1, characterized in that, The method further includes: if the configuration data of a key UI component is obtained before obtaining the configuration data of the UI component to be processed, then the configuration data of the key UI component is input into the visual style conversion model.

3. The method according to claim 1, characterized in that, The method further includes: determining the processing order of each UI component to be processed based on the priority relationship and the dependency relationship of each UI component to be processed; According to the processing order of each UI component to be processed, multiple image generation tasks are used to process multiple UI components to be processed in parallel. Each image generation task is used to call the visual style transfer model to process the UI component to be processed.

4. The method according to claim 3, characterized in that, The method further includes at least one of the following: If the image generation task fails due to a temporary error, a retry time for the image generation task is determined, and the image generation task is re-executed within the retry time. The temporary error is used to indicate that there is a possibility that the image generation task will succeed after re-execution. If the image generation task fails due to a permanent error, the image generation task is terminated. The permanent error is used to indicate that the image generation task needs to be repaired before execution. If the UI component generated based on the configuration data of the UI component to be processed does not meet the preset quality requirements, a random seed is assigned to the UI component to be processed, triggering the image generation task of the UI component to be processed. The image generation task of the UI component to be processed inputs the random seed into the visual style conversion model to call the visual style conversion model to regenerate the configuration data of the UI component to be processed.

5. The method according to claim 1, characterized in that, The method further includes: For the UI prefab in the UI currently displayed on the electronic device, obtain the rendering data of the UI component to be updated in the UI prefab, wherein the rendering data of the UI component is obtained according to the configuration data of the UI component; Based on the rendering data of the UI components to be updated in the UI prefab, the UI prefab is drawn in the UI; after all UI prefabs in the UI are drawn, the electronic device displays a UI that matches the style indicated by the style description data.

6. The method according to claim 5, characterized in that, The process of obtaining the rendering data of the UI components to be updated in the UI prefab includes: Based on the resource identifier of the UI component to be updated, determine whether the configuration data corresponding to the resource identifier is stored in the cache of the electronic device. The resource identifier is stored in the resource mapping table and can be found in the UI prefab through the identifier of the UI component to be updated. If the cache stores configuration data corresponding to the resource identifier, the rendering data of the UI component to be updated is updated using the configuration data corresponding to the resource identifier; If the configuration data corresponding to the resource identifier is not stored in the cache, the configuration data corresponding to the resource identifier is obtained from the disk of the electronic device; The configuration data corresponding to the resource identifier obtained from the disk is stored in the cache, and the rendering data of the UI component to be updated is updated using the configuration data corresponding to the resource identifier.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: after obtaining the configuration data of the UI component to be processed output by the visual style conversion model, fine-tuning the configuration data of the UI component to be processed according to the context data related to the UI component to be processed, wherein the context data includes the metadata of the UI component to be processed, the historical configuration data of the UI component to be processed, and the configuration data of UI components adjacent to the UI component to be processed.

8. A visual style transfer system, characterized in that, include: The first acquisition module is used to acquire style description data input by the user, wherein the style description data is used to indicate the style corresponding to the UI component to be processed after the style conversion is completed; The second acquisition module is used to acquire prompt data that matches the style description data, and the prompt data is used to optimize the style conversion effect of the UI component to be processed. The third acquisition module is used to acquire the metadata of the UI component to be processed and the image data of the UI component to be processed. The metadata of the UI component to be processed is structured data used to describe the basic configuration of the UI component to be processed. The conversion module is used to input the style description data, the image data of the UI component to be processed, the metadata of the UI component to be processed, and the prompt data into the visual style conversion model, and obtain the configuration data of the UI component to be processed output by the visual style conversion model. The configuration data of the UI component to be processed is used to obtain a UI component that matches the style indicated by the style description data without changing the control logic in the basic configuration of the UI component to be processed.

9. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the visual style transfer method as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the visual style transfer method as described in any one of claims 1 to 7.

Images