User interface generation method and apparatus, computer device, and storage medium

Abstract

A user interface generation method and apparatus, a computer device, and a storage medium, relating to the technical field of computers. The method comprises: in response to a generation instruction in an application, displaying a generation interface in the application, wherein the generation instruction indicates generation of a user interface for the application (201); acquiring a keyword inputted in the generation interface, wherein the keyword is used for describing a condition that a user interface to be generated needs to satisfy (202); generating a first user interface on the basis of the keyword, wherein the first user interface matches the keyword (203); and displaying the first user interface in the application (204). In the present application, a function of generating a user interface is provided in the application, so that personalization of the user interface is realized without being limited to user interfaces made by a designer, thereby enhancing the interestingness of the user interface, improving the user viscosity of the application. In addition, there is no need for a designer to make different user interfaces, thereby saving labor costs and improving the efficiency of generating a user interface.

Description

User interface generation method, device, computer equipment and storage medium

[0001] This application claims priority to the Chinese patent application filed on November 17, 2023, with application number 202311550312.5 and invention name “User Interface Generation Method, Device, Computer Equipment and Storage Medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of computer technology, and in particular to a method and apparatus for generating a user interface, a computer device, and a storage medium. Background Art

[0003] With the advancement of computer technology and the emergence of a wide variety of applications, users are increasingly concerned about the display quality of user interfaces (UIs) within applications. Creating UIs that meet these requirements has become a key focus. UIs are typically created by dedicated designers and then added to applications. These applications are then released to users, who then display the UI by running the application.

[0004] Summary of the Invention

[0005] The present application provides a method, apparatus, computer device, and storage medium for generating a user interface. The technical solution is as follows:

[0006] On the one hand, a user interface generation method is provided, the method comprising: displaying a generation interface in the application in response to a generation instruction in an application, the generation instruction instructing to generate a user interface for the application; obtaining keywords input in the generation interface, the keywords being used to describe conditions that need to be met by the user interface to be generated; generating a first user interface based on the keywords, the first user interface matching the keywords; and displaying the first user interface in the application.

[0007] On the other hand, a user interface generation device is provided, which includes: a display module for displaying a generation interface in the application in response to a generation instruction in the application, wherein the generation instruction indicates generation of a user interface for the application; an acquisition module for acquiring keywords input in the generation interface, wherein the keywords are used to describe conditions that the user interface to be generated needs to meet; a generation module for generating a first user interface based on the keywords, wherein the first user interface matches the keywords; and the display module for displaying the first user interface in the application.

[0008] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the user interface generation method described in the above aspects.

[0009] On the other hand, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the operations performed by the user interface generation method described in the above aspects.

[0010] On the other hand, a computer program product is provided, including a computer program, wherein the computer program is loaded and executed by a processor to implement the operations performed by the user interface generation method as described in the above aspects.

[0011] The solution of the embodiment of the present application provides a user interface generation function within the application. When a computer device is running the application, a user interface matching the keywords can be generated based on the input keywords. This achieves user interface personalization, eliminating the need to use designer-created user interfaces. This enhances the user experience of the user interface and improves user engagement with the application. Furthermore, it eliminates the need for designers to create numerous different user interfaces, saving labor costs and improving the efficiency of user interface generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] FIG1 is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0013] FIG2 is a flow chart of a method for generating a user interface provided in an embodiment of the present application;

[0014] FIG3 is a flowchart of another method for generating a user interface provided by an embodiment of the present application;

[0015] FIG4 is a schematic diagram of a generation interface provided in an embodiment of the present application;

[0016] FIG5 is a schematic diagram of another generation interface provided in an embodiment of the present application;

[0017] FIG6 is a schematic diagram of another generation interface provided in an embodiment of the present application;

[0018] FIG7 is a schematic diagram of another generation interface provided in an embodiment of the present application;

[0019] FIG8 is a flowchart of another method for generating a user interface provided by an embodiment of the present application;

[0020] FIG9 is a schematic diagram of a first image generation model provided in an embodiment of the present application;

[0021] FIG10 is a schematic diagram of another first image generation model provided in an embodiment of the present application;

[0022] FIG11 is a flowchart of another method for generating a user interface provided in an embodiment of the present application;

[0023] FIG12 is a schematic diagram of a line image provided in an embodiment of the present application;

[0024] FIG13 is a schematic diagram of a depth image provided in an embodiment of the present application;

[0025] FIG14 is a schematic diagram of another first image generation model provided in an embodiment of the present application;

[0026] FIG15 is a schematic diagram of an image processing process in a first model provided in an embodiment of the present application;

[0027] FIG16 is a schematic diagram of different images generated based on the same line image according to an embodiment of the present application;

[0028] FIG17 is a schematic diagram of different images generated based on the same depth image provided by an embodiment of the present application;

[0029] FIG18 is a schematic diagram of different images generated based on the same color image according to an embodiment of the present application;

[0030] FIG19 is a schematic diagram of an operation flow for generating a first user interface provided by an embodiment of the present application;

[0031] FIG20 is a schematic structural diagram of a user interface generating device provided in an embodiment of the present application;

[0032] FIG21 is a schematic structural diagram of another user interface generating device provided in an embodiment of the present application;

[0033] FIG22 is a schematic structural diagram of a terminal provided in an embodiment of the present application;

[0034] Figure 23 is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the following will further describe the implementation methods of the present application in detail with reference to the accompanying drawings. It should be noted that the information involved in this application (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) are all fully authorized by the user or relevant parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0036] In the related art, the user interface is usually created by a dedicated designer and then added to the application. The application is then released to the user, and the user displays the user interface by running the application. However, the user interfaces created by the designer are the same for different users, which lacks interest and results in low user stickiness of the application. Based on this, the embodiment of the present application provides an interface generation method that can improve the user stickiness of the application, without the need for designers to create many different user interfaces, thereby improving the efficiency of generating user interfaces. The interface generation method provided in the embodiment of the present application is described in detail below.

[0037] First, the terms involved in the embodiments of this application are explained as follows:

[0038] CLIP (Contrastive Language-Image Pre-Training) encoder: A pre-training model that compares text and images, with the purpose of linking text and images. The embodiments of this application mainly use the text encoding function of CLIP to convert text into text features so that the text features can be input into the model for image generation.

[0039] Text features: A string of digital codes used to describe the nature, attributes, structure, and other information of text. Since computer devices cannot recognize text, text is converted into text features. Computer devices can recognize text features and perform subsequent image generation processes based on text features.

[0040] Image features: A string of digital codes used to describe the size, color distribution, outline, texture and other information of an image. Since computer devices cannot recognize images, images are converted into image features. Computer devices can recognize images and perform subsequent image generation processes based on image features.

[0041] VAE (Variational Auto Encoder): A generative model based on probabilistic coding, consisting of an encoder and a decoder. The encoder is used to convert an image into image features in a latent space, and the decoder is used to convert image features in the latent space into pixel images.

[0042] DDPM (Diffusion Probabilistic Models): A generative model based on the diffusion model that generates data by diffusion in a high-dimensional space. It can simulate complex probability distributions and dynamically change the model structure during training to better fit the data.

[0043] Predicting noise: In the process of removing noise from an image, it starts with random noise. The DDPM algorithm is used to determine the noise image. The image obtained by removing the noise image from the original image is the generated new image.

[0044] FIG1 is a schematic diagram of an implementation environment provided by an embodiment of the present application. Referring to FIG1 , the implementation environment includes: a terminal 101 and a server 102 , and the terminal 101 and the server 102 are connected via a wireless or wired network.

[0045] The terminal 101 runs the application, and the server 102 is associated with the application, thereby providing data services for the application. The application is of various types such as an instant messaging application, a game application, or a resource recommendation application, and this embodiment of the present application does not limit this. Considering that if the user interface of the application is made by a designer, the user interfaces seen by different users will be the same, which lacks interest, the embodiment of the present application adds a function of customizing the user interface to the application. That is, after the terminal 101 runs the application, the user can trigger a generation instruction in the application and enter a keyword to generate a user interface that matches the keyword, without being limited to using the user interface made by the designer. Then, different users can generate personalized user interfaces when using the terminal 101 to run the application, without having to use the same user interface.

[0046] The user interface generation method provided in the embodiment of the present application is used in a computer device. Optionally, the computer device is a terminal 101 or a server 102.

[0047] In a possible implementation, the computer device is a terminal 101, and the terminal 101 generates a user interface through an application.

[0048] Optionally, the terminal 101 is a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, an intelligent voice interaction device, a smart home appliance, a vehicle terminal, an aircraft, etc., but is not limited thereto. The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc.

[0049] In another possible implementation, the computer device includes a terminal 101 and a server 102. After the terminal 101 runs the application, the user triggers a generation instruction in the application and enters a keyword. The terminal 101 uploads the keyword to the server 102. The server 102 generates a user interface and returns it to the terminal 101. The terminal 101 can then display the user interface through the application.

[0050] Optionally, the server 102 is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0051] In one possible implementation, the computer program involved in the embodiments of the present application can be deployed and executed on a computer device, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected through a communication network. Multiple computer devices distributed at multiple locations and interconnected through a communication network can constitute a blockchain system.

[0052] In one possible implementation, the computer device used to generate the user interface in the embodiment of the present application is a node in the blockchain system. The node can store the generated user interface in the blockchain, and then the node or the node corresponding to other devices in the blockchain can query the user interface by accessing the blockchain.

[0053] The user interface generation method provided in the embodiment of the present application can be applied to various scenarios of running applications.

[0054] For example, in a game scenario, a designer of a game application creates a user interface file that specifies the style of the user interface within the game application, adds the user interface file to the game application, and then publishes the game application. After players download and install the game application, they can play the game through the game application, and the user interface displayed within the game is the user interface created by the designer. However, players can also use the method provided in the embodiments of the present application to generate a personalized user interface in the game application. Then, when players play the game through the application, the user interface displayed within the game is the user-defined user interface. This provides a highly personalized and customized experience for players, and players can use their own creativity to create a variety of user interfaces.

[0055] FIG2 is a flow chart of a method for generating a user interface provided by an embodiment of the present application. The embodiment of the present application is executed by a computer device. Referring to FIG2 , the method includes:

[0056] 201. A computer device displays a generation interface in the application in response to a generation instruction in the application, where the generation instruction instructs generation of a user interface for the application.

[0057] In an embodiment of the present application, when a computer device is running an application, the user of the computer device is the user of the application. The user triggers a generation instruction in the application, and the generation instruction instructs the generation of a user interface (UI) for the application. The generation interface is used for the user to set keywords required to generate the user interface.

[0058] The generation instruction can be generated by any operation in the application. In one possible implementation, a user interface generation control is displayed in the application, and the user interface generation control is used to request the generation of a user interface for the application. The generation instruction is generated by a trigger operation on the user interface generation control, and the computer device displays the generation interface in response to the trigger operation on the user interface generation control. The trigger operation is a click operation, a double-click operation, etc. The user interface generation control is located in the main interface of the application, or in other interfaces of the application. This embodiment of the application does not limit this.

[0059] 202. The computer device obtains keywords input in the generation interface. The keywords are used to describe conditions that the user interface to be generated needs to meet.

[0060] Keywords are used to describe the conditions that need to be met for the user interface to be generated. Users can enter their own keywords in the generated interface based on their own needs to generate a user interface that matches the keywords. Different keywords are entered for different users, and different user interfaces are generated, thereby achieving personalized user interfaces. In addition, because the process of generating the user interface in the embodiments of the present application is intelligent rather than user-designed, the generated user interface has a certain degree of randomness. Therefore, even if the keywords entered are the same, the computer device can generate different user interfaces based on the same keywords.

[0061] 203. The computer device generates a first user interface based on the keyword, where the first user interface matches the keyword.

[0062] For the sake of distinction, the user interface that matches the keyword generated this time is referred to as the first user interface in the embodiment of the present application. The steps of the embodiment of the present application can be performed multiple times to generate multiple user interfaces.

[0063] Optionally, after the computer device generates the first user interface, it closes the generation interface. When the generation instruction is detected again, steps 201-203 are re-executed to generate a new user interface, and new keywords are input into the generation interface, thereby generating a user interface that matches the new keywords. Alternatively, if the computer device generates the first user interface and does not close the generation interface, the keywords can be re-entered into the generation interface to generate a user interface that matches the new keywords. Alternatively, if the computer device generates the first user interface and does not close the generation interface, the keywords originally input can be kept unchanged, triggering a regeneration control in the generation interface to generate another user interface that matches the keywords.

[0064] In an embodiment of the present application, a user interface file is provided in the application. The user interface file represents the style of the user interface, such as the shape, color, size, layout and spacing of multiple display elements, etc. During the operation of the application, the user interface file is displayed based on the user interface file, thereby displaying a visual user interface. Then, the computer device generates a user interface that matches the keyword, which means that the computer device generates a user interface file that matches the keyword, thereby ensuring that the user interface that matches the keyword can be displayed based on the user interface file during the subsequent operation of the application. The user interface file is in PNG (Portable Network Graphics) format or other formats, which is not limited in the embodiment of the present application.

[0065] Optionally, different user interface files are set for different functions in the application, and the user interface generation method provided in the embodiment of the present application can be applied to any function. For example, the application is a game application, and the game application is provided with a game function and a non-game function, and the game user interface file set by the game application represents the style of the user interface within the game, and the non-game user interface file represents the style of the user interface outside the game, such as the game application main interface, the game application account information display interface, etc. The style of the user interface within the game has a greater impact on the user's game operation, while the style of the user interface outside the game has little impact on the user's game operation. Therefore, the function of customizing the user interface can be enabled for the game function, and the function of customizing the user interface can be disabled for the non-game function. In this case, the generation instruction in the embodiment of the present application indicates that a user interface file is generated for the game function of the application, then the computer device will generate a game user interface file that matches the keyword, and keep the original non-game user interface file unchanged.

[0066] Optionally, the computer device is a terminal. The terminal generates a user interface file by executing steps 201-203. The terminal then validates the user interface file, and subsequent application execution can be based on the user interface file. For example, if the terminal generates multiple user interface files, the last generated user interface file can be validated by default, or the user can select any user interface file to be validated from the multiple user interface files.

[0067] In a possible implementation, after the terminal generates the user interface file, the terminal uploads the user interface file to the server through the application, and the server stores the account logged in by the terminal and the user interface file.

[0068] Optionally, the computer device includes a terminal and a server. After the terminal runs the application, the user triggers a generation instruction in the application and enters a keyword. The terminal uploads the keyword to the server. The server generates a user interface file matching the keyword and returns it to the terminal. The terminal can then make the user interface file effective through the application, thereby displaying a user interface matching the keyword based on the user interface file.

[0069] In another possible implementation, the server stores the terminal login account and the user interface file.

[0070] In another possible implementation, the computer device displays a user interface ranking list, where the user interface ranking list includes multiple user interfaces, and each user interface in the user interface ranking list is generated by the computer device running the application.

[0071] Optionally, the user interface ranking list includes a target number of user interfaces, which can be a pre-set number. That is, after at least one computer device running the application generates a user interface, it can evaluate multiple user interfaces, select a target number of user interfaces with better display effects, create a user interface ranking list, and publish it to the application. The computer device running the application can then display the user interface ranking list in the application. Moreover, each user interface in the user interface ranking list is generated by the computer device running the application, indicating that the selected user interface is not made by the designer of the application, but is customized by the user when using the application. Through the user interface ranking list, many users of the application can view the generated user interfaces with rich styles, thereby attracting more users to participate in the activity of customizing user interfaces. Among them, the user interfaces on the list can be selected by the operator of the application, or by user voting, and the embodiments of the present application do not limit this.

[0072] 204. The computer device displays the first user interface in the application.

[0073] In one possible implementation, after generating the first user interface, the computer device displays the first user interface in an application in response to an application instruction for the first user interface, thereby using the personalized first user interface.

[0074] The related technology is that during the application development process, the designer creates a user interface for the application. After the creation is completed, the user interface is added to the application and then the application is released. After the computer device downloads the application, it runs the application to display the application user interface. This operation process is very long and consumes a lot of manpower costs. The application user interface is the same for different users. Even if the designer creates different user interfaces for users to choose from, the range of user options is limited. Moreover, the display effect of the user interface depends on the designer, but the designer's creativity is limited, which leads to limitations in the user interface created by the designer.

[0075] The solution of the embodiment of the present application provides a user interface generation function within the application. When a computer device is running the application, a user interface matching the keyword input can be generated based on the keyword input. This achieves user interface personalization, eliminating the need to use designer-created user interfaces. This enhances the user interface's fun and improves user engagement with the application. Furthermore, it eliminates the need for designers to create numerous different user interfaces, saving labor costs and improving the efficiency of user interface generation.

[0076] Based on the embodiment shown in FIG. 2 , another embodiment of the present application further provides another method for generating a user interface. FIG. 3 is a flow chart of another method for generating a user interface provided by an embodiment of the present application. The embodiment of the present application is executed by a computer device. Referring to FIG. 3 , the method includes:

[0077] 301. The computer device displays an interface image of a second user interface and an input interface in a generated interface in response to a generation instruction in the application. The second user interface is a user interface currently used by the application, and the input interface is used to input keywords.

[0078] The generation instruction instructs the application to generate a user interface. The generated interface includes an interface image of a second user interface and an input interface. The input interface is used to input keywords. The second user interface is the user interface currently used by the application. The interface image of the second user interface refers to the image when the application displays the second user interface. The difference between the second user interface and the interface image of the second user interface is that when the application displays the second user interface, various operations can be performed based on the second user interface, such as clicking controls in the second user interface or responding to keyboard input commands when the second user interface is displayed. However, the interface image of the second user interface is only an image and cannot be operated based on the interface image of the second user interface. For example, the computer device will not respond to keyboard input commands when clicking controls in the interface image of the second user interface. The interface image of the second user interface can show the display effect of the application when using the second user interface. By displaying the interface image of the second user interface in the generated interface, the user can preview the display effect of the second user interface to determine whether they are satisfied with the display effect of the second user interface. The second user interface currently used by the application can be a user interface created by a designer or a user interface previously generated by the computer device.

[0079] In one possible implementation, the computer device generates an interface image of the second user interface based on the currently used user interface file. Since the currently used user interface file indicates the style of the second user interface, the interface image of the second user interface generated based on the user interface file can show the display effect when the application uses the second user interface.

[0080] In one possible implementation, the computer device divides the generated interface into a first display area and a second display area, displays the interface image of the second user interface in the first display area, and displays the input interface in the second display area. Alternatively, the computer device displays the interface image of the second user interface at the bottom layer of the generated interface and displays the input interface on top of the interface image of the second user interface. The input interface can be a transparent interface to avoid obstructing the interface image of the second user interface, or the input interface can be an opaque interface with a size smaller than the size of the interface image of the second user interface to ensure that the input interface only obstructs a portion of the interface image of the second user interface, and does not obstruct the entire area of ​​the interface image of the second user interface.

[0081] For example, the generation interface is shown in FIG4 , in which an interface image 401 of the second user interface and an input interface 402 are displayed. The input interface 402 is located on an upper layer of the interface image 401 of the second user interface.

[0082] 302. The computer device obtains keywords input in the input interface. The keywords are used to describe conditions that need to be met by the user interface to be generated.

[0083] The user enters a keyword in an input interface of the generated interface. After the computer device obtains the keyword based on the generated interface, it can generate a user interface matching the keyword.

[0084] In one possible implementation, referring to FIG4 , the input interface displays a generation control 403. After the user completes entering the keyword, the generation control 403 is triggered. The computer device responds to the triggering operation of the generation control 403 and obtains the keyword entered in the input interface to generate a user interface that matches the keyword.

[0085] In another possible implementation, referring to FIG4 , the input interface also displays an exit control 404. The user can trigger the exit control 404 before entering a keyword, or after entering a keyword and wanting to cancel the generation of the user interface. The computer device responds to the triggering operation of the exit control 404 by canceling the display of the generation interface, or by keeping the generation interface displayed and canceling the display of the input interface in the generation interface.

[0086] In another possible implementation, step 302 includes:

[0087] 3021. The computer device obtains the forward keyword input in the first input field of the input interface.

[0088] The computer device displays a first input bar on the input interface, and obtains forward keywords based on the input operation in the first input bar. The forward keywords are keywords that the user interface to be generated needs to be associated with. A first input bar is displayed in the input interface, and the first input bar is used to input forward keywords. Forward keywords are keywords that the user interface to be generated needs to be associated with, that is, the user interface that the user requires to be generated needs to be a user interface associated with the forward keywords. For example, the user interface to be generated displays display elements associated with the forward keywords, or the style of the user interface to be generated conforms to the style indicated by the forward keywords. For example, as shown in Figure 5, a first input bar 501 is displayed in the input interface, and the user enters multiple forward keywords in the first input bar 501 to indicate that the user interface to be generated needs to be associated with the multiple forward keywords. For example, if the forward keywords include "sports", a sports style will be presented in the user interface to be generated.

[0089] In the embodiment of the present application, a generation interface includes an input interface, and the input interface displays a first input field as an example. In other embodiments, the first input field can be located at any position in the generation interface, and the computer device obtains the forward keyword entered in the first input field of the generation interface.

[0090] In another possible implementation, step 302 further includes:

[0091] 3022. The computer device obtains the negative keyword input in the second input field of the input interface.

[0092] The computer device displays a second input bar on the generation interface, and based on the input operation in the second input bar, obtains negative keywords. Negative keywords are keywords that the user interface to be generated does not need to be associated with. A second input bar is displayed in the input interface, and the second input bar is used to enter negative keywords. Negative keywords are keywords that the user interface to be generated does not need to be associated with, that is, the user interface to be generated requires a user interface that is not associated with the negative keyword. For example, the display elements associated with the negative keyword are not displayed in the user interface to be generated, or the style of the user interface to be generated does not conform to the style indicated by the negative keyword. For example, as shown in Figure 5, a second input bar 502 is displayed in the input interface, and the user enters multiple negative keywords in the second input bar 502 to indicate that the user interface to be generated needs to be unassociated with the multiple negative keywords. For example, if the negative keyword includes "miss", the user interface to be generated does not contain game scenes related to "miss".

[0093] In the embodiment of the present application, a generation interface includes an input interface, and the input interface displays a second input field as an example. In other embodiments, the second input field can be located at any position in the generation interface, and the computer device obtains the negative keyword entered in the second input field of the generation interface.

[0094] In another possible implementation, the computer device may further perform the following steps 3023:

[0095] 3023. The computer device obtains a correlation degree represented by a position of a slider in a slider bar of an input interface.

[0096] The computer device displays a slider bar on a generation interface. The slider bar includes a slider block. The position of the slider block represents the degree of association between the user interface to be generated and the positive keyword. The degree of association represented by the position of the slider block is obtained based on a sliding operation on the slider block. The input interface also displays a slider bar with a slider block. The position of the slider block in the slider bar represents the degree of association between the user interface to be generated and the positive keyword, that is, the degree of association between the user interface to be generated and the positive keyword. The degree of association can be expressed in numerical form, as a percentage, or in other forms. A higher degree of association indicates a stronger association between the generated user interface and the positive keyword, while a lower degree of association indicates a weaker association between the generated user interface and the positive keyword. A user can change the position of the slider within the slider bar by dragging the slider, thereby changing the degree of association. For example, a lower degree of association results in smaller display elements associated with the positive keyword in the generated user interface, while a higher degree of association results in larger display elements associated with the positive keyword in the generated user interface. Alternatively, the smaller the correlation, the more display elements in the to-be-generated user interface that conform to the style of the positive keyword; the larger the correlation, the fewer display elements in the to-be-generated user interface that conform to the style of the positive keyword.

[0097] In one possible implementation, the degree of association is displayed in the display area of ​​the slider, allowing the user to intuitively view the magnitude of the degree of association. For example, as shown in FIG5 , a slider bar 503 is displayed in the generated interface, and the position of a slider 504 in the slider bar 503 represents the degree of association. A slider 504 located at the far left end of the slider bar 503 represents a degree of association of 0, while a slider 504 located at the far right end of the slider bar 503 represents a degree of association of 10. The position of the slider 504 in FIG5 represents a degree of association of 6.

[0098] In the embodiment of the present application, a generated interface includes an input interface, and the input interface displays a slider as an example. In other embodiments, the slider can be located at any position in the generated interface, and the computer device obtains the correlation corresponding to the position of the slider in the slider of the generated interface.

[0099] 303. The computer device generates a first user interface based on the keyword, where the first user interface matches the keyword.

[0100] For example, as shown in Figures 5 and 6, the prompt text displayed in the generation control 403 is "Generate". After the user enters the keyword and clicks the generation control 403, the computer device starts to generate the first user interface, and the prompt text displayed in the generation control 403 is changed to "Generating", and the progress prompt text "Generating...20%" for generating the first user interface is also displayed above the generation control 403. Moreover, during this process, if the user clicks the exit control 404, the generation of the first user interface stops. Alternatively, as shown in Figure 7, after the generation of the first user interface is completed, the prompt text displayed in the generation control 403 is changed to "Generate Again". If the user clicks the generation control 403 afterwards, the computer device will generate another user interface matching the keyword again.

[0101] In one possible implementation, when the keyword includes a positive keyword, the computer device generates an interface image associated with the positive keyword, and then generates the first user interface based on the interface image. For example, the interface image includes display elements (controls, colors, etc.) associated with the positive keyword, or the style of the interface image matches the style of the positive keyword.

[0102] In another possible implementation, when the keywords include negative keywords, the computer device generates an interface image associated with the positive keywords and not associated with the negative keywords, and then generates the first user interface based on the interface image. For example, the interface image includes display elements associated with the positive keywords but not display elements associated with the negative keywords, or the style of the interface image matches the style of the positive keywords but not the style of the negative keywords.

[0103] In another possible implementation, when the computer device obtains the positive keyword and the association degree, the computer device generates an interface image associated with the positive keyword and having an association degree equal to the association degree, and then generates the first user interface based on the interface image. For example, the interface image includes a display element associated with the positive keyword, and the size of the display element matches the size indicated by the association degree.

[0104] 304. After generating the first user interface, the computer device replaces the interface image of the second user interface with the interface image of the first user interface.

[0105] The interface image of the first user interface is the image when the application displays the first user interface. The interface image of the first user interface can show the display effect when the application uses the first user interface. The difference between the first user interface and the interface image of the first user interface is that: when the application displays the first user interface, various operations can be performed based on the first user interface, such as clicking on the controls in the first user interface, or responding to keyboard input instructions when the first user interface is displayed, while the interface image of the first user interface is only an image and cannot be operated based on the interface image of the first user interface, such as clicking on the controls in the interface image of the first user interface or the computer device will not respond when keyboard input instructions are received. By replacing the interface image of the second user interface with the interface image of the first user interface, the user can preview the display effect of the first user interface and can also intuitively view the difference between the first user interface and the second user interface.

[0106] The solution of the embodiment of the present application provides a user interface generation function within the application. When a computer device is running the application, a user interface matching the keyword input can be generated based on the keyword input. This achieves user interface personalization, eliminating the need to use designer-created user interfaces. This enhances the user interface's fun and improves user engagement with the application. Furthermore, it eliminates the need for designers to create numerous different user interfaces, saving labor costs and improving the efficiency of user interface generation.

[0107] In addition, before generating the first user interface, the interface image of the second user interface is displayed in the generation interface. This allows the user to preview the display effect of the second user interface and determine whether they are satisfied with the display effect of the second user interface. Furthermore, after generating the first user interface, the interface image of the second user interface in the generation interface is replaced with the interface image of the first user interface. This allows the user to preview the display effect of the first user interface and intuitively identify the differences between the first and second user interfaces. Furthermore, by entering a positive keyword in the generation interface, a user interface associated with the positive keyword is generated. This allows the user to specify which user interface to generate, thereby satisfying the user's user interface requirements. Furthermore, by entering different positive keywords, different user interfaces can be generated, increasing the diversity and interest of the user interfaces, thereby improving the user stickiness of the application. Furthermore, by entering a negative keyword in the generation interface, a user interface not associated with the negative keyword is generated. This allows the user to specify which user interface not to generate, thereby satisfying the user's user interface requirements. Furthermore, by entering different negative keywords, different user interfaces can be generated, increasing the diversity and interest of the user interfaces, thereby improving the user stickiness of the application. Furthermore, by entering a positive keyword and its relevance in the generation interface, a user interface associated with the positive keyword and having a relevance equal to the positive keyword is generated, thus meeting the user's requirements for a user interface. Furthermore, different user interfaces can be generated by entering different relevances, increasing the diversity and interest of user interfaces and thus improving user stickiness of the application.

[0108] In another possible implementation, step 303 includes: generating a target interface image based on the interface image and keywords of the second user interface through a first image generation model, and generating the first user interface based on the target interface image.

[0109] Among them, the second user interface is the user interface currently used by the application. The first image generation model is used to generate images, and the interface image and keywords of the second user interface are input into the first image generation model to generate a target interface image. The interface image and keywords of the second user interface are two important input contents of the first image generation model, which can ensure that the generated target interface image not only matches the keywords, but also contains part of the image information of the interface image of the second user interface, and will not differ too much from the interface image of the second user interface. In one possible implementation method, the interface image of the first user interface mentioned in the above step 304 is also the target interface image generated by the first image generation model.

[0110] The embodiment of the present application utilizes artificial intelligence technology to directly generate a user interface through an image generation model, allowing users to use their own creativity to create a variety of user interfaces, thereby improving the user experience, increasing the user stickiness of the application, and saving development costs.

[0111] Optionally, the application in the embodiments of the present application is a game application. In the related art, the UI of the game application requires designers to spend a lot of time to create, and players basically do not have a solution to choose the UI. Even if the designer makes several sets of UI for players to choose from, the players' range of choices is limited and the number is relatively small. These game applications do not use models to create game UIs, and cannot allow players to customize the UI. The embodiments of the present application utilize the combination of artificial intelligence technology and game UIs to allow players to directly generate UIs in game applications through image generation models. This is a highly personalized and customized experience for players. Players can use their own creativity to create a variety of UIs, customize their favorite buttons, interface themes, etc., and have more variety and richer options, which enhances the player's gaming experience and game fun. It also brings more activities and gameplay to the game application, improves the activity of the game application, improves the user stickiness of the game application, and saves the development cost of the game application, which is very beneficial to both the game application and the players.

[0112] This embodiment of the present application also provides another user interface generation method, which details the process of generating a first user interface using a first image generation model. Figure 8 is a flow chart of another user interface generation method provided by this embodiment of the present application. This embodiment of the present application is executed by a computer device. Referring to Figure 8, the method includes:

[0113] 801. A computer device displays a generation interface in the application in response to a generation instruction in the application, where the generation instruction instructs generation of a user interface for the application.

[0114] 802. The computer device obtains keywords input in the generation interface. The keywords are used to describe conditions that the user interface to be generated needs to meet.

[0115] The process of steps 801-802 is the same as that of steps 201-202, and will not be repeated here.

[0116] 803. The computer device encodes the interface image of the second user interface to obtain image features.

[0117] In the embodiment of the present application, although the computer device needs to generate a new first user interface, in order to ensure the normal operation of the application, the first user interface needs to be able to implement the functions originally set in the second user interface of the application. This requires that the difference between the first user interface and the second user interface cannot be too large, so when generating the second user interface, the interface image of the second user interface also needs to be considered. Therefore, the computer device encodes the interface image of the second user interface to obtain image features, and then performs dimensionality reduction processing on the interface image of the second user interface. In this way, the interface image of the second user interface can be reduced from the previous relatively large pixel level to a smaller size image feature, and the image features are input into the first image generation model so that the image features participate in the model calculation process.

[0118] Optionally, the computer device encodes the interface image of the second user interface through VAE to obtain image features, or encodes the interface image of the second user interface through a CLIP encoder to obtain image features, or other encoding methods can be used to encode the interface image of the second user interface. This embodiment of the present application is not limited to this.

[0119] 804. The computer device encodes the keywords to obtain text features.

[0120] In order to generate a first user interface matching the keyword, the keyword needs to be encoded to obtain text features, and then the text features are input into the first image generation model so that the text features participate in the model operation process.

[0121] Optionally, the computer device encodes the keywords through VAE to obtain text features, or encodes the keywords through CLIP encoder to obtain text features, or other encoding methods can be used to encode the keywords, which is not limited in the embodiments of the present application.

[0122] In one possible implementation, when the keywords include positive keywords, the positive keywords are encoded to obtain text features. In one possible implementation, when the keywords include positive keywords and negative keywords, the positive keywords and negative keywords are encoded to obtain text features. In another possible implementation, when the computer device obtains positive keywords and correlation, since the correlation itself is represented in the form of features, it can be directly input into the first image generation model without encoding. Therefore, the computer device encodes the positive keywords to obtain text features, and inputs the text features and correlation into the first image generation model. In another possible implementation, when the computer device obtains positive keywords, negative keywords and correlation, the computer device encodes the positive keywords and negative keywords to obtain text features, and then inputs the text features and correlation into the first image generation model.

[0123] Of course, in other possible implementations, when the relevance is obtained, the relevance may also be encoded so that the encoded text features include the relevance.

[0124] 805. The computer device encodes the text features and the image features through the first encoding sub-model to obtain encoding features.

[0125] In this embodiment of the present application, the first image generation model includes a first encoding sub-model, which is used to encode arbitrary features. The computer device inputs text features and image features into the first encoding sub-model, which encodes the text features and image features and outputs encoded features.

[0126] In one possible implementation, once the correlation is obtained, the computer device encodes the image features, text features, and correlation using a first encoding sub-model to generate encoded features. In the first image generation model, the text features and image features interact with each other, and the correlation influences parameters in the first image generation model during this interaction, thereby controlling the generated target interface image.

[0127] 806. The computer device decodes the encoded features through the decoding sub-model to obtain the target interface image.

[0128] In the embodiment of the present application, the first image generation model includes a decoding sub-model, which is used to decode any feature. The computer device inputs the encoded feature into the decoding sub-model, which decodes the encoded feature and outputs the target interface image.

[0129] In the above steps 803-806, the target interface image is generated based on the interface image and keywords of the second user interface through the first image generation model. In one possible implementation, as shown in Figure 9, the first image generation model includes a first encoding sub-model and a decoding sub-model, and the computer device inputs the image features and text features into the first image generation model. After encoding through the first encoding sub-model, the encoded features are decoded through the decoding sub-model to generate a new image, namely the target interface image. Using the model structure shown in Figure 9, since the process of generating the target interface image is affected by the interface image of the second user interface, it can be ensured that the generated target interface image will not be too different from the interface image of the second user interface, and will not affect the normal operation of the application. Since the process of generating the target interface image is affected by the keywords, it can be ensured that the generated target interface image is an image that matches the keywords and can meet the user's requirements.

[0130] 807. The computer device generates a first user interface based on the target interface image.

[0131] After the computer device generates the target interface image, it converts the target interface image into a user interface file. The user interface file represents the style of the first user interface, which is equivalent to the computer device generating the first user interface. The first user interface can be displayed based on the user interface file during the subsequent application operation.

[0132] In one possible implementation, a computer device recognizes a target interface image, obtains multiple interface elements and their respective positions in the target interface image, and generates a first user interface based on the multiple interface elements and their respective positions. Alternatively, the computer device obtains an interface template and populates the interface template with the multiple interface elements according to their positions to obtain the first user interface.

[0133] In one possible implementation, the computer device includes a terminal and a server. The terminal executes the above steps 801-802, and the server executes the above steps 803 to 806. After generating the target interface image, the file generation server generates a user interface file based on the target interface image, that is, step 807 includes: the server sends the target interface image to the file generation server, the file generation server converts the target interface image into a user interface file, and returns it to the server. The server receives the user interface file, sends the user interface file to the terminal, and the terminal stores the user interface file, so that the first user interface is displayed based on the user interface file during the application operation.

[0134] In one possible implementation, as shown in FIG10 , the first image generation model further includes a second encoding sub-model and a convolution sub-model. Then, as shown in FIG11 , the target interface image is generated based on the interface image and keywords of the second user interface through the first image generation model, and the following steps 808 and 809 are also included.

[0135] 808. The computer device encodes the reference interface image to obtain conditional features.

[0136] In an embodiment of the present application, the reference interface image includes partial image information of the interface image of the second user interface, the conditional feature is the image feature of the reference interface image, and the conditional feature indicates that the target interface image generated by the first image generation model needs to include the above image information. Subsequently, the conditional feature is input into the first image generation model, and the conditional feature can be involved in the calculation process so that the target interface image generated by the first image generation model includes the above image information.

[0137] Optionally, the reference interface image is an image obtained by processing the interface image of the second user interface, for example, the first image information is removed from the interface image of the second user interface, and the second image information is retained to obtain the reference interface image, the first image information is image information that is pre-set not to be retained, and the second image information is image information that is pre-set to be retained.

[0138] For example, the process of obtaining the reference interface image includes at least one of the following:

[0139] 1. Obtain a line image of the interface image of the second user interface, where the line image includes the lines in the interface image of the second user interface. For example, the interface image and line image of the second user interface are shown in FIG12 . By performing line extraction on the interface image of the second user interface, the lines of at least one display element in the interface image of the second user interface can be extracted, so that the lines of at least one display element are retained in the line image, while the color, depth and other information inside the lines of the display element are no longer retained. When the target interface image is subsequently generated based on the line image through the first image generation model, it can be ensured that the lines of the at least one display element in the target interface image are the same as or similar to the lines of the at least one display element in the interface image of the second user interface, and no major difference will be caused.

[0140] 2. Obtain a depth image of the interface image of the second user interface, the depth image including the depth of each position in the interface image of the second user interface. For example, the interface image and depth image of the second user interface are shown in FIG13 . By performing depth extraction on the interface image of the second user interface, the depth of each position in the interface image of the second user interface can be extracted, so that the depth of each position is retained in the depth image, while no longer retaining information such as the color of each position, thereby retaining the perspective relationship of each position in the interface image of the second user interface, and being able to use the front, middle, and back relationship of each position in the interface image of the second user interface as a reference object. Subsequently, when the target interface image is generated based on the depth image through the first image generation model, it can be ensured that the depth of each position in the target interface image is the same or similar to the depth of each position in the interface image of the second user interface, and no major difference is caused. The perspective feeling generated by the user viewing the target interface image is the same or similar to the perspective feeling generated by viewing the interface image of the second user interface.

[0141] 3. Obtain a color image of the interface image of the second user interface, the color image including the main color of each area in the interface image of the second user interface. By performing color recognition on the interface image of the second user interface, the interface image of the second user interface can be divided according to the difference in color, thereby determining the main color of each divided area, so that the main color of each area is retained in the color image, and the depth, lines and other information of each area are no longer retained. Subsequently, when the target interface image is generated based on the color image through the first image generation model, it can be ensured that the main color of each area in the target interface image is the same as or similar to the main color of each area in the interface image of the second user interface, and no major difference will be caused. In the embodiment of the present application, the reference interface image includes a line image, a depth image or a color image as an example, and in other embodiments, the reference interface image can also be other types of images, which is not limited in the embodiment of the present application.

[0142] Optionally, the reference interface image is encoded by a VAE or CLIP encoder to obtain conditional features.

[0143] 809. The computer device encodes the conditional feature and the image feature through the second encoding sub-model to obtain a first conditional feature, and performs a convolution operation on the first conditional feature through the convolution sub-model to obtain a second conditional feature.

[0144] In this possible implementation, step 806 includes:

[0145] 8061. The computer device decodes the encoding feature and the second conditional feature through the decoding sub-model to obtain the target interface image.

[0146] The computer device processes the conditional features and text features through the second encoding sub-model and the convolution sub-model to obtain the second conditional features. In order to make the generated target interface image controllable, the second conditional features are involved in the decoding process of the decoding sub-model. Therefore, the computer device decodes the encoding features and the second conditional features through the decoding sub-model to obtain the target interface image.

[0147] By introducing the reference interface image as a condition into the process of generating an image by the first image generation model, it can be ensured that the generated target interface image contains part of the image information of the interface image of the second user interface and will not differ significantly from the interface image of the second user interface. For example, the application is a game application with a basketball function, and the interface image of the second user interface includes operation controls such as shooting controls, ball control, switch controls, and escape controls. These operation controls are the operation controls that users need to use when participating in a basketball game. In this case, the target interface image generated based on the interface image of the second user interface will also include these operation controls to ensure that the basketball game can proceed normally, except that the style of the operation controls in the target interface image is different from the style of the operation controls in the interface image of the second user interface.

[0148] The solution of the embodiment of the present application provides a user interface generation function within the application. When a computer device is running the application, the image generation model can be used to automatically generate a user interface that matches the keywords based on the input keywords. This achieves user interface personalization, eliminating the need to use designer-created user interfaces. This enhances the user interface's fun and improves user engagement with the application. Furthermore, it eliminates the need for designers to create numerous different user interfaces, saving labor costs and improving the efficiency of user interface generation.

[0149] Based on the first image generation model shown in Figure 10, an embodiment of the present application also provides another first image generation model. The structure of the first image generation model is shown in Figure 14. The first image generation model includes a first encoding sub-model, a second encoding sub-model, a convolution sub-model and a decoding sub-model. In addition, the first encoding sub-model includes n first encoding blocks, the second encoding sub-model includes n second encoding blocks, the convolution sub-model includes n convolution layers, and the decoding sub-model includes n decoding blocks, where n is an integer greater than 1.

[0150] Step 805 includes: encoding the text features and image features through the first first coding block to obtain the first coding feature; encoding the text features and the x-1th coding feature through the xth first coding block to obtain the xth coding feature, until encoding the text features and the n-1th coding feature through the nth first coding block to obtain the nth coding feature, where x is an integer greater than 1 and less than n. The n first coding blocks are connected in sequence. The text features and image features are input into the first first coding block, and the first first coding block outputs the first coding feature. The text features and the first coding feature are input into the second first coding block, and the second first coding block outputs the second coding feature. The text features and the first coding feature are input into the third first coding block, and the third first coding block outputs the third coding feature, and so on, until the last first coding block outputs the last coding feature, that is, until the nth first coding block outputs the nth coding feature.

[0151] Step 809 includes: performing a convolution operation on the conditional features, fusing the conditional features obtained after the convolution operation with the image features to obtain a fused feature; encoding the text features and the fused features through the first second coding block to obtain the first first conditional feature; encoding the text features and the y-1th first conditional feature through the yth second coding block to obtain the yth first conditional feature, until encoding the text features and the n-1th first conditional feature through the nth second coding block to obtain the nth first conditional feature, where y is an integer greater than 1 and less than n; and performing convolution operations on the n first conditional features through n convolutional layers to obtain n second conditional features. The n second coding blocks are connected in sequence. The text features and the fused features are input to the first second coding block, which outputs the first first conditional feature. The text features and the first first conditional feature are input to the second second coding block, which outputs the second first conditional feature. The text feature and the first first conditional feature are input to the third second coding block, and the third second coding block outputs the third first conditional feature, and so on, until the last second coding block outputs the last first conditional feature, that is, until the nth second coding block outputs the nth first conditional feature. Wherein, each second coding block is connected to its own corresponding convolution layer. After each second coding block outputs the first conditional feature, the first conditional feature output by the second coding block is input to the convolution layer corresponding to the second coding block, and the convolution layer outputs a second conditional feature. Optionally, the mth second coding block is connected to the n+1-mth convolution layer, and the n+1-mth convolution layer outputs the n+1-mth second conditional feature. For example, after the first second coding block outputs the first first conditional feature, the first first conditional feature is input to the convolution layer corresponding to the first second coding block, and the convolution layer outputs the nth second conditional feature.

[0152] Step 8061 includes: decoding the text feature, the nth encoding feature and the first second conditional feature through the first decoding block to obtain the first decoding feature; decoding the text feature, the n+1-zth encoding feature and the zth second conditional feature through the zth decoding block to obtain the zth decoding feature, until decoding the text feature, the first decoding feature and the nth second conditional feature through the nth decoding block to obtain the nth decoding feature, where z is an integer greater than 1 and less than n; generating a target interface image based on the nth decoding feature.

[0153] In one possible implementation, as shown in FIG14 , a first intermediate block is further included between the first encoding sub-model and the decoding sub-model. The first intermediate block processes the nth encoding feature and inputs the processed encoding feature into the decoding sub-model for decoding. Furthermore, a second intermediate block is further included between the second encoding sub-model and the convolution sub-model. The second intermediate block processes the nth first conditional feature and inputs the processed first conditional feature into the first intermediate block for processing.

[0154] In another possible implementation, as shown in FIG14 , the computer device further performs iterative processing. Each time the nth decoding feature is obtained through the decoding sub-model, the nth decoding feature is input as an updated image feature into the first image generation model, and the first image generation model processes the feature again until the number of iterations reaches the target number T. The nth decoding feature output by the decoding sub-model and obtained in this iteration is decoded to obtain the target interface image, where T is an integer greater than 1, and the decoder used to decode the nth decoding feature can be a VAE decoder or other decoder. The time shown in FIG14 represents the current number of iterations, that is, the current number of iterations can be encoded at each iteration to obtain a time feature, which is also input into the first image generation model and encoded, decoded, and processed together with the text feature to obtain the nth decoding feature corresponding to the current number of iterations. The nth decoding feature corresponding to the current number of iterations is decoded to obtain the target interface image.

[0155] In one possible implementation, the training process of a first image generation model includes: training a second image generation model, the second image generation model including a first encoding sub-model and a decoding sub-model; when the second image generation model meets a training end condition, copying the first encoding sub-model to obtain a second encoding sub-model; adding the second encoding sub-model and a convolution sub-model to the second image generation model to obtain the first image generation model; and training the first image generation model while maintaining the model parameters of the first encoding sub-model unchanged until the first image generation model meets the training end condition. The first encoding sub-model may be referred to as a locked copy, and the second encoding sub-model may be referred to as a trainable copy. After preliminary training of the first encoding sub-model and the decoding sub-model is completed, the model parameters of the first encoding sub-model remain unchanged. The second encoding sub-model and the convolution sub-model are then added, and the second encoding sub-model reuses the model parameters of the first encoding sub-model. The first image generation model is trained while maintaining the model parameters of the first encoding sub-model unchanged. During the training process, the model parameters of the second encoding sub-model are fine-tuned until the training of the first image generation model is completed.

[0156] Therefore, the first image generation model is divided into two modules. The first module is the first encoding sub-model and the decoding sub-model. The first module retains the original process of generating images by the first image generation model. Its generation process is related to the model structure and model parameters of the first module itself. The second module is the second encoding sub-model, the convolution sub-model and the decoding sub-model. The second encoding sub-model is a copy model of the first encoding sub-model. After encoding by the first encoding sub-model, the second module will use the influence of the reference interface image (line image, depth image, color image, etc.) to act on the decoding process of the decoding sub-model, so that the generated target interface image is affected by the lines, depth and color of the interface image of the second user interface, that is, a target interface image that is different from the interface image of the second user interface but is controlled is generated to avoid the generated user interface affecting the normal operation of the application.

[0157] In one possible implementation, training the second image generation model includes training the second image generation model based on the first model. Training the first image generation model includes training the first image generation model based on at least one of the second, third, or fourth models. The first model includes at least one image, text keywords, and diffusion information for the at least one image; the second model includes at least one line image, text keywords, and diffusion information for the at least one line image; the third model includes at least one depth image, text keywords, and diffusion information for the at least one depth image; and the fourth model includes at least one color image, text keywords, and diffusion information for the at least one color image. The diffusion information for any image includes the noise image and noise obtained by adding noise to the image.

[0158] First, as shown in Figure 15, a first model is set up in the second image generation model. The first model includes multiple images. For each image, text keywords are set for that image, and the DDPM algorithm is used to process the image to obtain diffusion information. Specifically, a first noise is generated, and the first noise is superimposed on the image to obtain a first noise image. The first noise image and the first noise are recorded. Then, a second noise is generated, and the second noise is superimposed on the first noise image to obtain a second noise image. The second noise image and the second noise are recorded. This process is repeated, resulting in multiple noise images of the image and their corresponding noise. The number of noise additions can be any set value.

[0159] In addition, the first image generation model includes a second model, a third model, and a fourth model. The second model includes multiple line images, each of which is assigned a text keyword. Diffusion information for the line images is obtained by processing them using the DDPM algorithm, with reference to the lines within the line images. As shown in Figure 16, when line constraints are applied, the lines in different images generated from the same line image are identical or similar. The third model includes multiple depth images, each of which is assigned a text keyword. Diffusion information for the depth images is obtained by processing them using the DDPM algorithm, with reference to the depth within the depth images. As shown in Figure 17, when depth constraints are applied, the depths at the same location in different images generated from the same depth image are identical or similar. The fourth model includes multiple color images, each of which is assigned a text keyword. Diffusion information for the color images is obtained by processing them using the DDPM algorithm, with reference to the colors within the color images. As shown in Figure 18, when color constraints are applied, the dominant colors of the same region in different images generated from the same color image are identical or similar. The process of creating diffusion information for the image in the second model, the third model, and the fourth model is similar to that of the first model and will not be repeated here.

[0160] Optionally, the second image generation model is a diffusion model, such as a Stable Diffusion model or other diffusion models, and the second encoding sub-model and the convolution sub-model can constitute a ControlNet.

[0161] During the training process of the diffusion model, any one or more images from the first model can be used as input samples, and the noise added to these samples can be used as output samples to train the diffusion model. This allows the diffusion model to predict noise. In other words, the diffusion model can act as a noise predictor, simulating the process of generating new images using the diffusion model as the process of predicting and removing noise from the original image. After the introduction of ControlNet, during the training process, any one or more images from the second, third, and fourth models can be used as references to train the diffusion model and ControlNet. This allows the diffusion model to predict noise under the constraints of line images, depth images, or color images, improving the performance of the diffusion model and, in turn, the display quality of user interfaces generated by the diffusion model.

[0162] The operational flow for a computer device to generate a first user interface is shown in Figure 19. The diffusion model and ControlNet are trained based on the first, second, third, and fourth models. Once training is complete, the first image generation model can be deployed. The computer device then obtains the interface image of the second user interface, positive keywords, negative keywords, and relevance through an application. The positive and negative keywords are encoded to obtain text features, and the interface image of the second user interface is encoded to obtain image features. The text features, image features, and relevance are input into the diffusion model. Furthermore, the line image, depth image, and color image of the interface image of the second user interface are encoded to obtain conditional features, which are then input into the ControlNet. The diffusion model's processing can be viewed as a process of predicting and removing noise from the interface image of the second user interface. Conditional features are also involved in this noise removal process. The diffusion model then outputs image features, which are converted into a target interface image. The target interface image is the image obtained after removing noise from the interface image of the second user interface. The computer device uploads the target interface image to the file generation server, and the file generation server generates a user interface file based on the target interface image and returns it to the local computer device. The computer device displays a new first user interface based on the user interface file, or previews the first user interface.

[0163] Figure 20 is a schematic diagram of the structure of a user interface generation device provided in an embodiment of the present application. Referring to Figure 20 , the device includes: a display module 2001, configured to display a generation interface within the application in response to a generation instruction in the application, wherein the generation instruction instructs the application to generate a user interface; an acquisition module 2002, configured to acquire keywords entered within the generation interface, wherein the keywords describe the conditions that the user interface to be generated must meet; a generation module 2003, configured to generate a first user interface based on the keywords, wherein the first user interface matches the keywords; and display module 2001, further configured to display the first user interface within the application.

[0164] The solution of the embodiment of the present application provides a user interface generation function within the application. When a computer device is running the application, a user interface matching the keyword can be generated based on the input keyword. This achieves user interface personalization, eliminating the need to use a designer-created user interface. This enhances the user interface's fun and improves user engagement with the application. This eliminates the need for designers to create numerous different user interfaces, saving labor costs and improving the efficiency of user interface generation.

[0165] Optionally, referring to Figure 21, the acquisition module 2002 includes: a first acquisition unit 2012, used to display a first input bar in the generation interface; the first acquisition unit 2012 is also used to obtain positive keywords based on the input operation in the first input bar, and the positive keywords are keywords that need to be associated with the user interface to be generated.

[0166] Optionally, referring to Figure 21, the acquisition module 2002 includes: a second acquisition unit 2022, used to display a sliding bar on the generation interface, the sliding bar including a sliding block, the position of the sliding block represents the correlation, and the correlation is the correlation between the user interface to be generated and the positive keyword; the second acquisition unit 2022 is also used to obtain the correlation represented by the position of the sliding block based on the sliding operation of the sliding block; the generation module 2003 is used to generate a first user interface based on the correlation represented by the positive keyword and the position of the sliding block.

[0167] Optionally, referring to Figure 21, the acquisition module 2002 includes: a third acquisition unit 2032, used to display the second input field in the generation interface; the third acquisition unit 2032 is also used to obtain negative keywords based on the input operation in the second input field, and the negative keywords are keywords that do not need to be associated with the user interface to be generated.

[0168] Optionally, referring to Figure 21, the display module 2001 includes: a display unit 2011, used to display the interface image of the second user interface in the generated interface, the second user interface is the user interface currently used by the application; the display unit 2011 is also used to replace the interface image of the second user interface with the interface image of the first user interface after generating the first user interface.

[0169] Optionally, a user interface generation control is displayed in the application, and the display module 2001 is used to display a generation interface in response to a triggering operation on the user interface generation control.

[0170] Optionally, the display module 2001 is further configured to display a user interface ranking list, where the user interface ranking list includes multiple user interfaces, and each user interface in the user interface ranking list is generated by a computer device running an application.

[0171] Optionally, referring to Figure 21, the generation module 2003 includes: an image generation unit 2013, used to generate a target interface image based on the interface image and keywords of the second user interface through a first image generation model; an interface generation unit 2023, used to generate a first user interface based on the target interface image; wherein the second user interface is the user interface currently used by the application.

[0172] Optionally, referring to FIG. 21 , the interface generation unit 2023 is configured to: identify the target interface image to obtain multiple interface elements and their respective positions in the target interface image; and generate a first user interface based on the multiple interface elements and their respective positions.

[0173] Optionally, referring to FIG. 21 , the interface generating unit 2023 is configured to: obtain an interface template; and fill the interface template with multiple interface elements according to their positions to obtain a first user interface.

[0174] Optionally, the first image generation model includes a first encoding sub-model and a decoding sub-model, and the image generation unit 2013 is used to: encode the interface image of the second user interface to obtain image features; encode keywords to obtain text features; encode text features and image features through the first encoding sub-model to obtain encoding features; and decode the encoding features through the decoding sub-model to obtain the target interface image.

[0175] Optionally, the first coding sub-model includes n first coding blocks, where n is an integer greater than 1; the image generation unit 2013 is used to: encode the text features and the image features through the first first coding block to obtain the first coding feature; encode the text features and the x-1th coding features through the xth first coding block to obtain the xth coding feature, until the text features and the n-1th coding features are encoded through the nth first coding block to obtain the nth coding feature, where x is an integer greater than 1 and less than n.

[0176] Optionally, the first image generation model also includes a second encoding sub-model and a convolution sub-model, and the image generation unit 2013 is further used to: encode the reference interface image to obtain conditional features, the reference interface image contains partial image information of the interface image of the second user interface, and the conditional features indicate that the target interface image generated by the first image generation model needs to contain image information; encode the conditional features and image features through the second encoding sub-model to obtain the first conditional features, and perform a convolution operation on the first conditional features through the convolution sub-model to obtain the second conditional features; the image generation unit 2013 is used to: decode the encoding features and the second conditional features through the decoding sub-model to obtain the target interface image.

[0177] Optionally, the second encoding sub-model includes n second encoding blocks, the convolution sub-model includes n convolution layers, and the image generation unit 2013 is used to: perform a convolution operation on the conditional features, fuse the conditional features obtained after the convolution operation with the image features to obtain a fused feature; encode the text features and the fused features through the first second encoding block to obtain the first first conditional feature; encode the text features and the y-1th first conditional feature through the yth second encoding block to obtain the yth first conditional feature, until the text features and the n-1th first conditional feature are encoded through the nth second encoding block to obtain the nth first conditional feature, where y is an integer greater than 1 and less than n; and perform convolution operations on the n first conditional features through n convolution layers to obtain n second conditional features.

[0178] Optionally, the decoding sub-model includes n decoding blocks, and the image generation unit 2013 is used to: decode the text feature, the nth encoding feature and the first second conditional feature through the first decoding block to obtain the first decoding feature; decode the text feature, the n+1-zth encoding feature and the zth second conditional feature through the zth decoding block to obtain the zth decoding feature, until the text feature, the first decoding feature and the nth second conditional feature are decoded through the nth decoding block to obtain the nth decoding feature, where z is an integer greater than 1 and less than n; and generate the target interface image based on the nth decoding feature.

[0179] Optionally, the process of obtaining a reference interface image includes at least one of the following: obtaining a line image of the interface image of the second user interface, the line image including the lines in the interface image of the second user interface; obtaining a depth image of the interface image of the second user interface, the depth image including the depth of each position in the interface image of the second user interface; obtaining a color image of the interface image of the second user interface, the color image including the main color of each area in the interface image of the second user interface.

[0180] Optionally, referring to Figure 21, the device also includes a training module 2004, which is used to: train the second image generation model, the second image generation model includes a first encoding sub-model and a decoding sub-model; when the second image generation model meets the training end condition, copy the first encoding sub-model to obtain the second encoding sub-model; add the second encoding sub-model and the convolution sub-model to the second image generation model to obtain the first image generation model; while keeping the model parameters of the first encoding sub-model unchanged, train the first image generation model until the first image generation model meets the training end condition.

[0181] Optionally, the training module 2004 is used to: train the second image generation model based on the first model; train the first image generation model based on at least one of the second model, the third model or the fourth model; the first model includes at least one image, text keywords and diffusion information of at least one image, the second model includes at least one line image, text keywords and diffusion information of at least one line image, the third model includes at least one depth image, text keywords and diffusion information of at least one depth image, and the fourth model includes at least one color image, text keywords and diffusion information of at least one color image; the diffusion information of any image includes the noise image and noise obtained by adding noise to the image.

[0182] It should be noted that the user interface generation device provided in the above embodiment is merely exemplified by the division of the aforementioned functional modules. In actual applications, the aforementioned functions can be assigned to different functional modules as needed, i.e., the internal structure of a computer device can be divided into different functional modules to perform all or part of the functions described above. Furthermore, the user interface generation device provided in the above embodiment and the user interface generation method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be further elaborated here.

[0183] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed in the user interface generation method of the above embodiment.

[0184] Optionally, the computer device is provided as a terminal. FIG22 shows a schematic diagram of the structure of a terminal 2200 provided in an exemplary embodiment of the present application. The terminal 2200 includes: a processor 2201 and a memory 2202.

[0185] The processor 2201 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 2201 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), or PLA (Programmable Logic Array). The processor 2201 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 2201 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 2201 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0186] The memory 2202 may include one or more computer-readable storage media, which may be non-transitory. The memory 2202 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 2202 is used to store at least one computer program, which is used to be used by the processor 2201 to implement the user interface generation method provided in the method embodiment of the present application.

[0187] In some embodiments, terminal 2200 may also optionally include a peripheral device interface 2203 and at least one peripheral device. Processor 2201, memory 2202, and peripheral device interface 2203 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 2203 via a bus, signal lines, or circuit boards. Optionally, the peripheral device includes at least one of a radio frequency circuit 2204, a display screen 2205, a camera assembly 2206, and a power supply 2207.

[0188] Those skilled in the art will understand that the structure shown in FIG22 does not constitute a limitation on the terminal 2200 , and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0189] Optionally, the computer device is provided as a server. Figure 23 is a structural diagram of a server provided in an embodiment of the present application. The server 2300 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 2301 and one or more memories 2302, wherein at least one computer program is stored in the memory 2302, and at least one computer program is loaded and executed by the processor 2301 to implement the methods provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server may also include other components for implementing device functions, which will not be described in detail here.

[0190] An embodiment of the present application further provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the operations performed by the user interface generation method of the above embodiment.

[0191] An embodiment of the present application further provides a computer program product, including a computer program, which is loaded and executed by a processor to implement the operations performed by the user interface generation method of the above embodiment.

[0192] Those skilled in the art will understand that all or part of the steps of implementing the above embodiments may be accomplished by hardware, or by programs instructing related hardware to accomplish the steps. The programs may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk, or an optical disk, etc.

[0193] The above are only optional embodiments of the embodiments of the present application and are not intended to limit the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. A method for generating a user interface, executed by a computer device, the method comprising: In response to a generation instruction in the application, displaying a generation interface in the application, the generation instruction instructing to generate a user interface for the application; Acquire keywords input in the generation interface, where the keywords are used to describe conditions that need to be met by the user interface to be generated; generating a first user interface based on the keyword, wherein the first user interface matches the keyword; The first user interface is displayed in the application.

2. The method according to claim 1, wherein: The obtaining of the keywords input in the generation interface includes: Displaying a first input field on the generation interface; Based on the input operation in the first input field, a positive keyword is obtained, where the positive keyword is a keyword that needs to be associated with the user interface to be generated.

3. The method according to claim 1 or 2, wherein: The method further comprises: Displaying a sliding bar on the generation interface, the sliding bar including a sliding block, the position of the sliding block represents the degree of association, and the degree of association is the degree of association between the user interface to be generated and the forward keyword; Based on the sliding operation on the sliding block, obtaining the correlation degree represented by the position of the sliding block; The generating a first user interface based on the keyword includes: The first user interface is generated based on the association between the positive keyword and the position representation of the sliding block.

4. The method according to any one of claims 1 to 3, wherein: The obtaining of the keywords input in the generation interface includes: Displaying a second input field on the generation interface; Based on the input operation in the second input field, a negative keyword is acquired, where the negative keyword is a keyword that does not need to be associated with the user interface to be generated.

5. The method according to any one of claims 1 to 4, wherein: The method further comprises: Displaying an interface image of a second user interface in the generated interface, where the second user interface is the user interface currently used by the application; After the first user interface is generated, the interface image of the second user interface is replaced with the interface image of the first user interface.

6. The method according to any one of claims 1 to 5, wherein: The application displays a user interface generation control, and the step of displaying a generation interface in the application in response to a generation instruction in the application includes: In response to a triggering operation on the user interface generating control, the generating interface is displayed.

7. The method according to any one of claims 1 to 6, wherein: The method further comprises: A user interface ranking list is displayed, wherein the user interface ranking list includes a plurality of user interfaces, and each user interface in the user interface ranking list is generated by a computer device running the application.

8. The method according to any one of claims 1 to 7, wherein: The generating a first user interface based on the keyword includes: Generate a target interface image based on the interface image of the second user interface and the keyword through the first image generation model; Based on the target interface image, generating the first user interface; The second user interface is the user interface currently used by the application.

9. The method according to any one of claims 1 to 8, wherein: The generating the first user interface based on the target interface image includes: Identify the target interface image to obtain multiple interface elements and their respective positions in the target interface image; Based on the multiple interface elements and their respective positions, the first user interface is generated.

10. The method according to any one of claims 1 to 9, wherein: The generating the first user interface based on the plurality of interface elements and respective attribute information includes: Get the interface template; According to the positions of the multiple interface elements, the multiple interface elements are filled into the interface template to obtain the first user interface.

11. The method according to any one of claims 1 to 10, wherein: The first image generation model includes a first encoding sub-model and a decoding sub-model, and the generating of the target interface image based on the interface image of the second user interface and the keyword by the first image generation model includes: Encoding the interface image of the second user interface to obtain image features; Encoding the keywords to obtain text features; Encoding the text features and the image features through the first encoding sub-model to obtain encoding features; The encoding feature is decoded through the decoding sub-model to obtain the target interface image.

12. The method according to any one of claims 1 to 11, wherein: The first coding sub-model includes n first coding blocks, where n is an integer greater than 1; The encoding of the text features and the image features by the first encoding sub-model to obtain encoding features includes: Encoding the text feature and the image feature through the first of the first encoding blocks to obtain the first of the encoding features; The text feature and the x-1th encoding feature are encoded through the xth first encoding block to obtain the xth encoding feature, until the text feature and the n-1th encoding feature are encoded through the nth first encoding block to obtain the nth encoding feature, where x is an integer greater than 1 and less than n.

13. The method according to any one of claims 1 to 12, wherein: The first image generation model further includes a second encoding sub-model and a convolution sub-model, and the generating of the target interface image based on the interface image of the second user interface and the keyword by the first image generation model also includes: Encoding a reference interface image to obtain a conditional feature, wherein the reference interface image includes part of the image information of the interface image of the second user interface, and the conditional feature indicates that the target interface image generated by the first image generation model needs to include the image information; The conditional feature and the image feature are encoded by the second encoding sub-model to obtain a first conditional feature, and the first conditional feature is convolved by the convolution sub-model to obtain a second conditional feature; The decoding sub-model is used to decode the coding feature to obtain the target interface image, including: The encoding feature and the second conditional feature are decoded through the decoding sub-model to obtain the target interface image.

14. The method according to any one of claims 1 to 13, wherein: The second encoding sub-model includes n second encoding blocks, the convolution sub-model includes n convolution layers, the conditional feature and the image feature are encoded by the second encoding sub-model to obtain a first conditional feature, and the first conditional feature is convolved by the convolution sub-model to obtain a second conditional feature, including: Performing a convolution operation on the conditional feature, and fusing the conditional feature obtained after the convolution operation with the image feature to obtain a fused feature; Encoding the text feature and the fusion feature through the first second encoding block to obtain the first first conditional feature; By using the yth second coding block, the text feature and the y-1th first conditional feature are encoded to obtain the yth first conditional feature, until by using the nth second coding block, the text feature and the n-1th first conditional feature are encoded to obtain the nth first conditional feature, where y is an integer greater than 1 and less than n; Through the n convolutional layers, convolution operations are performed on the n first conditional features respectively to obtain n second conditional features.

15. The method according to any one of claims 1 to 14, wherein: The decoding sub-model includes n decoding blocks, and the decoding sub-model is used to decode the encoding feature and the second conditional feature to obtain the target interface image, including: Decoding the text feature, the nth encoding feature and the first second conditional feature through the first decoding block to obtain a first decoding feature; The text feature, the n+1-zth encoding feature and the zth second conditional feature are decoded through the zth decoding block to obtain the zth decoding feature, until the text feature, the first decoding feature and the nth second conditional feature are decoded through the nth decoding block to obtain the nth decoding feature, where z is an integer greater than 1 and less than n; Based on the nth decoding feature, the target interface image is generated.

16. The method according to any one of claims 1 to 15, wherein: The process of acquiring the reference interface image includes at least one of the following: Acquire a line image of the interface image of the second user interface, where the line image includes lines in the interface image of the second user interface; Acquire a depth image of the interface image of the second user interface, where the depth image includes the depth of each position in the interface image of the second user interface; A color image of the interface image of the second user interface is obtained, where the color image includes a main color of each area in the interface image of the second user interface.

17. The method according to any one of claims 1 to 16, wherein: The training process of the first image generation model includes: Training a second image generation model, where the second image generation model includes the first encoding sub-model and the decoding sub-model; When the second image generation model meets the training end condition, copying the first encoding sub-model to obtain the second encoding sub-model; Adding the second encoding sub-model and the convolution sub-model to the second image generation model to obtain the first image generation model; While keeping the model parameters of the first encoding sub-model unchanged, the first image generation model is trained until the first image generation model meets the training end condition.

18. The method according to any one of claims 1 to 17, wherein: The training of the second image generation model comprises: training the second image generation model based on the first model; The training of the first image generation model includes: based on the second model, the third model or the fourth model at least one model of the image generation model, training the first image generation model; The first model includes at least one image, text keywords and diffusion information of the at least one image, the second model includes at least one line image, text keywords and diffusion information of the at least one line image, the third model includes at least one depth image, text keywords and diffusion information of the at least one depth image, and the fourth model includes at least one color image, text keywords and diffusion information of the at least one color image; The diffusion information of any image includes a noise image obtained by adding noise to the image and the noise.

19. A user interface generating device, the device comprising: A display module, configured to display a generation interface in the application in response to a generation instruction in the application, wherein the generation instruction indicates generation of a user interface for the application; An acquisition module, used to acquire keywords input in the generation interface, wherein the keywords are used to describe conditions that need to be met by the user interface to be generated; A generating module, configured to generate a first user interface based on the keyword, wherein the first user interface matches the keyword; The display module is used to display the first user interface in the application.

20. A computer device, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the user interface generation method according to any one of claims 1 to 18.

21. A computer-readable storage medium, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor to implement the operations performed by the user interface generation method according to any one of claims 1 to 18.

22. A computer program product, comprising a computer program, wherein the computer program is loaded and executed by a processor to implement the operations performed by the user interface generating method according to any one of claims 1 to 18.

Images