Image processing method and device, electronic equipment and storage medium

Abstract

The invention discloses an image processing method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-processed first image, the first image being a two-dimensional image of a target virtual three-dimensional model, and the two-dimensional image carrying at least one model parameter of the target virtual three-dimensional model; inputting the first image into a target image generation tool, and processing the first image through a target image generation model in the target image generation tool to obtain a second image with a target style; and processing the second image based on the model parameters through an image processing module in the target image generation tool, and generating a target image which has the target style and is matched with the target virtual three-dimensional model. According to the method, the two-dimensional image is generated through automatic optimization, the making time of game designers in optimizing the two-dimensional image is saved, the image making steps are simplified, programming of image detail making is achieved, and the image making efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image processing, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] In order to meet people's pursuit of spiritual life, entertainment games that can be operated on terminals have emerged. For example, multiplayer online tactical competitive games and massively multiplayer online games developed based on client or server architecture are deeply loved by users due to their high fluency, good operating feel, real-time combat, etc. In the game, players can manipulate virtual objects on the screen to play in a virtual game scene, allowing players to experience the visual impact of the game in person, greatly enhancing the initiative and realism of the game.

[0003] At present, game scenes for virtual objects to interact in games can be generated using the "3D rendering 2D" technology. "3D rendering 2D" is a non-realistic rendering art style that can be displayed by converting the output of a 3D rendered virtual 3D model into a visual form of a 2D image, so that the game scene looks like it has a 3D perspective while maintaining a 2D effect. However, in the prior art, in order to obtain exquisite and high-quality 2D images, the existing production process of converting virtual 3D models to 2D images requires game designers to spend a lot of time manually adjusting the 2D images according to their own design requirements, resulting in cumbersome steps for the production of 2D images, complicated and troublesome processes, and low efficiency in the production of 2D images. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, device, electronic device and storage medium. By inputting an initial two-dimensional image of a virtual three-dimensional model into a target image generation tool, the two-dimensional image can be automatically optimized through a target image generation model configured by the target image generation tool, and the two-dimensional image can be automatically adjusted to a two-dimensional image of a target style. The two-dimensional image is generated through automated optimization, thereby saving production time for game designers when optimizing two-dimensional images, simplifying the steps of two-dimensional image production, realizing the procedural production of two-dimensional image details, and improving the efficiency of two-dimensional image production.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0006] Acquire a first image to be processed, where the first image is a two-dimensional image of a target virtual three-dimensional model, and the two-dimensional image carries at least one model parameter of the target virtual three-dimensional model;

[0007] Inputting the first image into a target image generation tool, and processing the first image by a target image generation model in the target image generation tool to obtain a second image with a target style, wherein the target image generation model is used to generate an image with a target style;

[0008] The second image is processed based on the model parameters by an image processing module in the target image generation tool to generate a target image having the target style and matching the target virtual three-dimensional model.

[0009] In a second aspect, an embodiment of the present application provides an image processing device, including:

[0010] an acquisition unit, configured to acquire a first image to be processed, wherein the first image is a two-dimensional image of a target virtual three-dimensional model, and the two-dimensional image carries at least one model parameter of the target virtual three-dimensional model;

[0011] a first processing unit, configured to input the first image into a target image generation tool, and process the first image through a target image generation model in the target image generation tool to obtain a second image with a target style, wherein the target image generation model is used to generate an image with a target style;

[0012] The second processing unit is used to process the second image based on the model parameters through the image processing module in the target image generation tool to generate a target image with the target style and matching the target virtual three-dimensional model.

[0013] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory storing a plurality of instructions; a processor loads instructions from the memory to execute the steps of any image processing method provided in the embodiment of the present application.

[0014] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a plurality of instructions suitable for loading by a processor to execute the steps of any image processing method provided in the embodiment of the present application.

[0015] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps of any image processing method provided in the embodiment of the present application.

[0016] By adopting the solution of the embodiment of the present application, the initial two-dimensional image of the virtual three-dimensional model is input into the target image generation tool. The target image generation model configured by the target image generation tool can automatically optimize the two-dimensional image to automatically adjust the two-dimensional image to a two-dimensional image of a target style. The two-dimensional image is generated through automated optimization, thereby saving the production time of game designers when optimizing the two-dimensional image, simplifying the steps of two-dimensional image production, realizing the procedural production of two-dimensional image details, and improving the efficiency of two-dimensional image production. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 is a schematic diagram of a scene of an image processing system provided in an embodiment of the present application;

[0019] Figure 2 It is a schematic diagram of an embodiment of the image processing method provided in the embodiment of the present application;

[0020] Figure 3 This is a schematic diagram of a scenario of the image processing method provided in an embodiment of the present application;

[0021] Figure 4a is another scenario schematic diagram of the image processing method provided in an embodiment of the present application;

[0022] Figure 4b is another scenario schematic diagram of the image processing method provided in an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0024] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application. At the same time, in the description of the embodiments of the present application, the terms "first", "second", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0026] In the prior art, a layer of semi-transparent texture material is usually added on top of an opaque virtual clothing material to create a wave effect on the virtual clothing. However, since the semi-transparent material is the material with the highest resource consumption during the rendering process, the resource consumption of the semi-transparent texture material is high, resulting in a large amount of resources required to realize the wave effect on the virtual clothing, thereby resulting in low efficiency in the production of virtual resources.

[0027] The present application embodiment provides an image processing method, device, electronic device and computer-readable storage medium, which can obtain a first image to be processed, wherein the first image is a two-dimensional image of a target virtual three-dimensional model, and the two-dimensional image carries at least one model parameter of the target virtual three-dimensional model; then, the first image is input into a target image generation tool, and the first image is processed by a target image generation model in the target image generation tool to obtain a second image with a target style, wherein the target image generation model is used to generate an image with a target style; finally, the second image is processed based on the model parameters by an image processing module in the target image generation tool to generate a target image with the target style and matching the target virtual three-dimensional model. In the present application embodiment, by inputting the initial two-dimensional image of the virtual three-dimensional model into the target image generation tool, the target image generation model configured by the target image generation tool can automatically optimize the two-dimensional image, realize automatic adjustment of the two-dimensional image to a two-dimensional image of the target style, and generate the two-dimensional image by automatic optimization, thereby saving the production time of the game designer when optimizing the two-dimensional image, simplifying the steps of two-dimensional image production, realizing the programming of two-dimensional image detail production, and improving the efficiency of two-dimensional image production.

[0028] The embodiments of the present application provide an image processing method, device, electronic device, and computer-readable storage medium. Specifically, the present embodiment will be described from the perspective of an image processing device, which can be integrated in an electronic device, that is, the image processing method of the embodiment of the present application can be executed by an electronic device, and optionally, the electronic device can include: a terminal device. The terminal device can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop computer, a game console, or a personal computer (PC) and other devices.

[0029] The image processing method provided in the embodiment of the present application can be applied to an image processing system. The image processing system may include a player terminal device and a server, and the terminal may be a device including both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing two-way communication on a two-way communication link. The player terminal device and the server may perform two-way communication via a network.

[0030] Optionally, the server may be an independent server, or a server network or server cluster composed of servers, including but not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0031] The image processing method in one embodiment of the present disclosure can be run on a local terminal device or a server. When the image processing method is run on a server, the method can be implemented and executed based on a cloud interaction system, wherein the cloud interaction system includes a server and a client device.

[0032] For example, when the image processing method is run on a terminal, the terminal device stores a game application and is used to present a virtual scene in a game screen. The terminal device is used to interact with a user through a graphical user interface, for example, by downloading and installing a game application through the terminal device and running it. The terminal device may provide a graphical user interface to a user in a variety of ways, for example, it may be rendered and displayed on a display screen of the terminal device, or the graphical user interface may be presented through holographic projection. For example, the terminal device may include a touch display screen and a processor, the touch display screen is used to present a graphical user interface and receive operation instructions generated by a user acting on the graphical user interface, the graphical user interface includes a game screen, and the processor is used to run the game, generate a graphical user interface, respond to operation instructions, and control the display of the graphical user interface on the touch display screen.

[0033] For example, when the image processing method is run on a server, it can be a cloud game. Cloud gaming refers to a gaming method based on cloud computing. In the operation mode of cloud gaming, the operating body of the game application and the main body of the game screen presentation are separated, and the storage and operation of the image processing method are completed on the cloud gaming server. The game screen presentation is completed on the client of the cloud gaming. The cloud gaming client is mainly used for receiving and sending game data and presenting the game screen. For example, the cloud gaming client can be a display device with data transmission function close to the user side, such as a mobile terminal, a TV, a computer, a handheld computer, a personal digital assistant, etc., but the terminal device for processing game data is a cloud gaming server in the cloud. When playing the game, the user operates the cloud gaming client to send an operation instruction to the cloud gaming server. The cloud gaming server runs the game according to the operation instruction, encodes and compresses the game screen and other data, and returns it to the cloud gaming client through the network. Finally, the cloud gaming client decodes and outputs the game screen.

[0034] See also Figure 1 , Figure 1 A schematic diagram of a scene of an image processing system provided by an embodiment of the present application. The system may include at least one terminal, at least one server, at least one database, and a network. The terminal held by the user can be connected to the server of different games through the network. The terminal is any device with computing hardware that can support and execute software products corresponding to the game. In addition, when the system includes multiple terminals, multiple servers, and multiple networks, different terminals can be connected to each other through different networks and different servers. The network can be a wireless network or a wired network, such as a wireless network such as a wireless local area network (WLAN), a local area network (LAN), a cellular network, a 2G network, a 3G network, a 4G network, a 5G network, etc. In addition, different terminals can also use their own Bluetooth network or hotspot network to connect to other terminals or to servers, etc. For example, multiple users can be online through different terminals and connected and synchronized with each other through appropriate networks to support multi-player games. In addition, the system may include multiple databases, multiple databases are coupled to different servers, and information related to the game environment can be continuously stored in the database when different users are online for multi-player games.

[0035] The embodiment of the present application provides an image processing method, which can be executed by a terminal or a server. The embodiment of the present application takes the execution of the image processing method by the terminal as an example for explanation. Among them, the terminal may include a touch display screen and a processor (of course, the terminal may also use peripherals such as a mouse and a keyboard as input devices, and only the touch display screen is used as an example for explanation here), and the touch display screen is used to present a graphical user interface and receive operation instructions generated by the user acting on the graphical user interface. When the user operates the graphical user interface through the touch display screen, the graphical user interface can control the local content of the terminal by responding to the received operation instructions, and can also control the content of the opposite server by responding to the received operation instructions. For example, the operation instructions generated by the user acting on the graphical user interface include instructions for starting a game application, and the processor is configured to start the game application after receiving the instructions for starting the game application provided by the user. In addition, the processor is configured to render and draw a graphical user interface associated with the game on the touch display screen. The touch display screen is a multi-touch sensitive screen that can sense touch or sliding operations performed simultaneously at multiple points on the screen. The user uses a finger to perform a touch operation on the graphical user interface, and when the graphical user interface detects the touch operation, the different virtual objects in the graphical user interface of the game are controlled to perform actions corresponding to the touch operation.

[0036] It should be noted that Figure 1 The scene diagram of the image processing system shown is merely an example. The image processing system and scene described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can appreciate that with the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0037] The following is a detailed description in conjunction with the accompanying drawings. In this embodiment, the execution subject is a terminal device as an example. It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments. Although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that shown in the accompanying drawings.

[0038] See also Figure 2 , Figure 2 The following is a flow chart of an image processing method provided in an embodiment of the present application. The specific flow of the image processing method may be as follows: Steps 101 to 103:

[0039] Step 101: Acquire a first image to be processed, where the first image is a two-dimensional image of a target virtual three-dimensional model, and the two-dimensional image carries at least one model parameter of the target virtual three-dimensional model.

[0040] In an embodiment of the present application, the first image may be a two-dimensional image obtained by image conversion of a virtual three-dimensional model, and the two-dimensional image may be stylized or adjusted in detail by inputting the two-dimensional image into a target image generation tool as a basis.

[0041] Optionally, the target virtual three-dimensional model to be processed may also be input into a target image generation tool to generate a target image having the target style and matching the target virtual three-dimensional model.

[0042] Among them, a virtual 3D model refers to a virtual object created in a computer with 3D spatial information, specifically a digital representation of an object or scene in the real world. A virtual 3D model usually consists of a series of geometric shapes, textures, materials, and other properties to accurately describe the appearance and behavior of an object in a computer. These virtual 3D models can be simple geometric bodies, such as a cube or a sphere, or complex objects, such as a virtual person or a virtual building.

[0043] In an embodiment of the present application, the model parameters of the virtual three-dimensional model include the depth information and contour edge information of the virtual three-dimensional model. Among them, the depth information refers to the distance of the virtual three-dimensional model relative to the observer, which represents the positional relationship of the virtual three-dimensional model in the virtual three-dimensional space. From the perspective of the observer, the virtual three-dimensional model closer to the observer will be considered to be more foreground, and the virtual three-dimensional model farther away from the observer will be considered to be more background. The occlusion relationship, projection effect and perspective transformation of the object can be determined through the depth information, so as to make the virtual scene more realistic and three-dimensional. In computer graphics, depth information is usually stored in the form of Z coordinates or depth buffers. The depth buffer is a buffer corresponding to the screen pixels one by one, which records the depth value of each pixel. By comparing the depth values ​​of the pixels, the front and back relationships can be determined, thereby achieving the correct rendering order and occlusion effect. Specifically, the embodiment of the present application can obtain the edge contour information of the virtual three-dimensional model through the SoftEdge model in ControlNet.

[0044] Step 102: input the first image into a target image generation tool, and process the first image through a target image generation model in the target image generation tool to obtain a second image with a target style, wherein the target image generation model is used to generate an image with a target style.

[0045] In an embodiment of the present application, the target image generation tool may be an AI painting generation tool, such as a Stable Diffusion tool. The Stable Diffusion tool may be configured with a target image generation model (such as a Low-Rank Adaptation model, also known as a Lora model). The Lora model is a lightweight model fine-tuning technology, which is particularly suitable for rapid adaptation and customization based on large pre-trained models. For example, the Lora model may be trained based on a Stable Diffusion framework to obtain a target image generation tool that can generate images in a target style.

[0046] The style may be a requirement for a certain game element of a two-dimensional image, for example, a first style requiring a virtual ground to be made of an old material, or a second style requiring a virtual ground to be made of a brick material.

[0047] Step 103: Process the second image based on the model parameters through the image processing module in the target image generation tool to generate a target image having the target style and matching the target virtual three-dimensional model.

[0048] In an embodiment of the present application, the target image generation tool is configured with a plurality of target parameter information, and the method further includes:

[0049] The second image is processed based on the model parameters and the target parameter information by the image processing module in the target image generation tool to generate a target image having the target style and matching the target virtual three-dimensional model and the target parameter information.

[0050] In a specific embodiment, the model parameters include depth information and contour edge information of the target virtual three-dimensional model; the step of "processing the second image based on the model parameters by the image processing module in the target image generation tool to generate a target image having the target style and matching the target virtual three-dimensional model" includes:

[0051] The second image is processed based on the depth information and the contour edge information by an image processing module in the target image generation tool to generate a target image having the target style and matching the target virtual three-dimensional model.

[0052] For example, see Figure 3The user can input the first image of the target virtual three-dimensional model into the target image generation tool, and the image processing module in the target image generation tool processes the second image output by the target image generation model based on the model parameters to generate a target image with the target style and matching the target virtual three-dimensional model.

[0053] The embodiment of the present application uses an AI painting generation tool (i.e., sd tool) to quickly implement style training, and the image production effect is stable. By combining the advantages of high quality and clear structure of the virtual three-dimensional model, the technical process of efficiently and high-quality generation of two-dimensional images corresponding to the virtual three-dimensional model can be realized. In addition, the embodiment of the present application can quickly package the material and convert the atmosphere style of the current virtual three-dimensional model to achieve an artistic effect that meets the requirements, and at the same time, it can also make the final generated degree and style adjustable, and define a variety of effects in combination with parameters.

[0054] Based on the above introduction, the game interaction method of the present application will be further illustrated by examples below. The specific embodiments of the game interaction method are described as follows.

[0055] In one embodiment, the target image generation tool is configured with a plurality of candidate image generation models, and a candidate image generation model is used to generate an image of a certain style; specifically, the step “before processing the first image by the target image generation model in the target image generation tool to obtain a second image having a target style” further includes:

[0056] displaying, via a graphical user interface, a first editing page of the target image generation tool, the first editing page including a model selection control;

[0057] In response to a triggering operation on a model selection control, displaying model identifiers of a plurality of candidate image generation models on the first editing page;

[0058] A target model identifier is selected from a plurality of candidate model identifiers, and an image generation model corresponding to the target model identifier is used as a target image generation model.

[0059] In an embodiment of the present application, a computer device can display a first editing page of the target image generation tool through a graphical user interface, and the first editing page includes a model selection control. A user can use the model selection control to display model identifiers of multiple candidate image generation models on the first editing page, and then select a target model identifier, and use the image generation model corresponding to the target model identifier as the target image generation model, thereby enabling the user to select image generation models of different styles according to their own needs, thereby generating a target image of a corresponding style based on the first image.

[0060] Optionally, the method further includes:

[0061] Acquire at least one two-dimensional image of a preset three-dimensional model and preset target style information;

[0062] Each of the two-dimensional images and the target style information is input into a specified image generation tool to perform model training in a preset image generation model configured in the specified image generation tool to obtain a trained target image generation model, wherein the target image generation model is used to generate a two-dimensional image of a target style.

[0063] Among them, the image generation model can be a Lora model. The Lora model is also used as a plug-in for the Stable Diffusion tool, allowing users to train a Lora model with a specific style using a small amount of data without modifying the SD tool, so as to convert the two-dimensional image of the virtual three-dimensional model into an image of the target style.

[0064] Furthermore, the step of “after inputting each of the two-dimensional images and the target style information into a designated image generation tool to perform model training in a preset image generation model configured by the designated image generation tool to obtain a trained target image generation model” also includes:

[0065] Displaying a second editing page of the designated image generation tool through a graphical user interface, wherein the second editing page includes a plurality of parameter interfaces, wherein the parameter interface is used to obtain parameter information of a parameter dimension corresponding to the parameter interface, and one type of parameter information is used to adjust an image attribute of a two-dimensional image;

[0066] Obtain target parameter information input through at least one parameter interface, associate the target parameter information with the target image generation model, and obtain a target image generation tool, so as to configure the target parameter information and the target image generation model as the default parameter information and the initial image generation model of the specified image generation tool, respectively.

[0067] In one embodiment, the step “after inputting the first image into a target image generation tool” further includes:

[0068] A third editing page of the target image generation tool is displayed through a graphical user interface, and the third editing page includes a parameter adjustment area and an image preview area, wherein the parameter adjustment area includes multiple parameter interfaces, parameter information received by a parameter interface is used to adjust an image attribute of the first image, and the image preview area is used to preview and display the target image.

[0069] For example, see Figure 4aThe computer device provides a graphical user interface, and displays a third editing page of the target image generation tool through the graphical user interface. The third editing page includes a parameter adjustment area and an image preview area, wherein the parameter adjustment area includes multiple parameter interfaces, and parameter information received by a parameter interface is used to adjust an image attribute of the first image. At this time, the image preview area is used to preview and display the target image.

[0070] Furthermore, the method further comprises:

[0071] In response to a confirmation operation of target parameter information input into the target parameter interface in the parameter interface, the target image is updated based on the target parameter information to obtain a new target image, and the new target image is displayed in the image preview area.

[0072] Optionally, the third editing page displays a model selection control and further includes:

[0073] In response to a trigger operation on a model selection control, displaying model identifiers of a plurality of candidate image generation models on the third editing page;

[0074] Selecting a new model identifier from a plurality of candidate model identifiers, determining an image generation model corresponding to the new model identifier as a new target image generation model, and replacing the target image generation model with the new target image generation model;

[0075] The target image is updated based on the new target image generation model to obtain a new target image, and the new target image is displayed in the image preview area.

[0076] Specifically, after obtaining the target image of the target style, the user can also display the model selection control through the third editing page, the user can perform a trigger operation on the model selection control, and the computer device responds to the trigger operation on the model selection control and displays the model identifiers of multiple candidate image generation models on the third editing page. The candidate image generation model corresponding to each model identifier corresponds to a style, and the user can select a new model identifier from the multiple candidate model identifiers. After determining the image generation model corresponding to the new model identifier, the computer device uses it as the new target image generation model and replaces the target image generation model with the new target image generation model; then, the target image is updated based on the new target image generation model to obtain a new target image of a new style, and the new target image is displayed through the image preview area.

[0077] Based on the above introduction, the image processing method of the present application will be further illustrated by examples below. The specific embodiments of the image processing method are described as follows.

[0078] The embodiment of the present application will provide a method for generating a target image generation tool. Specifically, the target image generation tool can be obtained by training a Lora model (i.e., an image generation model) of a material type in a StableDiffusion tool. The specific training process is as follows:

[0079] (1) First, you need to prepare material images of the target style, usually more than 15 images, and label them with keywords (such as buildings, rocks, or rivers, etc.), and put them into a training expansion module of the Stable Diffusion tool (such as Additional Networks) for training. The basic Lora model uses a relatively short period of time to learn the content of the material image and achieves a target Lora model with different intensities of style effects according to different training degrees. After obtaining this target Lora model, we will put it into the tool interface of the Stable Diffusion tool for parameter adjustment. The adjusted parameters involve large model type, plug-in vae model type, positive and negative keywords, sampling method, number of iterations, cfg and redraw amplitude, as well as proportion debugging with other Lora models, ControNet processor type, etc. Specifically, the embodiments of the present application can exhaustively enumerate the above parameters to find the Lora model that best meets the style requirements.

[0080] Optionally, the Lora model that best meets the style requirements can be determined by viewing the final effect of the test image obtained based on the material image. For example, if a game designer needs to generate a Lora model that meets the style of Project A, the current Lora model can be evaluated from several dimensions, such as art style conformity, picture completion (text-derived images and picture-derived images), stability, generalization, and landing effect verification, to determine the Lora model that best meets the style requirements.

[0081] (2) Then, the parameters of the Lora model can be exhaustively tested according to the project requirements to obtain an SD parameter that best meets the required effect. Specifically, the exhaustive test of the parameters is that the game production personnel set multiple different SD parameters and manually compare the corresponding image effects, and then perform the above tests to determine the final parameters. Among them, the SD parameters refer to the parameters such as high-definition restoration, total batch number, prompt word relevance, random seed, etc. used in the Stable Diffusion tool. For example, keyword descriptions are usually divided into three categories, including completion quality (such as 8K or best quality, etc.), followed by picture description (such as realism, daytime, architecture, etc.), and the third is the corresponding Lora model call noun (such as Lora:HGSyanshi:0.3, which is also the model identifier).

[0082] (3) Next, the Stable Diffusion tool and the Lora model obtained above can be tested. Specifically, the current parameters of the Stable Diffusion tool can be used with a randomly acquired two-dimensional image of a new virtual three-dimensional model for testing. The keywords and parameters tested previously can be used with ControlNet’s depth and SoftEdge to generate the model’s underlay map, thus obtaining the most suitable parameters and process.

[0083] In the embodiment of the present application, corresponding adjustments can be made according to the type of image that the project wants to generate. For example, to generate realistic bricks, first ensure that the quality meets the requirements and use the above parameters to match the keywords and Lora call nouns related to the quality completion to generate a realistic, high-quality realistic brick picture, then obtain the basic keywords, and then describe the picture according to whether the specific generated floor tiles are red, blue, or white, or floor tiles with grass or without grass, so as to obtain floor tiles that meet the corresponding requirements and obtain the target image of the target style.

[0084] (4) For details, please refer to Figure 4b The process settings of the above-mentioned Stable Diffusion tool can be made into a node process applet using comfyUI. When users need to use it in the future, they only need to adjust some key parameters according to their own needs to achieve programmatic optimization of two-dimensional images and improve users' image production efficiency.

[0085] For example, users can make some fine-tuning according to actual needs, such as adjusting the redrawing range of keyword descriptions, etc. The redrawing range of keyword descriptions depends on the actual usage scenario. For example, if red bricks are needed, or white bricks are needed, the parameters can be fine-tuned. However, if it involves changing a style or element, such as switching from a realistic style to a cartoon style, it involves reselecting the Lora model for replacement and changing the weight.

[0086] In summary, the embodiment of the present application provides an image processing method, which can obtain a first image to be processed, wherein the first image is a two-dimensional image of a target virtual three-dimensional model, and the two-dimensional image carries at least one model parameter of the target virtual three-dimensional model; then, the first image is input into a target image generation tool, and the first image is processed by a target image generation model in the target image generation tool to obtain a second image with a target style, wherein the target image generation model is used to generate an image with a target style; finally, the second image is processed based on the model parameters by an image processing module in the target image generation tool to generate a target image with the target style and matching the target virtual three-dimensional model. In the embodiment of the present application, by inputting the initial two-dimensional image of the virtual three-dimensional model into the target image generation tool, the target image generation model configured by the target image generation tool can automatically optimize the two-dimensional image to automatically adjust the two-dimensional image to a two-dimensional image of the target style, and the two-dimensional image is generated by automatic optimization, thereby saving the production time of game designers when optimizing the two-dimensional image, simplifying the steps of two-dimensional image production, realizing the programming of two-dimensional image detail production, and improving the efficiency of two-dimensional image production.

[0087] This embodiment also provides an image processing device, which can be integrated into a terminal device. Figure 5 As shown, the image processing device may include:

[0088] An acquisition unit 201 is used to acquire a first image to be processed, where the first image is a two-dimensional image of a target virtual three-dimensional model, and the two-dimensional image carries at least one model parameter of the target virtual three-dimensional model;

[0089] A first processing unit 202 is used to input the first image into a target image generation tool, and process the first image through a target image generation model in the target image generation tool to obtain a second image with a target style, wherein the target image generation model is used to generate an image with a target style;

[0090] The second processing unit 203 is used to process the second image based on the model parameters through the image processing module in the target image generation tool to generate a target image with the target style and matching the target virtual three-dimensional model.

[0091] In some embodiments, the image processing apparatus includes a processing subunit configured to:

[0092] displaying, via a graphical user interface, a first editing page of the target image generation tool, the first editing page including a model selection control;

[0093] In response to a triggering operation on a model selection control, displaying model identifiers of a plurality of candidate image generation models on the first editing page;

[0094] A target model identifier is selected from a plurality of candidate model identifiers, and an image generation model corresponding to the target model identifier is used as a target image generation model.

[0095] In some embodiments, the image processing apparatus includes a processing subunit configured to:

[0096] The second image is processed based on the depth information and the contour edge information by an image processing module in the target image generation tool to generate a target image having the target style and matching the target virtual three-dimensional model.

[0097] In some embodiments, the image processing apparatus includes a processing subunit configured to:

[0098] Acquire at least one two-dimensional image of a preset three-dimensional model and preset target style information;

[0099] Each of the two-dimensional images and the target style information is input into a specified image generation tool to perform model training in a preset image generation model configured in the specified image generation tool to obtain a trained target image generation model, wherein the target image generation model is used to generate a two-dimensional image of a target style.

[0100] In some embodiments, the image processing apparatus includes a processing subunit configured to:

[0101] Displaying a second editing page of the designated image generation tool through a graphical user interface, wherein the second editing page includes a plurality of parameter interfaces, wherein the parameter interface is used to obtain parameter information of a parameter dimension corresponding to the parameter interface, and one type of parameter information is used to adjust an image attribute of a two-dimensional image;

[0102] Obtain target parameter information input through at least one parameter interface, associate the target parameter information with the target image generation model, and obtain a target image generation tool, so as to configure the target parameter information and the target image generation model as the default parameter information and the initial image generation model of the specified image generation tool, respectively.

[0103] In some embodiments, the image processing apparatus includes a processing subunit configured to:

[0104] The second image is processed based on the model parameters and the target parameter information by the image processing module in the target image generation tool to generate a target image having the target style and matching the target virtual three-dimensional model and the target parameter information.

[0105] In some embodiments, the image processing apparatus includes a processing subunit configured to:

[0106] A third editing page of the target image generation tool is displayed through a graphical user interface, and the third editing page includes a parameter adjustment area and an image preview area, wherein the parameter adjustment area includes multiple parameter interfaces, parameter information received by a parameter interface is used to adjust an image attribute of the first image, and the image preview area is used to preview and display the target image.

[0107] In some embodiments, the image processing apparatus includes a processing subunit configured to:

[0108] In response to a confirmation operation of target parameter information input into the target parameter interface in the parameter interface, the target image is updated based on the target parameter information to obtain a new target image, and the new target image is displayed in the image preview area.

[0109] In some embodiments, the image processing apparatus includes a processing subunit configured to:

[0110] In response to a trigger operation on a model selection control, displaying model identifiers of a plurality of candidate image generation models on the third editing page;

[0111] Selecting a new model identifier from a plurality of candidate model identifiers, determining an image generation model corresponding to the new model identifier as a new target image generation model, and replacing the target image generation model with the new target image generation model;

[0112] The target image is updated based on the new target image generation model to obtain a new target image, and the new target image is displayed in the image preview area.

[0113] The embodiment of the present application discloses an image processing device, wherein an acquisition unit 201 acquires a first image to be processed, wherein the first image is a two-dimensional image of a target virtual three-dimensional model, and the two-dimensional image carries at least one model parameter of the target virtual three-dimensional model; a first processing unit 202 inputs the first image into a target image generation tool, and processes the first image through a target image generation model in the target image generation tool to obtain a second image with a target style, wherein the target image generation model is used to generate an image with a target style; a second processing unit 203 processes the second image based on the model parameters through an image processing module in the target image generation tool to generate a target image with the target style and matching the target virtual three-dimensional model. In the embodiment of the present application, by inputting the initial two-dimensional image of the virtual three-dimensional model into the target image generation tool, the target image generation model configured by the target image generation tool can automatically optimize the two-dimensional image to automatically adjust the two-dimensional image to a two-dimensional image of the target style, and generate the two-dimensional image through automatic optimization, thereby saving the production time of game designers when optimizing the two-dimensional image, simplifying the steps of two-dimensional image production, realizing the programming of two-dimensional image detail production, and improving the efficiency of two-dimensional image production.

[0114] Accordingly, an embodiment of the present application further provides an electronic device, which may be a terminal, and the terminal may be a smart phone, a tablet computer, a laptop computer, a touch screen, a game console, a personal computer (PC, Personal Computer), a personal digital assistant (Personal Digital Assistant, PDA) and other terminal devices. Alternatively, the electronic device may be a server.

[0115] like Figure 6 As shown, Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 300 includes a processor 301 having one or more processing cores, a memory 302 having one or more computer-readable storage media, and a computer program stored in the memory 302 and executable on the processor. The processor 301 is electrically connected to the memory 302. Those skilled in the art will appreciate that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or arrange components differently.

[0116] The processor 301 is the control center of the electronic device 300. It uses various interfaces and lines to connect various parts of the entire electronic device 300, and executes various functions and processes data of the electronic device 300 by running or loading software programs and / or units stored in the memory 302, and calling data stored in the memory 302, so as to monitor the electronic device 300 as a whole. The processor 301 can be a central processing unit CPU, a graphics processing unit GPU, a network processor (Network Processor, NP), etc., and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application.

[0117] In the embodiment of the present application, the processor 301 in the electronic device 300 will load instructions corresponding to the processes of one or more application programs into the memory 302 according to the following steps, and the processor 301 will run the application programs stored in the memory 302 to implement various functions, such as:

[0118] Acquire a first image to be processed, where the first image is a two-dimensional image of a target virtual three-dimensional model, and the two-dimensional image carries at least one model parameter of the target virtual three-dimensional model;

[0119] Inputting the first image into a target image generation tool, and processing the first image by a target image generation model in the target image generation tool to obtain a second image with a target style, wherein the target image generation model is used to generate an image with a target style;

[0120] The second image is processed based on the model parameters by an image processing module in the target image generation tool to generate a target image having the target style and matching the target virtual three-dimensional model.

[0121] The electronic device provided in the embodiment of the present application can obtain a first image to be processed, wherein the first image is a two-dimensional image of a target virtual three-dimensional model, and the two-dimensional image carries at least one model parameter of the target virtual three-dimensional model; then, the first image is input into a target image generation tool, and the first image is processed by a target image generation model in the target image generation tool to obtain a second image with a target style, wherein the target image generation model is used to generate an image with a target style; finally, the second image is processed based on the model parameters by an image processing module in the target image generation tool to generate a target image with the target style and matching the target virtual three-dimensional model. In the embodiment of the present application, by inputting the initial two-dimensional image of the virtual three-dimensional model into the target image generation tool, the target image generation model configured by the target image generation tool can automatically optimize the two-dimensional image to automatically adjust the two-dimensional image to a two-dimensional image of the target style, and the two-dimensional image is generated by automatic optimization, thereby saving the production time of game designers when optimizing the two-dimensional image, simplifying the steps of two-dimensional image production, realizing the programming of two-dimensional image detail production, and improving the efficiency of two-dimensional image production.

[0122] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0123] Optional, such as Figure 6 As shown, the electronic device 300 further includes: a touch screen 303, a radio frequency circuit 304, an audio circuit 305, an input unit 306, and a power supply 307. The processor 301 is electrically connected to the touch screen 303, the radio frequency circuit 304, the audio circuit 305, the input unit 306, and the power supply 307, respectively. Those skilled in the art can understand that Figure 6 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0124] The touch display screen 303 can be used to display a graphical user interface and receive operation instructions generated by the user acting on the graphical user interface. The touch display screen 303 may include a display panel and a touch panel. Among them, the display panel can be used to display information input by the user or information provided to the user and various graphical user interfaces of the electronic device, and these graphical user interfaces can be composed of graphics, text, icons, videos and any combination thereof. Optionally, the display panel can be configured in the form of a liquid crystal display (LCD, Liquid Crystal Display), an organic light-emitting diode (OLED, Organic Light-Emitting Diode) and the like. The touch panel can be used to collect the user's touch operation on or near it (such as the user uses any suitable object or attachment such as a finger, a stylus, etc. on the touch panel or near the touch panel), and generate corresponding operation instructions, and the operation instructions execute corresponding programs. Optionally, the touch panel may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch direction, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 301, and can receive the command sent by the processor 301 and execute it. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it is transmitted to the processor 301 to determine the type of touch event, and then the processor 301 provides a corresponding visual output on the display panel according to the type of touch event. In an embodiment of the present application, the touch panel and the display panel can be integrated into the touch display screen 303 to realize the input and output functions. However, in some embodiments, the touch panel and the touch panel can be used as two independent components to realize the input and output functions. That is, the touch display screen 303 can also be used as a part of the input unit 306 to realize the input function.

[0125] The radio frequency circuit 304 may be used to send and receive radio frequency signals, so as to establish wireless communication with a network device or other electronic devices through wireless communication, and to send and receive signals with the network device or other electronic devices.

[0126] The audio circuit 305 can be used to provide an audio interface between the user and the electronic device through a speaker and a microphone. The audio circuit 305 can transmit the electrical signal converted from the received audio data to the speaker, which is converted into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 305 and converted into audio data, and then the audio data is output to the processor 301 for processing, and then sent to another electronic device through the radio frequency circuit 304, or the audio data is output to the memory 302 for further processing. The audio circuit 305 may also include an earplug jack to provide communication between an external headset and the electronic device.

[0127] The input unit 306 may be used to receive input numbers, character information or user feature information (such as fingerprint, iris, facial information, etc.), and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0128] The power supply 307 is used to supply power to various components of the electronic device 300. Optionally, the power supply 307 can be logically connected to the processor 301 through a power management system, so that the power management system can manage charging, discharging, and power consumption. The power supply 307 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0129] although Figure 6 Not shown, the electronic device 300 may also include a camera, a sensor, a wireless fidelity module, a Bluetooth module, etc., which will not be described in detail here.

[0130] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0131] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0132] To this end, an embodiment of the present application provides a computer-readable storage medium, in which multiple computer programs are stored, and the computer program can be loaded by a processor to execute any image processing method provided in the embodiment of the present application. The computer program can execute the steps of the following image processing method:

[0133] Acquire a first image to be processed, where the first image is a two-dimensional image of a target virtual three-dimensional model, and the two-dimensional image carries at least one model parameter of the target virtual three-dimensional model;

[0134] Inputting the first image into a target image generation tool, and processing the first image by a target image generation model in the target image generation tool to obtain a second image with a target style, wherein the target image generation model is used to generate an image with a target style;

[0135] The second image is processed based on the model parameters by an image processing module in the target image generation tool to generate a target image having the target style and matching the target virtual three-dimensional model.

[0136] Due to the computer program stored in the storage medium, a first image to be processed can be obtained, wherein the first image is a two-dimensional image of a target virtual three-dimensional model, and the two-dimensional image carries at least one model parameter of the target virtual three-dimensional model; then, the first image is input into a target image generation tool, and the first image is processed by a target image generation model in the target image generation tool to obtain a second image with a target style, wherein the target image generation model is used to generate an image with a target style; finally, the second image is processed based on the model parameters by an image processing module in the target image generation tool to generate a target image with the target style and matching the target virtual three-dimensional model. In an embodiment of the present application, by inputting an initial two-dimensional image of a virtual three-dimensional model into a target image generation tool, the target image generation model configured by the target image generation tool can automatically optimize the two-dimensional image to automatically adjust the two-dimensional image to a two-dimensional image of a target style, and the two-dimensional image is generated by automated optimization, thereby saving the production time of game designers when optimizing two-dimensional images, simplifying the steps of two-dimensional image production, realizing the programming of two-dimensional image detail production, and improving the efficiency of two-dimensional image production.

[0137] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0138] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0139] Since the computer program stored in the computer-readable storage medium can execute any image processing method provided in the embodiments of the present application, the beneficial effects that can be achieved by any image processing method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0140] According to one aspect of the present application, a computer program product or a computer program is also provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the methods provided in various optional implementations of the above embodiments.

[0141] In the above-mentioned embodiments of the image processing device, computer-readable storage medium, electronic device, and computer program product, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the above-mentioned image processing device, computer-readable storage medium, computer program product, electronic device, and corresponding units can refer to the description of the image processing method in the above embodiment, and will not be repeated here.

[0142] The above is a detailed introduction to an image processing method, device, electronic device, computer-readable storage medium and computer program product provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: include: Acquire a first image to be processed, where the first image is a two-dimensional image of a target virtual three-dimensional model, and the two-dimensional image carries at least one model parameter of the target virtual three-dimensional model; Inputting the first image into a target image generation tool, and processing the first image by a target image generation model in the target image generation tool to obtain a second image with a target style, wherein the target image generation model is used to generate an image with a target style; The second image is processed based on the model parameters by an image processing module in the target image generation tool to generate a target image having the target style and matching the target virtual three-dimensional model.

2. The method according to claim 1, characterized in that The target image generation tool is configured with a plurality of candidate image generation models, and a candidate image generation model is used to generate an image of a certain style; Before the first image is processed by the target image generation model in the target image generation tool to obtain a second image with a target style, the method further includes: displaying, via a graphical user interface, a first editing page of the target image generation tool, the first editing page including a model selection control; In response to a triggering operation on a model selection control, displaying model identifiers of a plurality of candidate image generation models on the first editing page; A target model identifier is selected from a plurality of candidate model identifiers, and an image generation model corresponding to the target model identifier is used as a target image generation model.

3. The method according to claim 1, characterized in that The model parameters include depth information and contour edge information of the target virtual three-dimensional model; The processing of the second image based on the model parameters by the image processing module in the target image generation tool to generate a target image having the target style and matching the target virtual three-dimensional model includes: The second image is processed based on the depth information and the contour edge information by an image processing module in the target image generation tool to generate a target image having the target style and matching the target virtual three-dimensional model.

4. The method according to claim 1, characterized in that: The method further comprises: Acquire at least one two-dimensional image of a preset three-dimensional model and preset target style information; Each of the two-dimensional images and the target style information is input into a specified image generation tool to perform model training in a preset image generation model configured in the specified image generation tool to obtain a trained target image generation model, wherein the target image generation model is used to generate a two-dimensional image of a target style.

5. The method according to claim 4, characterized in that After inputting each of the two-dimensional images and the target style information into a designated image generation tool to perform model training in a preset image generation model configured by the designated image generation tool to obtain a trained target image generation model, the method further includes: Displaying a second editing page of the designated image generation tool through a graphical user interface, wherein the second editing page includes a plurality of parameter interfaces, wherein the parameter interface is used to obtain parameter information of a parameter dimension corresponding to the parameter interface, and one type of parameter information is used to adjust an image attribute of a two-dimensional image; Obtain target parameter information input through at least one parameter interface, associate the target parameter information with the target image generation model, and obtain a target image generation tool, so as to configure the target parameter information and the target image generation model as the default parameter information and the initial image generation model of the specified image generation tool, respectively.

6. The method according to claim 5, characterized in that The target image generation tool is configured with a plurality of target parameter information, and the method further comprises: The second image is processed based on the model parameters and the target parameter information by the image processing module in the target image generation tool to generate a target image having the target style and matching the target virtual three-dimensional model and the target parameter information.

7. The method according to claim 1, characterized in that After inputting the first image into a target image generation tool, the method further includes: A third editing page of the target image generation tool is displayed through a graphical user interface, and the third editing page includes a parameter adjustment area and an image preview area, wherein the parameter adjustment area includes multiple parameter interfaces, parameter information received by a parameter interface is used to adjust an image attribute of the first image, and the image preview area is used to preview and display the target image.

8. The method according to claim 7, characterized in that The method further comprises: In response to a confirmation operation of target parameter information input into the target parameter interface in the parameter interface, the target image is updated based on the target parameter information to obtain a new target image, and the new target image is displayed in the image preview area.

9. The method according to claim 7, characterized in that: The third editing page displays a model selection control and also includes: In response to a trigger operation on a model selection control, displaying model identifiers of a plurality of candidate image generation models on the third editing page; Selecting a new model identifier from a plurality of candidate model identifiers, determining an image generation model corresponding to the new model identifier as a new target image generation model, and replacing the target image generation model with the new target image generation model; The target image is updated based on the new target image generation model to obtain a new target image, and the new target image is displayed in the image preview area.

10. An image processing device, characterized in that: include: an acquisition unit, configured to acquire a first image to be processed, wherein the first image is a two-dimensional image of a target virtual three-dimensional model, and the two-dimensional image carries at least one model parameter of the target virtual three-dimensional model; a first processing unit, configured to input the first image into a target image generation tool, and process the first image using a target image generation model in the target image generation tool to obtain a second image having a target style, wherein the target image generation model is used to generate an image of a target style; The second processing unit is used to process the second image based on the model parameters through the image processing module in the target image generation tool to generate a target image with the target style and matching the target virtual three-dimensional model.

11. An electronic device, characterized in that: The system comprises a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute the steps of the image processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps of the image processing method according to any one of claims 1 to 9.

Images