Image generation method, device and equipment, readable storage medium and program product

Abstract

The invention provides an image generation method and device, equipment, a readable storage medium and a program product, and can be applied to the fields of games, artificial intelligence and the like, the method comprises the steps that input texts of a game object description text and an image feature description text are acquired, and the game object description text comprises a game object style identifier and a game object category; image generation processing is carried out according to the input text, multiple candidate game object images are obtained, and the candidate game object images comprise target game objects; quality detection is conducted on the multiple candidate game object images, a target game object image is determined from the multiple candidate game object images according to the quality detection result, and the target game object image is the game object image with the highest image quality of the target game object included in the multiple candidate game object images. By adopting the mode, the game object image can be efficiently generated according to the input text, and the image quality of the game object contained in the generated game object image is high.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image generating method, an image generating apparatus, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Electronic games, also known as video games, refer to all interactive games that rely on electronic equipment platforms to run. With the rapid development of computer technology, the number of electronic games is increasing, and the number of electronic game players is also increasing. Players' demand for electronic games is changing with each passing day. Therefore, in order to meet the needs of the player group, electronic game developers usually iterate and update electronic games.

[0003] In the process of updating electronic games, it is necessary to update the original image containing a specific game object while retaining the main features of the original image. The existing method is usually for professional technicians to redraw the original image containing a specific game object to obtain a new game image. However, this method is severely limited by the drawing technology level and imagination of professional technicians, and there are problems such as unstable image quality of the new game image and low efficiency of drawing images. Therefore, how to efficiently and high-quality generate game object images containing specific game objects is a problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide an image generation method, apparatus, device, readable storage medium, and program product, which can efficiently generate a game object image based on input text, and the image quality of the game object contained in the generated game object image is high.

[0005] On the one hand, an embodiment of the present application provides an image processing method, the method comprising:

[0006] Acquire input text, the input text includes game object description text and image feature description text, the game object description text includes a game object style identifier and a game object category, and the game object indicated by the game object style identifier and the game object category is a target game object;

[0007] Performing image generation processing according to the input text to obtain a plurality of candidate game object images, wherein the candidate game object images include the target game object, and image features displayed by the candidate game object images match the image features described by the image feature description text, and the image features described by the image feature description text include one or two of the following: image scene, and features of the game object in the image;

[0008] Quality detection is performed on the multiple candidate game object images, and a target game object image is determined from the multiple candidate game object images based on the quality detection result, wherein the target game object image is the game object image with the highest image quality among the multiple candidate game object images.

[0009] On the one hand, an embodiment of the present application provides an image generating device, the device comprising:

[0010] An acquisition unit, used to acquire input text, the input text includes game object description text and image feature description text, the game object description text includes a game object style identifier and a game object category, and the game object indicated by the game object style identifier and the game object category is a target game object;

[0011] A processing unit, configured to perform image generation processing according to the input text to obtain a plurality of candidate game object images, wherein the candidate game object images include the target game object, and image features displayed by the candidate game object images match the image features described by the image feature description text, and the image features described by the image feature description text include one or two of the following: image scene, and features of the game object in the image;

[0012] A detection unit is used to perform quality detection on the multiple candidate game object images, and determine a target game object image from the multiple candidate game object images based on the quality detection result, wherein the target game object image is the game object image with the highest image quality among the multiple candidate game object images.

[0013] On the one hand, an embodiment of the present application provides a computer device, comprising: a processor and a memory, the processor and the memory are interconnected, wherein the memory stores executable program code, and the processor is used to call the executable program code to implement the image generation method provided in the embodiment of the present application.

[0014] Accordingly, an embodiment of the present application further provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is executed on a computer, the computer implements the image generation method provided in the embodiment of the present application.

[0015] Accordingly, the embodiment of the present application further provides a computer program product, the computer program product includes a computer program or a computer instruction, and the computer program or the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer program or the computer instruction from the computer-readable storage medium, and the processor executes the computer program or the computer instruction, so that the computer device implements the image generation method provided in the embodiment of the present application.

[0016] In the present application, an input text including a game object description text and an image feature description text is obtained, the game object description text includes a game object style identifier and a game object category, and the image feature description text describes an image feature including one or two of the following: an image scene, and features of a game object in the image; image generation processing is performed according to the input text to obtain a plurality of candidate game object images, wherein the candidate game object images include a target game object; quality detection is performed on the plurality of candidate game object images, and a target game object image is determined from the plurality of candidate game object images according to the quality detection results, wherein the target game object image is a game object image with the highest image quality of the target game object included in the plurality of candidate game object images. Through the image generation method provided in the embodiment of the present application, multiple candidate game object images can be generated according to the input text containing game object description text and image feature description text, and each candidate game object image includes the target game object, thereby realizing efficient generation of game object images containing the target game object (i.e., specific game object); the image features displayed by the candidate game object images match the image features described by the image feature description text, that is, it is possible to generate images of the target game object in different scenes and styles or images of different forms of the target game object, thereby realizing controllable image scenes and game object features in images, and greatly enriching the diversity of images associated with the target game object; quality detection of multiple candidate game object images can realize image screening, so that the images containing the target game object images are of high quality, the details of the target game objects included in the images are richer, and they are more in line with art requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a schematic diagram of the system architecture of an image generation system provided in an embodiment of the present application;

[0019] Figure 2 It is a flowchart of an image generation method provided in an embodiment of the present application;

[0020] Figure 3 An image generation method provided by an embodiment of the present application is shown;

[0021] Figure 4 A method for detecting the quality of a game object image provided by an embodiment of the present application is shown;

[0022] Figure 5 It is a flowchart of another model training method of an image generation model provided in an embodiment of the present application;

[0023] Figure 6 A model training method provided by an embodiment of the present application is shown;

[0024] Figure 7 Another model training method provided by an embodiment of the present application is shown;

[0025] Figure 8 is a schematic diagram of a model training interface provided in an embodiment of the present application;

[0026] Fig. 9 is a structural block diagram of an image generating device provided in an embodiment of the present application;

[0027] Fig.10 It is a structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] It should be noted that the descriptions of "first", "second", etc. involved in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the technical features defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0030] In the process of updating electronic games, it is necessary to update the secondary features of the game objects without changing the main features of the game objects. The existing method is usually for professional technicians (such as game painters) to update and draw according to the original image of the game object and the needs of players to obtain a game object image containing the game object. However, the quality of the game object image in this method mainly depends on the drawing technology level and imagination of the professional technicians. For complex game objects or more requirements, this method may result in poor quality of the game object image and low efficiency in drawing the game object image.

[0031] Based on this, an embodiment of the present application provides an image generation method, which can obtain input text, the input text includes game object description text and image feature description text, the game object description text includes game object style identification and game object category, the game object indicated by the game object style identification and the game object category is the target game object; image generation processing is performed according to the input text to obtain multiple candidate game object images, the candidate game object images include the target game object, and the image features displayed by the candidate game object images match the image features described by the image feature description text, the image features described in the image feature description text include one or two of the following: image scene, features of the game object in the image; quality detection is performed on the multiple candidate game object images, and the target game object image is determined from the multiple candidate game object images based on the quality detection results, the target game object image is the game object image with the highest image quality of the target game object included in the multiple candidate game object images. According to the image generation method provided in the embodiment of the present application, multiple candidate game object images can be generated according to the input text containing game object description text and image feature description text, thereby realizing efficient generation of game object images of the target game object (i.e., specific game object); the image features displayed by the candidate game object images match the image features described by the image feature description text, that is, it is possible to generate images of the target game object in different scenes and styles or images of the target game object in different forms, thereby realizing controllable image scenes and game object features in images, and greatly enriching the diversity of images associated with the target game object; quality detection of multiple candidate game object images can realize image screening, so that the image quality of the target game object image is high.

[0032] The image generation method provided in the embodiment of the present application can be applied to the field of cloud computing. Cloud computing is a computing model that can distribute computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called a "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, expanded at any time, and paid for by use. When the target game object is more complex, the image features are more complex, or a large number of game object images need to be generated, the image generation method provided in the embodiment of the present application can be implemented using a cloud computing device. For example: the terminal device can obtain input text, the input text includes a game object description text and an image feature description text, the game object description text includes a game object style identifier and a game object category, the terminal device sends the input text to a cloud computing server, and the cloud computing server generates multiple candidate game object images using the image generation method provided in the embodiment of the present application, and performs quality detection on the multiple candidate game object images to obtain the target game object image; the cloud computing server then sends the target game object image to the terminal device. The image generation method provided in the embodiment of the present application can efficiently and high-quality generate a game object image containing a specific game object.

[0033] The architecture of the image generation system provided in the embodiments of the present application will be introduced below in conjunction with the accompanying drawings.

[0034] See also Figure 1 , which is a schematic diagram of the system architecture of an image generation system provided by an embodiment of the present application, and the image generation system includes: a terminal device 101, a server 102 and a database 103, and the server 102 can exchange data with the terminal device 101 and the database 103. Among them:

[0035] The terminal device 101 can exchange data with the server 102, and the terminal device 101 can also interact with the object to obtain the relevant text input by the object. The terminal device 101 can be a handheld device with communication function (such as a smart phone, a tablet computer), a computing device (such as a personal computer (PC), a car terminal, an intelligent voice interaction device, a wearable device or other intelligent device, but is not limited to this.

[0036] The server 102 can receive text sent by the terminal device 101, and generate an image of a game object according to the text, or send the image to the terminal device 101. The server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0037] Database 103 can be used to store data related to game objects, and can also be used for images of game objects generated by server 102 based on text. Database 103 can be a database located in server 102, that is, it can be a database built into or carried by the server; it can also be an external database connected to server 102, such as a cloud database (i.e., a database deployed in the cloud), which can be specifically based on any one of private cloud, public cloud, hybrid cloud, edge cloud, etc., so that the cloud database focuses on different functions. For example: For a database deployed in a private cloud, the basic cloud hardware is the user's personal equipment, and it focuses more on serving a small number of users, while a database deployed in a public cloud is deployed based on a cloud platform provided by a third party, which allows the data stored in the database to achieve data sharing, and any user's data can be stored in the database, and any user can also use the data in the database.

[0038] The following will explain in detail Figure 1 The image generation system shown works as follows:

[0039] The terminal device 101 can display a text input interface and interact with the object to obtain the text to be processed of the object input; the terminal device 101 determines the input text according to the text to be processed, and the input text includes a game object description text and an image feature description text, wherein the game object description text includes a game object style identifier and a game object category, and the game object indicated by the game object style identifier and the game object category is the target game object; the terminal device 101 sends the input text to the server 102, and the server 102 performs image generation processing according to the input text to obtain multiple candidate game object images, wherein the candidate game object images include the target game object, and the image features displayed by the candidate game object images match the image features described in the image feature description text, and the image features described in the image feature description text include one or two of the following: image scene, features of the game object in the image; the server 102 then performs quality detection on the multiple candidate game object images, and determines the target game object image from the multiple candidate game object images according to the quality detection result, and the target game object image is the game object image with the highest image quality of the target game object included in the multiple candidate game object images. The server 102 stores the target game object image of the input text in the database 103, and sends the target game object image to the terminal device 101. The image generation method provided in the embodiment of the present application can generate multiple candidate game object images according to the input text containing the game object description text and the image feature description text, so as to realize the efficient generation of the game object image containing the target game object; the image features displayed by the candidate game object image match the image features described by the image feature description text, that is, it is possible to generate images of the target game object in different scenes and different styles or images of different forms of the target game object, so as to realize the controllability of the image scene and the game object features in the image, and greatly enrich the diversity of the images associated with the target game object; quality detection of multiple candidate game object images can realize image screening, so that the image quality of the target game object image is higher and the details of the target game object included in the image are richer.

[0040] It is to be understood that the schematic diagram of the image generation system described in the embodiment of the present application is to more clearly illustrate the image generation method of the embodiment of the present application, and does not constitute a limitation on the image generation method provided in the embodiment of the present application. For example, the image generation method provided in the embodiment of the present application can be executed by the server 102, and can also be executed by other servers or server clusters that are different from the server 102 and can communicate with the terminal device 101 and / or the server 102. It is known to those skilled in the art that Figure 1The number of terminal devices and servers in the embodiment is only illustrative. According to the business implementation needs, any number of terminal devices and servers can be configured. Moreover, with the evolution of the system architecture and the emergence of new business scenarios, the image generation method provided in the embodiment of the present application is also applicable to similar technical problems.

[0041] See also Figure 2 , Figure 2 The following is a flow chart of an image generation method provided as an example in an embodiment of the present application. The image generation method can be implemented by the above-mentioned server 102, or by other server devices. The flow of the image generation method provided in the embodiment of the present application includes but is not limited to:

[0042] S201. Obtain input text, wherein the input text includes game object description text and image feature description text, wherein the game object description text includes a game object style identifier and a game object category, and the game object indicated by the game object style identifier and the game object category is a target game object.

[0043] In the embodiment of the present application, the target game object can be a game object in a certain electronic game, and the characteristics of the target game object can include main characteristics and secondary characteristics. The main characteristics are characteristics that cannot be modified, such as: the name, number, theme style, image, etc. of the target game object; and the secondary characteristics are characteristics that can be modified, such as: the equipment, clothing, posture, and background of the target game object in the image. The main characteristics of the target game object can distinguish it from other game objects, and the secondary characteristics of the target game object can make the target game object have a change space, which is conducive to the update of the game object. When the electronic game is updated, it is usually necessary to modify the secondary characteristics of the target game object to meet actual needs. When modifying the secondary characteristics of the target game object, the image generation method provided in the embodiment of the present application can be used to generate a game object image containing the target game object with changed secondary characteristics according to the input text. The input text includes a game object description text and an image feature description text, and the game object description text includes a game object style identifier and a game object category identifier. The game object indicated by the game object style identifier and the game object category identifier is the target game object. The game object description text corresponds to the target game object one-to-one. The game object style identifier is used to indicate the theme style of the target game object; the game object category identifier is used to indicate the category to which the target game object belongs; and the image feature description text is used to describe the image features of the generated image. The game object description text in the input text can make the target game object in the generated image retain its main features, while the image feature description text can make the secondary features of the target game object in the generated image have certain changes. For example: the game object style identifier of the target game object is X, the game object category of the target game object is "people", and the image feature description text is "on the beach". The game object style identifier and the game object category can indicate the target game object; the image feature description text indicates the features of the generated image, that is, the scene in which the target game object is located in the generated image is "on the beach".

[0044] In one embodiment, the game object style identifier may indicate the theme style corresponding to the target game object, or may indicate the target game object itself. When the game object style identifier indicates the theme style corresponding to the target game object, the theme style of the image generated according to the input text matches the theme style corresponding to the target game object; when the game object style identifier indicates the target game object itself, the game object included in the image generated according to the input text matches the target game object, but the theme style of the generated image may change.

[0045] In one embodiment, the game object category of the target game object can be determined according to the category to which the target game object itself belongs. For example, if the target game object is "a dog", then the game object category of the target game object can be the "dog" category; if the target game object is "a boy", then the game object category of the target game object can be the "boy" category; the game object category can also be determined according to the game to which the target game object belongs. For example, if the target game object is "a character in game A", then the game object category of the target game object can be the "game A" category.

[0046] S202. Perform image generation processing according to the input text to obtain multiple candidate game object images, wherein the candidate game object images include the target game object, and image features displayed by the candidate game object images match the image features described by the image feature description text, and the image features described by the image feature description text include one or two of the following: image scene, features of the game object in the image.

[0047] In an embodiment of the present application, image generation processing is performed based on the input text to obtain multiple candidate game object images, and each candidate game object image includes the target game object. The candidate game object image may only contain a part of the target game object, for example: the target game object is a humanoid object, the candidate game object image may only contain the head of the target game object, that is, the candidate game object image may be a head portrait of the target game object; the candidate game object image may also contain the entire target game object, for example: the target game object is a humanoid object, the candidate game object image may contain the entire target game object, that is, the candidate game object image may be a full-body image of the target game object. The image features displayed by the candidate game object image match the image features described by the image feature description text of the input text, that is, the image content displayed in the candidate game object image matches the content described by the image feature description text of the input text. The image feature description text may include one or both of the image scene and the features of the game object in the image. The image scene may indicate the scene where the game object in the image is located, for example, the image scene may be "on the roof", "on the lawn", "in the forest" and other scenes; the features of the game object in the image may indicate the posture of the game object, the relative relationship between the game object and the equipment, the number of game objects, the angle of observing the game object and other features related to the game object. For example, the features of the game object in the image may indicate that the posture of the game object is "standing", "walking" or "squatting", and the features of the game object in the image may indicate that the relative relationship between the game object and the equipment is "carrying equipment", "holding equipment in the left hand" or "hanging equipment on the side of the body", etc.; the features of the game object in the image may indicate that the number of game objects is "two" or "multiple", etc.; the features of the game object in the image may indicate that the angle of observing the game object is "observing the front of the game object", "observing the side of the game object" or "observing the back of the game object", etc. The features of the game object in the image may indicate one or more of the above features at the same time. The method provided in the embodiment of the present application can generate candidate game object images of various angles, various states, and various scenes of the target game object based on the input text, thereby enriching the number of images containing the target game object, which is facilitating determining the update direction of the target game object based on the candidate game object images.

[0048] In one embodiment, the implementation method of performing image generation processing according to input text to obtain multiple candidate game object images can be: performing image generation processing according to the input text using a target image generation model to obtain multiple candidate game object images; wherein the target image generation model is obtained by adjusting some model parameters of the intermediate image generation model according to target difference data, and the target difference data is determined according to the first difference data and the second difference data; the first difference data is determined according to the original image and the predicted image of the target game object, and the predicted image is determined by performing image generation processing according to the game object style identifier and the game object category associated with the target game object using the intermediate image generation model; the second difference data is determined according to the initial category image and the predicted category image, the initial category image is determined by performing image generation processing according to the game object category using the initial image generation model, the predicted category image is determined by performing image generation processing according to the game object category using the intermediate image generation model, and the intermediate image generation model is determined according to the initial image generation model; the game object style displayed by the candidate game object image matches the game object style displayed by the original image. By using the target image generation model to perform image generation processing according to the input text, multiple images of candidate game objects can be obtained, and the target image generation model is trained based on the original image of the target game object and the initial image generation model. The specific training process of the target image generation model can be shown in the following steps S501-S504. The method provided in the embodiment of the present application can use the image generation model to generate multiple candidate object images according to the input text, which effectively improves the efficiency of generating images including the target game object and reduces the manpower cost consumed by generating the image.

[0049] See also Figure 3 , which shows an image generation method provided in an embodiment of the present application. Figure 3 , the upper left corner shows the original image containing the target game object. The input text is "X boy running on the road", where "X" is the game object style identifier of the target game object, "boy" is the game object category of the target game object, and "running on the road" is the image feature description text, which is used to indicate the image scene in the generated image and the game object features in the image. The input text is processed for image generation using the target image generation model to obtain multiple candidate game object images as shown in the figure, and the image quality of different candidate game object images may be different. The image generation method provided in the embodiment of the present application can quickly and efficiently generate game object images containing specific game objects, improve the image generation efficiency, and reduce the manpower cost consumed in generating images.

[0050] S203, performing quality inspection on the multiple candidate game object images, and determining a target game object image from the multiple candidate game object images based on the quality inspection result, wherein the target game object image is a game object image with the highest image quality of the target game object included in the multiple candidate game object images.

[0051] In the embodiment of the present application, the obtained multiple candidate game object images all contain the target game object, and the image features displayed by each candidate game object image match the image features described in the image feature description text, but the image quality of the target game object in different candidate game object images may be different, and some candidate game object images may have image feature errors, for example: the target game object in the candidate game object image has fewer similar features and larger image differences with the real target game object, that is, the target game object in the candidate game object image is different from the real target game object. In order to improve the quality of the output game object image, it is necessary to perform quality detection on the multiple candidate game object images, and determine the target game object image from the multiple candidate game object images based on the quality detection results, and the target game object image is the game object image with the highest image quality of the target game object included in the multiple candidate game object images.

[0052] In one embodiment, the implementation method of quality detection of multiple candidate game object images can be: determining first quality detection data of each candidate game object image in the multiple candidate game object images, the first quality detection data including one or more of an image pixel mean, an image grayscale standard deviation value, an image average gradient value, and an image entropy value; performing feature extraction processing on the foreground image area in each candidate game object image to obtain a first foreground image feature of each candidate game object image, the foreground image area being an image area where a target game object in the candidate game object image is located; performing feature extraction processing on the foreground image area in the original image of the target game object to obtain a second foreground image feature of the original image; determining second quality detection data of each candidate game object image based on the first foreground image feature of each candidate game object image and the second foreground image feature of the original image, the second quality detection data including one or both of an image mean square error value and an image peak signal-to-noise ratio; determining a quality detection result of each candidate game object image based on the first quality detection data and the second quality detection data of each candidate game object image, the quality detection result being used to evaluate the image quality of the target game object included in the candidate game object image. Each of the candidate game object images in the plurality of candidate game object images is processed to obtain first quality detection data of each candidate game object image, and the first quality detection data may include one or more of an image pixel mean, an image grayscale standard deviation, an image average gradient, and an image entropy value. The image pixel mean refers to the average value of the pixel values ​​of each pixel in the image, which reflects the average brightness of the image. The larger the image pixel mean, the better the image quality. The image grayscale standard deviation reflects the degree of dispersion of the pixel grayscale values ​​of each pixel in the image relative to the grayscale mean. The larger the image grayscale standard deviation, the better the image quality. The image average gradient value can reflect the detail contrast and texture transformation in the image, which reflects the clarity of the image to a certain extent. The larger the image average gradient value, the better the image quality. The image entropy value can indicate the average amount of information of the image, which measures the amount of information in the image from the perspective of information theory. The larger the image entropy value, the more information the image contains, and the better the image quality.The original image of the target game object is an image containing the target game object. In order to determine the image quality of the target game object in the candidate game object image, the image features of the target game object in the candidate game object image can be compared with the image features of the target game object in the original image. Therefore, the foreground image area in the candidate game object image can be subjected to feature extraction processing to obtain the first foreground image features of each candidate game object image, and the foreground image area is the image area where the target game object is located in the candidate game object image; the foreground image area in the original image of the target game object is subjected to feature extraction processing to obtain the second foreground image features of the original image; according to the first foreground image features of each candidate game object image and the second foreground image features of the original image, the second quality detection data of each candidate game object image is determined, and the second quality detection data can include one or both of the image mean square error value and the image peak signal-to-noise ratio. The calculation principle of the image mean square error value is to calculate the square of the difference between the feature value of each pixel point in the original image and the feature value of each pixel point in the candidate game object image, and then sum the squares of the differences corresponding to all pixels and average them. The image mean square error value can reflect the error between the candidate game object image and the original image. The smaller the image mean square error value, the smaller the difference between the candidate game object image and the original image. The image peak signal-to-noise ratio can be determined based on the image mean square error value, which can reflect the similarity between the candidate game object image and the original image. The larger the image peak signal-to-noise ratio, the more similar the candidate game object image is to the original image. The quality detection result of each candidate game object image is determined based on the first quality detection data and the second quality detection data of each candidate game object image, and the quality detection result can be used to evaluate the image quality of the target game object included in the candidate game object image. Please refer to. Figure 4 , which shows a method for detecting the quality of a game object image provided in an embodiment of the present application. Figure 4 The target game object is the alarm clock in the original image, and the image scene of the candidate game object image is "the target game object is placed together with the vase". For any candidate game object image among the determined multiple candidate game object images, determine the image pixel mean, image grayscale standard deviation, image average gradient value and image entropy value of the candidate game object image, and determine the first quality detection data based on these parameters; for the foreground image area (such as Figure 4 The area A enclosed by the dotted circle in the middle (which may be determined by the image detection method for the target game object or the image extraction method for the target game object) is subjected to feature extraction processing to obtain the first foreground image feature of the candidate game object image; the foreground image area (such as Figure 4The area B encircled by the dotted circle (which may be determined by the image detection method for the target game object or the image extraction method for the target game object) is subjected to feature extraction processing to obtain the second foreground image feature of the original image, and the image mean square error value and the image peak signal-to-noise ratio are determined according to the first foreground image feature of the candidate game object image and the second foreground image feature of the original image, and the second quality detection data is determined according to these two parameters; the quality detection result of the candidate game object image is determined according to the first quality detection data and the second quality detection data. The method provided in the embodiment of the present application can determine the first quality detection data from the image quality of the candidate game object image itself, can determine the second quality detection data according to the difference between the candidate game object image and the original image, and comprehensively determine the quality detection result of the candidate game object image according to the two quality detection data, so that the image quality of the target game object included in the candidate game object image can be accurately judged, and the quality of the target game object image determined from the candidate game object image can be effectively guaranteed to be good.

[0053] In one embodiment, the method for determining the foreground image area in the image can be a template matching method or an image segmentation method. The template matching method specifically allows the template image (i.e., the image of the target game object) to slide in the input image (i.e., the candidate game object image), traverses the entire image pixel by pixel for comparison, and finds the part that best matches the template image. This method can extract the foreground image area of ​​the image more accurately, but the complexity of this method is high and the image processing efficiency is low. The image segmentation method mainly includes a watershed image segmentation algorithm and a pyramid image segmentation algorithm, wherein the principle of the watershed image segmentation algorithm is to use the difference between the grayscale value of each pixel point in the grayscale image of the original image and the grayscale value of the surrounding pixels to determine the edge contour of the object in the image, thereby extracting the foreground image area in the image; while the pyramid image segmentation algorithm processes the image from the perspective of image resolution. Since the resolution of the foreground image area in the image is usually higher than that of other areas in the image, the foreground image area of ​​the image can be determined according to the image resolution of each part of the image. The image segmentation algorithm can determine the foreground image area in the image more quickly, but the accuracy is lower than the above-mentioned template matching method. The two image processing methods provided in the embodiments of the present application can both be used to determine the foreground image area (i.e. Figure 4 region A in the original image of the target game object) and the foreground image region in the original image of the target game object (i.e. Figure 4 The method provided in the embodiment of the present application can determine the foreground image area in the image, which is conducive to accurately determining the quality detection result of the candidate game object image.

[0054] In one embodiment, the implementation method of determining the quality detection results of each candidate game object image according to the first quality detection data and the second quality detection data of each candidate game object image can be: obtaining image quality weight data; performing weighted summation on the first quality detection data and the second quality detection data of each candidate game object image according to the image quality weight data to obtain the quality detection results of each candidate game object image; the implementation method of determining the target game object image from multiple candidate game object images according to the quality detection results can be: comparing and processing the quality detection results of each candidate game object image, and determining the candidate game object image with the highest image quality of the target game object included in the multiple candidate game object images as the target game object image. The image quality weight data can include the weight data of the first quality detection data and the weight data of the second quality detection data. The first quality detection data can include one or more of the image pixel mean, the image grayscale standard deviation value, the image average gradient value and the image entropy value, and the weight data corresponding to each parameter can be different; the second quality detection data can include one or two of the image mean square error value and the image peak signal-to-noise ratio, and the weight data corresponding to each parameter can be different. After determining the quality detection results of each candidate game object image, the quality detection results of each candidate game object image can be compared to obtain the candidate game object image with the best quality detection result, and the candidate game object image with the best quality detection result can be determined as the target game object image.

[0055] In one embodiment, when a target image generation model is used to generate candidate game object images based on input text, quality difference data can also be determined based on the quality detection results of each candidate game object image, and the quality difference data can be used to adjust the model parameters of the target image generation model, thereby further improving the prediction accuracy of the target image generation model and improving the image quality of the target game object in the candidate game object images generated by the target image generation model.

[0056] Based on the above embodiments, the beneficial effects of the present application are as follows: the image generation method provided in the embodiments of the present application can generate a game object image containing a specific game object based on an input text, and can generate a game object image containing features of different image scenes and different game objects based on an image feature description text in the input text, thereby achieving controllability of image scenes and game object features in images, greatly enriching the diversity of images associated with target game objects, and effectively improving the efficiency of generating images; quality detection can be performed on the obtained multiple candidate game object images, thereby achieving image screening, so that the image quality of the determined target game object image is higher, the details of the target game object included in the image are richer, and more in line with art requirements, and the target game object image is more instructive for the update of game objects.

[0057] See also Figure 5 , Figure 5 1 is a flow chart of a model training method for an image generation model provided by way of example in an embodiment of the present application. The model training method can be implemented by the above-mentioned server 102, or by other server devices. The image generation model obtained by using the model training method can be the target image generation model in the above-mentioned embodiment. The flow of the model training method provided in the embodiment of the present application includes but is not limited to:

[0058] S501. Perform image generation processing according to the game object style identifier and the game object category using the intermediate image generation model to obtain the predicted image, and determine the first difference data according to the original image and the predicted image.

[0059] In the embodiment of the present application, the intermediate image generation model is a model for generating images according to text, which is obtained by adjusting the model parameters according to the initial image generation model, and the intermediate image generation model may include a text feature extraction module and an image construction module. For a target game object, the game object style identifier and the game object category of the target game object can be obtained, the game object style identifier can indicate the theme style corresponding to the target game object, and can also indicate the target game object itself, and the game object category can indicate the category to which the target game object belongs. The original image of the target game object can be an image of different angles and different backgrounds of the target game object. The intermediate image generation model is used to perform image generation processing according to the game object style identifier and the game object category to obtain a predicted image. Since the game object style identifier and the game object category can be used to indicate the target game object, the predicted image generated by the intermediate image generation model according to the game object style identifier and the game object category is also associated with the target game object. The first difference data is determined according to the original image and the predicted image, and the first difference data can reflect the difference between the target game object in the predicted image and the target game object in the original image, and then the model parameter adjustment direction of the intermediate image generation model can be indicated.

[0060] It should be noted that, in actual application, the intermediate image generation model can be used to generate multiple predicted images, and the multiple predicted images and the multiple original images are used together to determine the first difference data. The embodiment of the present application takes one predicted image as an example for explanation. The original image of the target game object in the model training process may be different from the original image of the target game object in the embodiment corresponding to step S203 above, but both include the target game object.

[0061] In one embodiment, the implementation method of using the intermediate image generation model to perform image generation processing according to the game object style identifier and the game object category to obtain the predicted image can be: obtaining the game object style identifier, the game object category and the original image of the target game object; obtaining the game object quantity information of the target game object in the original image; generating a sample description text of the target game object according to the game object style identifier, the game object category and the game object quantity information; using the text feature extraction module of the intermediate image generation model to extract text features of the sample description text to obtain the text features of the sample description text; using the image construction module of the intermediate image generation model to perform image construction processing according to the text features of the sample description text to obtain the predicted image. The game object quantity information of the target game object in the original image is obtained, so that the intermediate image generation model can be trained using the game object quantity information later, so that the trained image generation model has better generalization ability. Generate a sample description text of the target game object according to the game object style identifier, the game object category and the game object quantity information. For example: if the game object style identifier of a certain target game object is M, the game object category is "animal", and the game object quantity information of the target game object in the original image is "one", then the sample description text "one M animal" of the target game object can be generated according to the above three parameters. The text feature extraction module of the intermediate image generation model is used to extract text features of the sample description text to obtain the text features of the sample description text; the image construction module of the intermediate image generation model is used to perform image construction processing based on the text features of the sample description text to obtain a predicted image.

[0062] In one embodiment, the method for obtaining the game object style identification of the target game object can be: obtaining multiple sample style participles and the quantity information of each sample style participle, the quantity information of the sample style participle is used to indicate the number of game objects associated with the sample style participle; determining multiple candidate style participles according to the quantity information of each sample style participle, the number indicated by the quantity information of the candidate style participle is less than or equal to the quantity threshold; determining the game object style identification of the target game object from the style identification corresponding to each candidate style participle. The game object style identification of the target game object is an exclusive identification of the target game object, which should be unique and iconic, and cannot be associated with other game objects. The game object style identifier is a combination of ordinary text or letters and numbers, which may be associated with other game objects when the text feature extraction module performs text feature extraction. Therefore, the method provided in the embodiment of the present application can be used: obtain sample style segmentation and the quantity information of each sample style segmentation from the segmentation list corresponding to the text feature extraction module, and the quantity information of the sample style segmentation is used to indicate the number of game objects associated with the sample style segmentation; determine multiple candidate style segmentations from multiple sample style segmentations according to the quantity information of each sample style segmentation, and the number indicated by the quantity information of the candidate style segmentation is less than or equal to the quantity threshold. Since the game object style identifier is uniquely identifiable, the game object associated with the segmentation corresponding to the game object style identifier should be less than 1, so the value range of the quantity threshold can be [0,1]. Performing inverse segmentation processing on the candidate style segmentation, the style identifier corresponding to each candidate style segmentation can be obtained, and the game object style identifier of the target game object can be determined from the style identifier corresponding to each candidate segmentation. The method provided in the embodiment of the present application can determine a game object style identifier with a unique identifier, so that the image generation model can distinguish the target game object from other game objects based on the game object style identifier, which is conducive to the model accurately generating a game object image containing the target game object based on the game object style identifier.

[0063] In one embodiment, an intermediate image generation model may be used to perform image generation processing according to a game object style identifier and a game object category to obtain a plurality of predicted images, and first difference data may be determined according to the difference between each predicted image and the original image.

[0064] In one embodiment, the initial image generation model can be a pre-trained DALLE.2 model. The DALLE.2 model is a hierarchical diffusion model that generates images based on text. It can generate original images based on descriptive text, and can also modify the image's features such as light and texture based on the text. Pre-training refers to using a large amount of training data to train the initially constructed model so that the model has good universality. After obtaining the pre-trained model, it is only necessary to fine-tune the model parameters of the pre-trained model using a small amount of training data for the current scene to obtain a model suitable for the current scene. This method can greatly improve the training efficiency of the model. The pre-trained DALLE.2 model obtained by training the game object image can be determined as the initial image generation model in the embodiment of the present application, and other text image generation models can also be determined as the initial image generation model in the embodiment of the present application.

[0065] S502: Utilize the intermediate image generation model to perform image generation processing according to the game object category to obtain the predicted category image.

[0066] In the embodiment of the present application, during the model training of the intermediate image generation model, if only the first difference data obtained in the above step S401 is used to adjust the model parameters, the image generation model will be overfitted or language drifted. Overfitting or language drift refers to the training process of the image generation model, in which the image generation model only learns the relevant image features of the target game object, but forgets the features of other game objects of the category to which the target game object belongs, thereby causing the image features of other game objects predicted by the image generation model to be the same as the image features of the target game object, and the model no longer has discrimination. For example: the target game object is a specific female game object, its game object style is identified as "W", and the game object category is "woman". Only using the above first difference data to train the intermediate image generation model will lead to overfitting of the model. Under normal circumstances, when the text "awoman" is input into the trained image generation model, the model will generate images containing a variety of female game objects, but when the text "awoman" is input into the overfitted image generation model, the model will only generate images containing the target game object (i.e., a specific female game object). Therefore, other difference data need to be introduced in the training process of the intermediate image generation model. The game object category is used to indicate the category to which the target game object belongs. The intermediate image generation model can be used to generate images according to the game object category to obtain a predicted category image. The predicted category image is an image including the game object generated by the intermediate image generation model according to the game object category in the current state. There can be multiple predicted category images, which can reflect to a certain extent whether the intermediate image generation model in the current state has overfitting or language drift.

[0067] S503: Acquire the initial category image, and determine the second difference data according to the initial category image and the predicted category image.

[0068] In an embodiment of the present application, an initial category image is obtained, and the initial category image is obtained by performing image generation processing on an initial image generation model according to the game object category of a target game object. The initial image generation model is a model that has not undergone model parameter adjustment, and there is no overfitting or language drift. The second difference data is determined based on the initial category image and the predicted category image, and the second difference data can reflect whether overfitting or language drift has occurred in the intermediate image generation model after the model parameters are adjusted. Using the second difference data to adjust the model parameters of the intermediate image generation model can effectively avoid overfitting or language drift in the image generation model, thereby ensuring the prediction accuracy of the image generation model.

[0069] In one embodiment, multiple initial category images can be obtained, and the initial category images can be any several of the multiple images generated by the initial image generation model according to the game object category; the intermediate image generation model can be used to perform image generation processing according to the game object category to obtain multiple predicted category images; based on the image feature differences between the multiple initial category images and the multiple predicted category images, the second difference data is determined.

[0070] S504: Determine the target difference data according to the first difference data and the second difference data, and adjust model parameters of the image construction module of the intermediate image generation model according to the target difference data to obtain the target image generation model.

[0071] In the embodiment of the present application, the intermediate image generation model includes a text feature extraction module and an image construction module, the text feature extraction module is mainly used to extract the text features of the input text, and the image construction module is mainly used to generate the corresponding image according to the extracted text features. In order to enable the model to generate an image of a specific game object, the target difference data can be determined according to the first difference data and the second difference data, and the model parameters of the image construction module of the intermediate image generation model can be adjusted according to the target difference data. The model parameter adjustment shown in the above steps can be performed on the intermediate image generation model multiple times. When the intermediate image generation model meets the preset conditions, the intermediate image generation model can be determined as the target image generation model. The method provided in the embodiment of the present application can not only enable the image generation model to learn the relationship between the specific game object and the game object style identifier, but also effectively avoid the overfitting of the image generation model and ensure the generalization of the image generation model; at the same time, the method provided in the embodiment of the present application only adjusts the parameters of the image construction module, which can effectively improve the training efficiency of the model while ensuring the accuracy of model prediction.

[0072] In one embodiment, the intermediate image generation model includes a text feature extraction module and an image construction module, and the image construction module includes a super-resolution unit. The image construction module of the intermediate image generation model is adjusted according to the target difference data to obtain the target image generation model. The implementation method can be: using the target difference data to adjust the model parameters of the image construction module of the intermediate image generation model to obtain the intermediate image generation model after preliminary adjustment; adjusting the parameters of the super-resolution unit of the intermediate image generation model after preliminary adjustment according to the original image to obtain the target image generation model. The super-resolution unit in the image construction module can be used to process the low-resolution image generated by the model to obtain a high-resolution image, so that the resolution of the image finally generated by the image generation model is higher and the details of the game object in the generated image are better. The image construction module of the intermediate image generation model of the target difference data object is used to adjust the model parameters to obtain the intermediate image generation model after preliminary adjustment; adjusting the parameters of the super-resolution unit of the intermediate image generation model after preliminary adjustment according to the original image of the target game object to obtain the target image generation model. The method provided in the embodiment of the present application can effectively ensure the resolution of the image finally generated by the image generation model, so that the details of the game object in the generated image are clearer and more in line with the game object in the original image.

[0073] See also Figure 6 , which shows a model training method provided in an embodiment of the present application. Figure 6In the example, the image generation model includes a text feature extraction module and an image construction module. The game object category of the target game object is determined (assuming it is a "female game character"), and the initial image generation model is used to perform image generation processing according to the game object category of the target game object to obtain multiple initial category images. The original image of the target game object is obtained, and the game object quantity information of the target game object in the original image is determined (assuming it is "one"), and the sample description text (assuming it is "a female game character of Y") is determined according to the game object style identification (assuming it is "Y"), the game object category and the game object quantity information of the target game object; during the first training, the initial image generation model is used to generate multiple predicted images according to the sample description text, and the first difference data is determined according to the predicted image and the original image; the image construction model of the initial image generation model is adjusted using the first difference data to obtain an intermediate image generation model. During the Nth training (N is a positive integer greater than 1), the intermediate image generation model is used to generate multiple predicted images according to the sample description text, and the first difference data is determined according to the predicted image and the original image; the intermediate image generation model is used to generate multiple predicted category images according to the game object category of the target game object, and the second difference data is determined according to the predicted category image and the initial category image; the target difference data is determined according to the first difference data and the second difference data, and the target difference data is used to adjust the model parameters of the image generation module of the intermediate image generation model. When the intermediate image generation model meets the preset conditions, the original image of the target game object can be used to fine-tune the super-resolution unit of the intermediate image generation model to obtain the target image generation model. The model training method provided in the embodiment of the present application can enable the image generation model to learn the image features of a specific game object, and the model will not have overfitting or language drift; only the image construction module of the image generation model is adjusted, which can effectively improve the training efficiency of the model; fine-tuning the super-resolution unit of the image construction module can effectively reduce the generation of artifacts in the generated image, so that the texture details of the generated image are richer and the clarity of the generated image is higher.

[0074] In one embodiment, during the training of the intermediate image generation model, a checkpoint can be added to save the parameters of the model so that after the model is used for reasoning or interrupted, the saved model parameters can be used to train the model again. For example: during the model training process, the model parameters saved at the checkpoint can be used to verify the prediction accuracy of the model in real time, and the model parameters with the highest prediction accuracy can be saved as the final model parameters. The checkpoint can be set according to the number of gradient descents corresponding to the model parameters (or the number of model parameter adjustments). For example: the checkpoint saving frequency can be set to 500, that is, the model parameters in the current image generation model are saved every 500 times the gradient corresponding to the model parameters decreases. After determining the checkpoint saving frequency during model training, the earlystop method is also used to determine whether the intermediate image generation model meets the preset conditions. Earlystop means that when training the model, it is not necessary to stop the training until the number of model training times is saturated on the training set, but the model training is stopped when the model achieves the best effect on the test set. Because if the training is stopped only when the number of model training times on the training set is saturated, the model may be overfitted, resulting in a good prediction effect of the model on the training set, but a poor prediction effect on the test set. The method provided in the embodiment of the present application can quickly and efficiently complete the training process of the image generation model, effectively avoiding the overfitting of the image generation model.

[0075] In one embodiment, the above model training process can be implemented using a model training platform. Figure 7 , which shows another model training method provided by the embodiment of the present application. Figure 7 As shown, the model training process can be to use the initial image generation model to perform image generation processing based on the input text to obtain an initial predicted image, compare the initial predicted image with the original image of the target game object, and determine the image prediction effect of the initial image generation model on the target game object at this time. Obtain multiple original images of the target game object from different angles, different backgrounds, and different actions; obtain the game object style identification of the target game object; use the method described in steps S501-S504 to train the initial image generation model to obtain a target image generation model. The image prediction effect of the target image generation model on the target game object is better than that of the initial image generation model on the target game object. Please refer to Figure 8 , Figure 8 It is a schematic diagram of a model training interface provided in an embodiment of the present application. Figure 8In the "Model Prediction" section, you can use the trained model to make predictions based on the input text, and in the "Model Training" section, you can build a model and train the built model. The "Model Training" section mainly includes 5 functional modules, namely "Model Creation", "Load Parameters", "Generate Detection Point File", "Cancel" and "Training". The "Model Creation" section includes two steps: "Create Model" and "Train Model". In the "Create Model" step, you can set the name of the model to be created ( Figure 8 The model name is "Model1"), and the "Base Model" is used to indicate the structure of the model to be created ( Figure 8 In the example above, the model training platform can determine the corresponding pre-trained model (i.e., the initial image generation model in the above embodiment) according to the input value of the "basic model". The setting of the "basic model" can change according to the different target game objects. For example, when the target game object is an "anime character", you can choose to use the model pre-trained with data related to the "anime character". When the target game object is an "animal", you can choose to use the model pre-trained with data related to the "animal". The "sampler" is used for the denoising process of the diffusion model and is a relatively important part of model training. The more sampling steps, the better the image quality. Figure 8 The "sampler" selected in the interface is "pndm", which is a pseudo-numerical method for diffusion models (Pseudo Numerical Methods for Diffusion Models on Manifolds), a commonly used diffusion model sampling method. The lower right part of the interface is used to obtain the original image of the uploaded target game object ( Figure 8 The prompt message is "Please upload the original image"). Click the "Create" button to create the initial image generation model, and then adjust the model parameters of the created initial image generation model in the "Train Model" step to obtain the target image generation model. The method provided in the embodiment of the present application can effectively simplify the steps of building and training the image generation model and improve the training efficiency of the model.

[0076] Based on the above embodiments, the beneficial effects of the present application are as follows: the image generation method provided in the embodiments of the present application can utilize the image generation model to generate a game object image containing a specific game object according to the input text, and can make the game object image contain different image scenes and different angles, so that the game object in the image presents different postures, thereby realizing the controllability of the image scene and the game object features in the image, greatly enriching the diversity of images associated with the target game object, ensuring the clarity and accuracy of the generated image, and effectively improving the efficiency of generating images; in the model training process, the method provided in the embodiments of the present application can avoid overfitting or language drift when the model learns the image features of a specific game object, thereby ensuring the prediction accuracy and generalization of the image generation model; when adjusting the model parameters, only adjusting the parameters of the image construction module in the model can effectively improve the training efficiency of the model, so that the trained image generation model can predict a game object image with higher clarity, finer texture, and more suitable for a specific game object.

[0077] See also Fig. 9 , Fig. 9 This is a structural block diagram of an image generation device provided in an embodiment of the present application. The device includes:

[0078] An acquisition unit 901 is used to acquire input text, wherein the input text includes a game object description text and an image feature description text, wherein the game object description text includes a game object style identifier and a game object category, and the game object indicated by the game object style identifier and the game object category is a target game object;

[0079] A processing unit 902 is configured to perform image generation processing according to the input text to obtain a plurality of candidate game object images, wherein the candidate game object images include the target game object, and image features displayed by the candidate game object images match the image features described by the image feature description text, and the image features described by the image feature description text include one or two of the following: image scenes, and features of game objects in the images;

[0080] The detection unit 903 is used to perform quality detection on the multiple candidate game object images, and determine a target game object image from the multiple candidate game object images based on the quality detection result, wherein the target game object image is the game object image with the highest image quality of the target game object included in the multiple candidate game object images.

[0081] In one embodiment, when the processing unit 902 performs image generation processing according to the input text to obtain multiple candidate game object images, it is specifically used to: perform image generation processing according to the input text using a target image generation model to obtain multiple candidate game object images; wherein the target image generation model is obtained by adjusting some model parameters of the intermediate image generation model according to target difference data, and the target difference data is determined according to first difference data and second difference data; the first difference data is determined according to the original image and predicted image of the target game object, and the predicted image is determined by performing image generation processing according to the game object style identifier and game object category associated with the target game object using the intermediate image generation model; the second difference data is determined according to an initial category image and a predicted category image, and the initial category image is determined by performing image generation processing according to the game object category using the initial image generation model, and the predicted category image is determined by performing image generation processing according to the game object category using the intermediate image generation model, and the intermediate image generation model is determined according to the initial image generation model; the game object style displayed by the candidate game object image matches the game object style displayed by the original image.

[0082] In one embodiment, when the detection unit 903 performs quality detection on the multiple candidate game object images, it is specifically used to: determine the first quality detection data of each of the multiple candidate game object images, the first quality detection data including one or more of the image pixel mean, the image grayscale standard deviation value, the image average gradient value and the image entropy value; perform feature extraction processing on the foreground image area in each of the candidate game object images to obtain the first foreground image feature of each of the candidate game object images, the foreground image area being the image area where the target game object in the candidate game object image is located; extract the original image of the target game object A feature extraction process is performed on the foreground image area in the image to obtain the second foreground image features of the original image; second quality detection data of each of the candidate game object images is determined according to the first foreground image features of each of the candidate game object images and the second foreground image features of the original image, the second quality detection data including one or both of the image mean square error value and the image peak signal-to-noise ratio; a quality detection result of each of the candidate game object images is determined according to the first quality detection data and the second quality detection data of each of the candidate game object images, the quality detection result being used to evaluate the image quality of the target game object included in the candidate game object image.

[0083] In one embodiment, when the detection unit 903 determines the quality detection result of each of the candidate game object images based on the first quality detection data and the second quality detection data of each of the candidate game object images, it is specifically used to: obtain image quality weight data; and perform weighted summation of the first quality detection data and the second quality detection data of each of the candidate game object images according to the image quality weight data to obtain the quality detection result of each of the candidate game object images.

[0084] In one embodiment, the intermediate image generation model includes a text feature extraction module and an image construction module, and the processing unit 902 is further used to: use the intermediate image generation model to perform image generation processing according to the game object style identifier and the game object category to obtain the predicted image, and determine the first difference data according to the original image and the predicted image; use the intermediate image generation model to perform image generation processing according to the game object category to obtain the predicted category image; obtain the initial category image, and determine the second difference data according to the initial category image and the predicted category image; determine the target difference data according to the first difference data and the second difference data, and adjust the model parameters of the image construction module of the intermediate image generation model according to the target difference data to obtain the target image generation model.

[0085] In one embodiment, when the processing unit 902 uses the intermediate image generation model to perform image generation processing according to the game object style identifier and the game object category to obtain the predicted image, it is specifically used to: obtain the game object style identifier, game object category and original image of the target game object; obtain the game object quantity information of the target game object in the original image; generate a sample description text of the target game object according to the game object style identifier, the game object category and the game object quantity information; use the text feature extraction module of the intermediate image generation model to extract text features from the sample description text to obtain text features of the sample description text; use the image construction module of the intermediate image generation model to perform image construction processing according to the text features of the sample description text to obtain the predicted image.

[0086] In one embodiment, when the processing unit 902 obtains the game object style identifier of the target game object, it is specifically used to: obtain multiple sample style participles and quantity information of each of the sample style participles, the quantity information of the sample style participles is used to indicate the number of game objects associated with the sample style participles; determine multiple candidate style participles based on the quantity information of each sample style participle, the number indicated by the quantity information of the candidate style participles is less than or equal to the quantity threshold; determine the game object style identifier of the target game object from the style identifiers corresponding to each of the candidate style participles.

[0087] In one embodiment, the image construction module includes a super-resolution unit, and when the processing unit 902 adjusts the model parameters of the image construction module of the intermediate image generation model according to the target difference data to obtain the target image generation model, it is specifically used to: adjust the model parameters of the image construction module of the intermediate image generation model using the target difference data to obtain a preliminarily adjusted intermediate image generation model; and adjust the parameters of the super-resolution unit of the preliminarily adjusted intermediate image generation model according to the original image to obtain the target image generation model.

[0088] It can be understood that the functions of each functional unit of the image generating device of the embodiment of the present application can be specifically implemented according to the image generating method in the above method embodiment, and its specific implementation process can refer to the relevant description in the above image generating method embodiment, which will not be repeated here.

[0089] Based on the above embodiments, the beneficial effects of the present application are as follows: the image generation method provided in the embodiments of the present application can utilize the image generation model to generate a game object image containing a specific game object according to the input text, and can make the game object image contain different image scenes and different angles, so that the game object in the image presents different postures, thereby realizing the controllability of the image scene and the game object features in the image, greatly enriching the diversity of images associated with the target game object, ensuring the clarity and accuracy of the generated image, and effectively improving the efficiency of generating images; in the model training process, the method provided in the embodiments of the present application can avoid overfitting or language drift when the model learns the image features of a specific game object, thereby ensuring the prediction accuracy and generalization of the image generation model; when adjusting the model parameters, only adjusting the parameters of the image construction module in the model can effectively improve the training efficiency of the model, so that the trained image generation model can predict a game object image with higher clarity, finer texture, and more suitable for a specific game object.

[0090] See also Fig.10 , Fig.10The block diagram of a computer device provided in the embodiment of the present application. The computer device described in the embodiment of the present application includes: a processor 1001 and a memory 1002. The processor 1001 and the memory 1002 may be connected via a bus or other means, and the embodiment of the present application takes the connection via a bus as an example.

[0091] Among them, the processor 1001 (or CPU (Central Processing Unit)) is the computing core and control core of the computer device, which can parse various instructions in the computer device and process various data of the computer device. For example, the CPU can be used to parse the power on and off instructions sent by the user to the computer device, and control the computer device to perform power on and off operations; another example: the CPU can transmit various interactive data between the internal structures of the computer device, and so on. The memory 1002 (Memory) is a memory device in the computer device, which is used to store programs and data. It can be understood that the memory 1002 here can include the built-in memory of the computer device, and of course, it can also include the extended memory supported by the computer device. The memory 1002 provides a storage space, which stores the operating system of the computer device, which may include but is not limited to: Android system, iOS system, Windows Phone system, etc., and this application does not limit this.

[0092] In the embodiment of the present application, the processor 1001 performs the following operations by running the executable program code in the memory 1002:

[0093] Acquire input text, the input text includes game object description text and image feature description text, the game object description text includes a game object style identifier and a game object category, and the game object indicated by the game object style identifier and the game object category is a target game object;

[0094] Performing image generation processing according to the input text to obtain a plurality of candidate game object images, wherein the candidate game object images include the target game object, and image features displayed by the candidate game object images match the image features described by the image feature description text, and the image features described by the image feature description text include one or two of the following: image scene, and features of the game object in the image;

[0095] Quality detection is performed on the multiple candidate game object images, and a target game object image is determined from the multiple candidate game object images based on the quality detection result, wherein the target game object image is the game object image with the highest image quality among the multiple candidate game object images.

[0096] In one embodiment, when the processor 1001 performs image generation processing according to the input text to obtain multiple candidate game object images, it is specifically used to: perform image generation processing according to the input text using a target image generation model to obtain multiple candidate game object images; wherein the target image generation model is obtained by adjusting some model parameters of the intermediate image generation model according to target difference data, and the target difference data is determined according to first difference data and second difference data; the first difference data is determined according to the original image and predicted image of the target game object, and the predicted image is determined by performing image generation processing according to the game object style identifier and game object category associated with the target game object using the intermediate image generation model; the second difference data is determined according to an initial category image and a predicted category image, and the initial category image is determined by performing image generation processing according to the game object category using the initial image generation model, and the predicted category image is determined by performing image generation processing according to the game object category using the intermediate image generation model, and the intermediate image generation model is determined according to the initial image generation model; the game object style displayed by the candidate game object image matches the game object style displayed by the original image.

[0097] In one embodiment, when the processor 1001 performs quality detection on the multiple candidate game object images, it is specifically used to: determine the first quality detection data of each of the multiple candidate game object images, the first quality detection data including one or more of the image pixel mean, the image grayscale standard deviation value, the image average gradient value and the image entropy value; perform feature extraction processing on the foreground image area in each of the candidate game object images to obtain the first foreground image feature of each of the candidate game object images, the foreground image area being the image area where the target game object in the candidate game object image is located; extract the original image of the target game object A feature extraction process is performed on the foreground image area in the image to obtain the second foreground image features of the original image; second quality detection data of each of the candidate game object images is determined according to the first foreground image features of each of the candidate game object images and the second foreground image features of the original image, the second quality detection data including one or both of the image mean square error value and the image peak signal-to-noise ratio; a quality detection result of each of the candidate game object images is determined according to the first quality detection data and the second quality detection data of each of the candidate game object images, the quality detection result being used to evaluate the image quality of the target game object included in the candidate game object image.

[0098] In one embodiment, when the processor 1001 determines the quality detection result of each of the candidate game object images based on the first quality detection data and the second quality detection data of each of the candidate game object images, it is specifically used to: obtain image quality weight data; and perform weighted summation of the first quality detection data and the second quality detection data of each of the candidate game object images according to the image quality weight data to obtain the quality detection result of each of the candidate game object images.

[0099] In one embodiment, the intermediate image generation model includes a text feature extraction module and an image construction module, and the processor 1001 is further used to: use the intermediate image generation model to perform image generation processing according to the game object style identifier and the game object category to obtain the predicted image, and determine the first difference data according to the original image and the predicted image; use the intermediate image generation model to perform image generation processing according to the game object category to obtain the predicted category image; obtain the initial category image, and determine the second difference data according to the initial category image and the predicted category image; determine the target difference data according to the first difference data and the second difference data, and adjust the model parameters of the image construction module of the intermediate image generation model according to the target difference data to obtain the target image generation model.

[0100] In one embodiment, when the processor 1001 uses the intermediate image generation model to perform image generation processing according to the game object style identifier and the game object category to obtain the predicted image, it is specifically used to: obtain the game object style identifier, game object category and original image of the target game object; obtain the game object quantity information of the target game object in the original image; generate a sample description text of the target game object according to the game object style identifier, the game object category and the game object quantity information; use the text feature extraction module of the intermediate image generation model to extract text features from the sample description text to obtain text features of the sample description text; use the image construction module of the intermediate image generation model to perform image construction processing according to the text features of the sample description text to obtain the predicted image.

[0101] In one embodiment, when the processor 1001 obtains the game object style identifier of the target game object, it is specifically used to: obtain multiple sample style participles and quantity information of each of the sample style participles, the quantity information of the sample style participles is used to indicate the number of game objects associated with the sample style participles; determine multiple candidate style participles based on the quantity information of each sample style participle, the number indicated by the quantity information of the candidate style participles is less than or equal to the quantity threshold; determine the game object style identifier of the target game object from the style identifiers corresponding to each of the candidate style participles.

[0102] In one embodiment, the image construction module includes a super-resolution unit. When the processor 1001 adjusts the model parameters of the image construction module of the intermediate image generation model according to the target difference data to obtain the target image generation model, it is specifically used to: adjust the model parameters of the image construction module of the intermediate image generation model using the target difference data to obtain the intermediate image generation model after preliminary adjustment; and adjust the parameters of the super-resolution unit of the intermediate image generation model after preliminary adjustment according to the original image to obtain the target image generation model.

[0103] In a specific implementation, the processor 1001 and the memory 1002 described in the embodiment of the present application can execute the implementation method of the server described in an image generation method provided in an embodiment of the present application, and can also execute the implementation method described in an image generation device provided in an embodiment of the present application, which will not be repeated here.

[0104] Based on the above embodiments, the beneficial effects of the present application are as follows: the image generation method provided in the embodiments of the present application can utilize the image generation model to generate a game object image containing a specific game object according to the input text, and can make the game object image contain different image scenes and different angles, so that the game object in the image presents different postures, thereby realizing the controllability of the image scene and the game object features in the image, greatly enriching the diversity of images associated with the target game object, ensuring the clarity and accuracy of the generated image, and effectively improving the efficiency of generating images; in the model training process, the method provided in the embodiments of the present application can avoid overfitting or language drift when the model learns the image features of a specific game object, thereby ensuring the prediction accuracy and generalization of the image generation model; when adjusting the model parameters, only adjusting the parameters of the image construction module in the model can effectively improve the training efficiency of the model, so that the trained image generation model can predict a game object image with higher clarity, finer texture, and more suitable for a specific game object.

[0105] The present application also provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the image generation method as described in the present application. The specific implementation method can be referred to the above description, and will not be repeated here.

[0106] The embodiment of the present application also provides a computer program product, which includes a computer program or a computer instruction, and the computer program or the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer program or the computer instruction from the computer-readable storage medium, and the processor executes the computer program or the computer instruction, so that the computer device performs the image generation method as described in the embodiment of the present application. The specific implementation method can be referred to the above description, which will not be repeated here.

[0107] It should be noted that, for the above-mentioned various method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0108] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0109] The above disclosure is only part of the embodiments of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An image generation method, characterized in that: The method comprises: Acquire input text, the input text includes game object description text and image feature description text, the game object description text includes a game object style identifier and a game object category, and the game object indicated by the game object style identifier and the game object category is a target game object; Performing image generation processing according to the input text to obtain a plurality of candidate game object images, wherein the candidate game object images include the target game object, and image features displayed by the candidate game object images match the image features described by the image feature description text, and the image features described by the image feature description text include one or two of the following: image scene, and features of the game object in the image; Quality detection is performed on the multiple candidate game object images, and a target game object image is determined from the multiple candidate game object images based on the quality detection result, wherein the target game object image is the game object image with the highest image quality among the multiple candidate game object images.

2. The method according to claim 1, characterized in that The image generation process is performed according to the input text to obtain a plurality of candidate game object images, including: Using a target image generation model to perform image generation processing according to the input text, to obtain a plurality of candidate game object images; Among them, the target image generation model is obtained by adjusting some model parameters of the intermediate image generation model according to the target difference data, and the target difference data is determined according to the first difference data and the second difference data; the first difference data is determined according to the original image and the predicted image of the target game object, and the predicted image is determined by using the intermediate image generation model to perform image generation processing according to the game object style identifier and the game object category associated with the target game object; the second difference data is determined according to the initial category image and the predicted category image, and the initial category image is determined by using the initial image generation model to perform image generation processing according to the game object category, and the predicted category image is determined by using the intermediate image generation model to perform image generation processing according to the game object category, and the intermediate image generation model is determined according to the initial image generation model; the game object style displayed by the candidate game object image matches the game object style displayed by the original image.

3. The method according to claim 1 or 2, characterized in that: The performing quality detection on the plurality of candidate game object images comprises: Determine first quality detection data of each of the plurality of candidate game object images, wherein the first quality detection data includes one or more of an image pixel mean, an image grayscale standard deviation, an image average gradient value, and an image entropy value; Performing feature extraction processing on the foreground image area in each of the candidate game object images to obtain a first foreground image feature of each of the candidate game object images, wherein the foreground image area is an image area where the target game object in the candidate game object image is located; Performing feature extraction processing on a foreground image region in an original image of the target game object to obtain a second foreground image feature of the original image; Determine second quality detection data of each of the candidate game object images according to the first foreground image feature of each of the candidate game object images and the second foreground image feature of the original image, wherein the second quality detection data includes one or both of an image mean square error value and an image peak signal-to-noise ratio; According to the first quality detection data and the second quality detection data of each of the candidate game object images, a quality detection result of each of the candidate game object images is determined, and the quality detection result is used to evaluate the image quality of the target game object included in the candidate game object image.

4. The method according to claim 3, characterized in that Determining the quality detection result of each of the candidate game object images according to the first quality detection data and the second quality detection data of each of the candidate game object images includes: Obtain image quality weight data; The first quality detection data and the second quality detection data of each of the candidate game object images are weightedly summed according to the image quality weight data to obtain the quality detection results of each of the candidate game object images.

5. The method according to claim 2, characterized in that: The intermediate image generation model includes a text feature extraction module and an image construction module, and the method further includes: Using the intermediate image generation model to perform image generation processing according to the game object style identifier and the game object category to obtain the predicted image, and determining the first difference data according to the original image and the predicted image; Using the intermediate image generation model to perform image generation processing according to the game object category to obtain the predicted category image; Acquire the initial category image, and determine the second difference data according to the initial category image and the predicted category image; The target difference data is determined according to the first difference data and the second difference data, and model parameters of the image construction module of the intermediate image generation model are adjusted according to the target difference data to obtain the target image generation model.

6. The method according to claim 5, characterized in that The step of using the intermediate image generation model to perform image generation processing according to the game object style identifier and the game object category to obtain the predicted image includes: Obtain the game object style identifier, game object category and original image of the target game object; Obtaining game object quantity information of the target game object in the original image; Generate a sample description text of the target game object according to the game object style identifier, the game object category and the game object quantity information; Using the text feature extraction module of the intermediate image generation model to extract text features from the sample description text, to obtain text features of the sample description text; The image construction module of the intermediate image generation model is used to perform image construction processing according to the text features of the sample description text to obtain the predicted image.

7. The method according to claim 6, characterized in that The step of obtaining the game object style identifier of the target game object includes: Acquire multiple sample style segmentations and quantity information of each of the sample style segmentations, where the quantity information of the sample style segmentations is used to indicate the number of game objects associated with the sample style segmentations; Determine a plurality of candidate style participles according to the quantity information of each sample style participle, wherein the quantity indicated by the quantity information of the candidate style participles is less than or equal to a quantity threshold; The game object style identifier of the target game object is determined from the style identifiers corresponding to the candidate style segmentations.

8. The method according to any one of claims 5 to 7, characterized in that: The image construction module includes a super-resolution unit, and the image construction module of the intermediate image generation model is adjusted according to the target difference data to obtain the target image generation model, including: Using the target difference data, adjusting model parameters of an image construction module of the intermediate image generation model to obtain a preliminarily adjusted intermediate image generation model; The super-resolution unit of the initially adjusted intermediate image generation model is parameter-adjusted according to the original image to obtain the target image generation model.

9. An image generating device, characterized in that: The device comprises: An acquisition unit, used to acquire input text, the input text includes game object description text and image feature description text, the game object description text includes a game object style identifier and a game object category, and the game object indicated by the game object style identifier and the game object category is a target game object; A processing unit, configured to perform image generation processing according to the input text to obtain a plurality of candidate game object images, wherein the candidate game object images include the target game object, and image features displayed by the candidate game object images match the image features described by the image feature description text, and the image features described by the image feature description text include one or two of the following: image scene, and features of the game object in the image; A detection unit is used to perform quality detection on the multiple candidate game object images, and determine a target game object image from the multiple candidate game object images based on the quality detection result, wherein the target game object image is the game object image with the highest image quality among the multiple candidate game object images.

10. A computer device, characterized in that: include: A processor and a memory, wherein the processor and the memory are connected to each other, wherein the memory stores an executable program code, and the processor is used to call the executable program code to implement the image generation method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which, when executed on a computer, enable the computer to implement the image generating method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The computer program product comprises a computer program or a computer instruction, and when the computer program or the computer instruction is executed by a processor, the image generation method according to any one of claims 1 to 8 is implemented.

Images