Image generation method and device, electronic equipment, storage medium and product

Abstract

The invention provides an image generation method and device, electronic equipment, a computer readable storage medium and a computer program product, and the method comprises the steps: displaying a target text which comprises a text corresponding to at least one object; in response to an image generation instruction for the target text, displaying object images corresponding to the generated objects; based on the generated object images, a generated target image associated with the target text is determined to be displayed, and the target image comprises at least one object; in response to an issuing instruction for the target image, issuing the target image, and displaying image indication information in an associated area of the target text, the image indication information being used for indicating that an image associated with the target text exists; and when a trigger operation for the image indication information is received, displaying a target image associated with the target text. According to the invention, the efficiency and convenience of the image generation process can be improved.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to an image generation method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] In the related art, when users read novels, they have no visual concept of certain objects mentioned in the novels, and it is difficult to associate pictures in their minds. The reading apps in the related art only present the illustrations that come with the books. If users want to generate illustrations for a certain paragraph, they need to go to related products, and then based on such products, they need to enter a text prompt to generate the corresponding picture. The form is relatively simple and limited, and the process is also relatively complicated, resulting in low efficiency and convenience in the image generation process. Summary of the invention

[0003] The embodiments of the present application provide an image generation method, device, electronic device, computer-readable storage medium, and computer program product, which can improve the efficiency and convenience of the image generation process.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides an image generation method, including:

[0006] Displaying a target text, wherein the target text includes text corresponding to at least one object;

[0007] In response to the image generation instruction for the target text, displaying the generated object image corresponding to each of the objects;

[0008] Determining a target image associated with the target text based on the generated object images, wherein the target image includes at least one of the objects;

[0009] In response to a publishing instruction for the target image, the target image is published, and image indication information is displayed in an associated area of ​​the target text, wherein the image indication information is used to indicate that there is an image associated with the target text;

[0010] When a trigger operation for the image indication information is received, the target image associated with the target text is displayed.

[0011] An embodiment of the present application provides an image generating device, including:

[0012] A first display module, used to display a target text, wherein the target text includes a text corresponding to at least one object;

[0013] A second display module, configured to display the generated object image corresponding to each of the objects in response to the image generation instruction for the target text;

[0014] A determination module, configured to determine a target image associated with the target text based on the generated object images, wherein the target image includes at least one of the objects;

[0015] a publishing module, configured to publish the target image in response to a publishing instruction for the target image, and display image indication information in an associated area of ​​the target text, wherein the image indication information is used to indicate that there is an image associated with the target text;

[0016] The third display module is used to display the target image associated with the target text when a trigger operation for the image indication information is received.

[0017] In the above scheme, the device also includes a fourth display module, which is used to display a content editing area; the determination module is also used to display at least one selected target object image in the content editing area in response to a selection operation on at least one of the generated object images; based on at least one of the target object images displayed in the content editing area, in response to an image generation instruction, a generated target image associated with the target text is displayed, and the target image is generated based on the target object image in the content editing area.

[0018] In the above scheme, each object image is associated with an add control; the selection operation is a trigger operation for the add control associated with the object image, and the determination module is also used to display at least one added target object image in the content editing area in response to the trigger operation performed on the add control associated with at least one object image.

[0019] In the above scheme, the device also includes a first switching module, which is used to switch the add control associated with the target object image to the add completion control; wherein the add completion control is used to indicate that the target object image has been added to the content editing area; when a trigger operation for the add completion control is received, in the content editing area, the target object image is canceled from being displayed, and the add completion control is switched to the add control.

[0020] In the above scheme, the device also includes a fifth display module, which is used to display a content editing area; the determination module is also used to display at least one selected target object image in the content editing area in response to a selection operation on at least one of the generated object images; in response to an editing operation on the target object image triggered in the content editing area, display the edited target editing content; in response to an image generation instruction for the target editing content, display the generated target image associated with the target text, wherein the target image is generated based on the target editing content.

[0021] In the above scheme, the determination module is also used to display at least one operation control in response to a trigger operation on the target object image, and the operation control is used to perform an editing operation on the target object image, and different operation controls indicate different editing operations; in response to a trigger operation on a target operation control in the at least one operation control, the target editing operation indicated by the target operation control is performed on the target object image; when the target editing operation on the target object image is completed, the target editing content obtained by editing is displayed.

[0022] In the above scheme, the device also includes a sixth display module, which is used to display at least one drawing control, and different drawing controls correspond to different drawing tools; the determination module is also used to control the content editing area to be in a drawing mode in response to a triggering operation on a target drawing control in the at least one drawing control; in the drawing mode, in response to a drawing operation triggered by the drawing tool corresponding to the target drawing control, the target editing content obtained by drawing and including the target object image is displayed.

[0023] In the above scheme, the device also includes a seventh display module, which is used to display at least one image style control, and the image style control is used to indicate the image style of the generated target image; in response to the selection operation of the target image style control in the at least one image style control, the target image style control is controlled to be in a selected state; the determination module is also used to display the target image with the target image style in response to the image generation instruction for each of the generated object images when the target image type control is in the selected state; wherein the target image style is the image style indicated by the target image style control.

[0024] In the above scheme, the device also includes an eighth display module, and the eighth display module is used to display an image style editing box for setting the image style of the target image in response to a display instruction of the image style editing box; in the image style editing box, the input text is displayed, and the text is used to indicate the target image style that the target image is expected to have; the determination module is also used to generate the target image with the target image style based on the generated object images in response to a determination instruction for the text entered in the image style editing box.

[0025] In the above scheme, the determination module is also used to display waiting prompt information in response to the image generation instruction for each generated object image; wherein the waiting prompt information is used to indicate that the target image is being generated; when it is determined that the generation of the target image is completed based on the waiting prompt information, the generated target image associated with the target text is displayed.

[0026] In the above scheme, the device also includes a ninth display module, and the ninth display module is used to display an information introduction page of the object corresponding to the target object image in response to a trigger operation for the target object image in each of the generated object images; wherein the information introduction page includes introduction information, and the introduction information is used to introduce the target object corresponding to the target object image.

[0027] In the above scheme, the determination module is also used to display at least two candidate images associated with the target text in response to the image generation instructions for each of the generated object images; in response to the selection operation of the target candidate image among the at least two candidate images, determine the selected target candidate image as the target image.

[0028] In the above scheme, the device also includes a tenth display module, which is used to display a refresh control for the target image; in response to a trigger operation for the refresh control, the target image is switched to a new target image associated with the target text; wherein the new target image is different from the target image, and the new target image includes at least one of the objects.

[0029] In the above scheme, the target text belongs to a target work, and the device also includes an eleventh display module, which is used to display the text content included in the target work; in response to a selection operation on text in the text content, the selected text is displayed using a target style; wherein the target style is used to distinguish between the selected text and the unselected text in the text content; and the selected text is determined as the target text.

[0030] In the above solution, the device further includes a twelfth display module, and the twelfth display module is used to display an image generation control; in response to a trigger operation on the image generation control, the image generation instruction is received.

[0031] In the above solution, the device further includes a thirteenth display module, and the thirteenth display module is used to display a publishing control; in response to a trigger operation on the publishing control, the publishing instruction is received.

[0032] In the above scheme, the image indication information is character information, and the publishing module is also used to display the character information below the target text; the third display module is also used to display the target image associated with the target text when a trigger operation for the character information is received.

[0033] In the above scheme, the device also includes a fourteenth display module, which is used to display at least one interactive control for the target image; wherein the interactive control is used to indicate the execution of an interactive operation on the target image, and different interactive controls indicate different interactive operations; in response to a trigger operation on a target interactive control among the at least one interactive control, the target interactive operation indicated by the target interactive control is executed for the target image.

[0034] In the above scheme, the second display module is also used to respond to the image generation instruction for the target text, and display the object image corresponding to each of the objects in the image editing interface; the device also includes a fifteenth display module, and the fifteenth display module is used to display the text corresponding to each of the objects in the target text in the image editing interface using a first display style, and use a second display style to display the text in the target text except the text corresponding to each of the objects; wherein the first display style is different from the second display style.

[0035] An embodiment of the present application provides an electronic device, including:

[0036] A memory for storing executable instructions;

[0037] The processor is used to implement the image generation method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0038] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute and implement the image generation method provided in the embodiment of the present application.

[0039] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the image generation method provided in the embodiment of the present application.

[0040] The embodiments of the present application have the following beneficial effects:

[0041] First, a target text including at least one object is displayed, and then in response to an image generation instruction for the target text, a target image associated with the target text is determined based on the object images corresponding to the displayed objects, and then the target image is released, and image indication information is displayed, so that when a trigger operation for the image indication information is received, the target image associated with the target text is displayed. In this way, by identifying the objects included in the target text, determining the target image and directly releasing the target image, and then after the target image is released, the target image is displayed based on the trigger operation for the image indication information, thereby simplifying the operation in the target image generation process and improving the efficiency and convenience of the image generation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a schematic diagram of the architecture of an image generation system 100 provided in an embodiment of the present application;

[0043] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0044] Figure 3 It is a flowchart of the image generation method provided in the embodiment of the present application;

[0045] Figure 4 is a schematic diagram of a target text determination process provided by an embodiment of the present application;

[0046] Figure 5 is a schematic diagram of an object image corresponding to each object provided in an embodiment of the present application;

[0047] Figure 6 is a schematic diagram of an image generation control provided in an embodiment of the present application;

[0048] Figure 7 is a schematic diagram of displaying at least one selected target object image in a content editing area provided by an embodiment of the present application;

[0049] Figure 8 is a schematic diagram of a target object image selection process provided by an embodiment of the present application;

[0050] Fig. 9is a schematic diagram of an operation control corresponding to a target object image provided in an embodiment of the present application;

[0051] Fig.10 is a schematic diagram of a drawing control provided in an embodiment of the present application;

[0052] Fig.11 is a schematic diagram of an image style control provided in an embodiment of the present application;

[0053] Fig.12 is a schematic diagram of a generation process of an image style control provided in an embodiment of the present application;

[0054] Fig.13 This is a schematic diagram of providing waiting prompt information in an embodiment of the present application;

[0055] Fig.14 is a schematic diagram of a target image selection process provided by an embodiment of the present application;

[0056] Fig.15 is a schematic diagram of an image operation control provided in an embodiment of the present application;

[0057] Fig.16 is a schematic diagram showing a target image and image indication information provided by an embodiment of the present application;

[0058] Fig.17 It is a flowchart of the image generation method provided in the embodiment of the present application;

[0059] Fig.18 It is a schematic diagram of the process of generating an image based on an image model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0061] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0062] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0064] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0065] 1) In response, it is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or have a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed are executed.

[0066] 2) Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0067] 3) Language model, which is a program that can understand language rules and context. It can predict the next word or character that may appear based on the previous text content.

[0068] 4) Word segmentation is to segment a continuous text according to certain rules or algorithms, and divide it into meaningful words or characters to facilitate language model processing and prediction. In Chinese, word segmentation usually divides a continuous Chinese character text into words, for example, "I love China" is divided into three words: "I", "love", and "China".

[0069] 5) Encyclopedia page: An encyclopedia page is a page created for a specific topic or entry on an online encyclopedia website. These pages usually contain a detailed introduction to the topic or entry, historical background, related people, events, places, things, and other information, as well as related pictures, videos, reference materials, etc.

[0070] 6) Prompt, also known as "prompt", "input prompt" or "input statement". It is a piece of text or language input provided to the machine to allow it to generate specific text or language output. In tasks such as natural language processing, machine translation, and dialogue generation, a prompt can be a user's question or answer, or a piece of text or language input generated by the system.

[0071] 7) Image model, also known as image generation model, is an artificial intelligence model that can analyze, identify and process images. Image models are usually trained using deep learning technology to better understand the features and patterns in images and perform tasks such as classification, detection, and segmentation. Common image models include convolutional neural networks, recurrent neural networks, and generative adversarial networks. These models are widely used in computer vision, image recognition, object detection, image segmentation, image generation, and other fields. In this case, image models are mainly used for image segmentation and image generation.

[0072] 8) Image segmentation is an image processing technique that divides an image into multiple regions or pixels, each of which has similar features or attributes.

[0073] 9) Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0074] See also Figure 1 , Figure 1It is an architectural diagram of an image generation system 100 provided in an embodiment of the present application. To realize an application scenario of image generation (for example, the application scenario of image generation may be that when a user is reading, a target text is displayed. When an image generation instruction for the target text is received, an object image corresponding to the generated object is displayed. Then, by editing the object image, an edited target image associated with the target text is obtained. Then, the target image is published and an underline is displayed below the target text. When a trigger operation for the underline is received, the target image is displayed), an image generation client 401 (such as a browser APP) is provided on a terminal (a terminal 400 is shown as an example). The terminal 400 is connected to the server 200 via a network 300, wherein the network 300 may be a wide area network or a local area network, or a combination of the two, and data transmission is realized using a wireless or wired link.

[0075] The server 200 is used to send the target text to the terminal 400;

[0076] Terminal 400 is also used to receive a target text; display the target text, which includes text corresponding to at least one object; in response to an image generation instruction for the target text, display the generated object images corresponding to each object; based on the generated object images, determine to display a target image associated with the target text, the target image includes at least one of the objects; in response to a release instruction for the target image, release the target image and display image indication information in an associated area of ​​the target text, the image indication information being used to indicate that there is an image associated with the target text; and when a trigger operation for the image indication information is received, display the target image associated with the target text.

[0077] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Deliver Network), and big data and artificial intelligence platforms. The terminal 400 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a set-top box, an intelligent voice interaction device, a smart home appliance, a virtual reality device, a vehicle-mounted terminal, an aircraft, and a mobile device (e.g., a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device, an intelligent speaker, and a smart watch), etc., but is not limited thereto. The terminal device and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0078] Next, an electronic device that implements the image generation method provided in the embodiment of the present application is described. Figure 2 , Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, with the electronic device as Figure 1 Take the terminal shown in as an example, Figure 2 The electronic device shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not shown in FIG. Figure 2 Various buses are labeled as bus system 440 .

[0079] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0080] The user interface 430 includes one or more output devices 431 that enable display of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0081] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0082] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0083] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0084] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0085] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB), etc.;

[0086] a display module 453 for enabling display of information via one or more output devices 431 (e.g., display screens, speakers, etc.) associated with the user interface 430 (e.g., a user interface for operating peripheral devices and displaying content and information);

[0087] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.

[0088] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2 The image generating device 455 stored in the memory 450 is shown, which can be software in the form of a program and a plug-in, etc., and includes the following software modules: a first display module 4551, a second display module 4552, a determination module 4553, a publishing module 4554, and a third display module 4555. These modules are logical, and therefore can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0089] In other embodiments, the device provided in the embodiments of the present application can be implemented in hardware. As an example, the image generating device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the image generating method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs) or other electronic components.

[0090] In some embodiments, the terminal or server can implement the image generation method provided in the embodiments of the present application by running a computer program. For example, the computer program can be a native program or software module in the operating system; it can be a native application (APP, Application), that is, a program that needs to be installed in the operating system to run, such as an instant messaging APP, a web browser APP; it can also be a small program, that is, a program that can be run only by downloading it to a browser environment; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plug-in.

[0091] Based on the above description of the image generation system and electronic device provided by the embodiment of the present application, the image generation method provided by the embodiment of the present application is described below. In actual implementation, the image generation method provided by the embodiment of the present application can be implemented by the terminal or the server alone, or by the terminal and the server in collaboration, so that Figure 1 The terminal 400 in the embodiment of the present application alone executes the image generation method provided in the embodiment of the present application as an example for explanation. Figure 3 , Figure 3 is a flow chart of the image generation method provided in the embodiment of the present application, which will be combined with Figure 3 The steps shown are explained.

[0092] Step 101: The terminal displays a target text, where the target text includes text corresponding to at least one object.

[0093] In actual implementation, the terminal is provided with a client supporting image generation, such as a reading client, etc. When the user opens the client on the terminal and the terminal runs the client, the terminal can display the target text based on the client. The target text displayed here can be pre-stored locally in the terminal or obtained by the terminal from the outside world (such as the Internet).

[0094] It should be noted that the target text is the text content selected by the user, and the target text belongs to the target work, which includes but is not limited to literary works, and media information including at least one of picture information and article information in the information flow page. Based on this, before displaying the target text, the text content included in the target work can also be displayed; in response to the selection operation of the text in the text content, the selected text is displayed using the target style; wherein the target style is used to distinguish the selected text from the unselected text in the text content; and the selected text is determined as the target text.

[0095] It should be noted that the target style can be pre-set, such as shadow display, or black frame display, etc., and the selection operation can include at least one of a pressing operation, a sliding operation or a dragging operation. Specifically, in response to the selection operation of the text in the text content, the process of displaying the selected text using the target style can be, in response to the pressing operation on the text content, displaying the text selection cursor; in response to the dragging operation on the text selection cursor, displaying the process of dragging the text selection cursor from the first position in the text content, and displaying the text content skipped by the text selection cursor during the dragging process; in response to the release operation on the text selection cursor, determining the position of the text selection cursor in the text content when the release operation is received as the second position, and using the target style to display the text content between the first position and the second position; determining the text content between the first position and the second position as the selected text. The object here can refer to the entity in the target text, that is, an object that can be displayed in an objective form, such as flowers, grass, cars, tables, refrigerators, etc.

[0096] For example, see Figure 4 , Figure 4 is a schematic diagram of the target text determination process provided by the embodiment of the present application, based on Figure 4 , Figure 4 Figure a in the middle is the text content included in the target work. In response to the selection operation on the text in the text content, the selected text is displayed in a black frame as shown in 401, that is, the text shown in FIG. Figure 4 The content shown in Figure b.

[0097] Step 102 : In response to an image generation instruction for the target text, display the generated object images corresponding to the objects.

[0098] In actual implementation, when an image generation instruction for a target text is received, object recognition is performed on the target text in response to the image generation instruction for the target text to obtain text corresponding to at least one object in the target text; then semantic analysis is performed on the text corresponding to each object to generate an object image corresponding to each object.

[0099] It should be noted that, for the process of performing object recognition on the target text and obtaining the text corresponding to at least one object in the target text, it can be that, through an AI model such as a language model, based on word attributes such as verbs, nouns, etc., the target text is segmented to obtain at least one segmentation result, and then a target segmentation result indicating a noun is selected from the at least one segmentation result, so as to determine the target segmentation result as the text corresponding to at least one object in the target text; and for the process of performing semantic analysis on the text corresponding to each object and generating the object image corresponding to each object, it can be that a pre-trained AI model such as an image generation model is obtained, based on the pre-trained AI model The AI ​​model performs semantic analysis on the text corresponding to each object and generates an object image corresponding to each object; here, the object image is the image corresponding to the entity in the target text. When the object is an entity, object recognition is performed on the target text to obtain the target text including at least one text corresponding to the object, that is, the target text is subjected to entity recognition to obtain the target text including at least one text corresponding to the entity. For example, when the object is an entity and the text corresponding to the entity is azalea, entity recognition is performed on the target text to obtain the target text including azalea text, so that the entity image corresponding to the generated entity can be an image including azalea.

[0100] For example, see Figure 5 , Figure 5 is a schematic diagram of the object image corresponding to each object provided in the embodiment of the present application, based on Figure 5 , where the object is an entity, the text in the black box shown in the dotted box 501 is the text corresponding to at least one object, i.e., the entity, and the object image, i.e., the entity image, corresponding to each object, i.e., the entity is shown in the dotted box 502. Specifically, object recognition, i.e., entity recognition, is performed on the target text to obtain a target text including at least one object, i.e., the text corresponding to the entity as shown in the dotted box 501; then, semantic analysis is performed on the text corresponding to each object, i.e., the entity, to generate an object image, i.e., the entity image, corresponding to each object, i.e., the entity as shown in the dotted box 502.

[0101] In actual implementation, before displaying the generated object images corresponding to each object in response to the image generation instruction for the target text, the image generation control may be displayed for receiving the image generation instruction; in response to the trigger operation for the image generation control, the image generation instruction is received; here, after displaying the target text, the image generation control may be displayed directly, or, after displaying the target text, in response to the pressing operation for the target text, the image generation control may be displayed. For example, see Figure 6 , Figure 6 is a schematic diagram of an image generation control provided in an embodiment of the present application, based on Figure 6After displaying the target text as indicated by 602, the image generation control as indicated by the dotted box 601 is displayed, thereby, in response to the trigger operation for the image generation control as indicated by the dotted box 601, the image generation instruction is received.

[0102] In some embodiments, the process of displaying the generated object images corresponding to each object in response to an image generation instruction for a target text may also be displaying an image editing interface in response to an image generation instruction for the target text, thereby displaying the object images corresponding to each object in the image editing interface; here, the image editing interface may also include the target text, thereby also displaying the target text in the image editing interface, specifically, using a first display style to display the text corresponding to each object in the target text, and using a second display style to display the text in the target text except the text corresponding to each object; wherein the first display style is different from the second display style.

[0103] It should be noted that the image editing interface can be a pop-up window, such as a pop-up window displayed from bottom to top, or a pop-up window displayed from left to right, etc.; and the positions of the target text and the object image included in the image editing interface can be pre-set, for example, the target text is displayed above the object image, or the target text is displayed in the area below the object image, etc.; here, the first display style and the second display style are different pre-set display styles, for example, the first display style can be yellow brush, highlight, black frame or underline display, etc., and the second display style can be normal display, such as display without display effect, etc. For example, continue to refer to Figure 5 ,based on Figure 5 In the image editing interface as indicated by 504, the target text shown in the dotted box 501 includes texts corresponding to multiple objects. Therefore, the text corresponding to each object in the target text is displayed in a black box, and the text in the target text except the text corresponding to each object is displayed normally.

[0104] In actual implementation, the display order of at least one object image can be corresponding to the display order of the text corresponding to the object in the target text, or it can be randomly arranged. The embodiment of the present application does not limit this. Specifically, as described above, after generating the object image corresponding to each object, it is also possible to obtain the appearance order of the text corresponding to each object in the target text; and display the object image corresponding to each object in sequence according to the appearance order.

[0105] Step 103: Based on the generated object images, determine and display the generated target image associated with the target text, wherein the target image includes at least one object.

[0106] In actual implementation, at least one object included in the target image is associated with at least one object included in the target text; after each object image is generated, a content editing area may also be displayed, so that the selected object image is displayed in the content editing area, and a target image associated with the target text is generated based on the selected object image. Here, as described above, the image editing interface may also include a content editing area, and the position of the content editing area in the image editing interface may be pre-set, and this application does not limit this; for example, continue to refer to Figure 5 ,based on Figure 5 , 503 indicates a displayed content editing area, and thus, in the content editing area, the selected object image is displayed, and a target image associated with the target text is generated based on the selected object image.

[0107] In some embodiments, after displaying the content editing area, the selected object image can be directly displayed in the content editing area, so that a target image associated with the target text can be generated based on the selected object image; thus, the process of determining the target image associated with the target text based on the generated object images can be, in response to a selection operation on at least one of the generated object images, displaying at least one selected target object image in the content editing area; based on at least one target object image displayed in the content editing area, in response to an image generation instruction, displaying the generated target image associated with the target text, the target image is generated based on the target object image in the content editing area, that is, the image content of the target image is integrated with at least one target object image displayed in the content editing area.

[0108] It should be noted that, in response to a selection operation on at least one of the generated object images, after the selected at least one target object image is displayed in the content editing area, a determination function item is also displayed while displaying at least one object image based on the content editing area. The determination function item is used to determine the generation of a target image from at least one object image displayed in the content editing area. Thus, in response to a trigger operation on the determination function item, an image generation instruction is received, and then in response to the image generation instruction, the generated target image associated with the target text is displayed.

[0109] For example, see Figure 7 , Figure 7 is a schematic diagram of displaying at least one selected target object image in a content editing area provided by an embodiment of the present application, based on Figure 7In response to the selection operation for the object image shown in the dotted box 701 in Figure a, in the content editing area, at least one selected target object image as shown in the dotted box 702 in Figure b is displayed, and at the same time, the determination function item as shown in 703 in Figure b is displayed, thereby responding to the trigger operation for the determination function item shown in 703, receiving the image generation instruction, and then responding to the image generation instruction, the generated target image associated with the target text is displayed.

[0110] In actual implementation, for the process of displaying the selected at least one target object image in the content editing area in response to a selection operation on at least one of the generated object images, there may be multiple forms of selection operations, each object image is associated with an add control, so the selection operation may be a trigger operation on the add control associated with the object image; or the selection operation may be a drag operation on the object image, which is not limited in the embodiments of the present application. Next, based on different forms of selection operations, the process of displaying the selected at least one target object image in the content editing area in response to a selection operation on at least one of the generated object images is described.

[0111] In some embodiments, each object image is associated with an add control; the selection operation is a trigger operation for the add control associated with the object image, so that in response to the selection operation for at least one of the generated object images, the process of displaying the selected at least one target object image in the content editing area may be to display the added at least one target object image in the content editing area in response to the trigger operation respectively performed for the add control associated with at least one object image. For example, see Figure 8 , Figure 8 is a schematic diagram of the target object image selection process provided by the embodiment of the present application, based on Figure 8 , the dotted box 801 indicates the add controls associated with each object image, and in response to the triggering operation of the first add control in the dotted box 801, the added target object image as indicated by 802 is displayed in the content editing area.

[0112] In some other embodiments, the selection operation is a drag operation on the object image, so that in response to the selection operation on at least one of the generated object images, the process of displaying the selected at least one target object image in the content editing area may be that in response to the drag operation performed on the target object image in the at least one object image, at least one added target object image is displayed in the content editing area. Specifically, in response to the drag operation performed on the target object image in the at least one object image, the process of dragging the target object image to the content editing area is displayed; when the target object image is located above the content editing area, in response to the release operation on the target object image, the added target object image is displayed in the content editing area.

[0113] In actual implementation, after displaying at least one added target object image, the add control associated with the target object image can be switched to an add completion control; wherein the add completion control is used to indicate that the target object image has been added to the content editing area. Here, the add completion control is displayed differently from the add control. Figure 8 ,based on Figure 8 , 803 indicates an adding completion control associated with the target object image. Here, the adding control is displayed in the form of a plus sign, indicating that the related object image has not been added to the content editing area. The adding completion control is displayed in the form of a check mark, indicating that the target object image has been added to the content editing area. Specifically, in the content editing area, after displaying at least one selected target object image, the first adding control in the dotted box 801 is switched to the adding completion control as indicated by 803.

[0114] In actual implementation, the target object image in the content editing area may also be cancelled. Specifically, when a trigger operation for the add completion control is received, the target object image is cancelled in the content editing area, and the add completion control is switched to the add control. Figure 8 ,based on Figure 8 , 803 indicates the addition completion control associated with the target object image. When a trigger operation is received for the addition completion control indicated by 803, the target object image is canceled in the content editing area, and the addition completion control indicated by 803 is switched to the addition control indicated by 804.

[0115] It should be noted that the Add Complete control is also used to cancel the display of multiple object images with one click, that is, it is used to cancel the display of all target object images associated with the Add Complete control in the content editing area. Specifically, when there are multiple target object images associated with the Add Complete control in the content editing area, in response to the trigger operation of the Add Complete control, a cancel display instruction for all target object images associated with the Add Complete control in the content editing area is received, and thus all target object images associated with the Add Complete control in the content editing area are canceled in response to the cancel display instruction.

[0116] In other embodiments, after displaying the content editing area, a selected object image can be displayed in the content editing area, and then an editing operation can be performed on the selected object image to generate a target image associated with the target text based on the target editing content obtained by editing; thus, the process of determining the target image associated with the target text based on the generated object images can be, in response to a selection operation on at least one of the generated object images, displaying at least one selected target object image in the content editing area; in response to an editing operation on the target object image triggered in the content editing area, displaying the target editing content obtained by editing; in response to an image generation instruction for the target editing content, displaying the generated target image associated with the target text, the target image being generated based on the target editing content.

[0117] It should be noted that, in response to the selection operation for at least one of the generated object images, after displaying the selected at least one target object image in the content editing area, an editing operation can also be performed on the target object image in the content editing area, and the editing operation here includes but is not limited to deletion, copying, rotation, graffiti, movement, enlargement and reduction, etc. Thus, in response to the editing operation for the target object image triggered in the content editing area, the edited target editing content is displayed; here, while displaying the edited target editing content based on the content editing area, a determination function item is also displayed, and the determination function item is used to determine the edited target editing content displayed in the content editing area, so that, in response to the triggering operation for the determination function item, an image generation instruction is received, and then in response to the image generation instruction, the generated target image associated with the target text is displayed.

[0118] Next, based on different editing operations, a process of displaying target editing content obtained by editing in response to an editing operation on a target object image triggered in a content editing area is described.

[0119] In some embodiments, the editing operation may be a copy operation, a rotation operation, and a deletion operation, etc.; specifically, in response to an editing operation triggered on the target object image in the content editing area, the process of displaying the target editing content obtained by editing may be, in response to a triggering operation on the target object image, such as a click operation, a double-click operation, or a press operation, displaying at least one operation control, the operation control being used to perform an editing operation on the target object image, different operation controls indicating different editing operations; in response to a triggering operation on a target operation control in at least one operation control, performing a target editing operation indicated by the target operation control on the target object image; when the target editing operation on the target object image is completed, displaying the target editing content obtained by editing.

[0120] It should be noted that the at least one operation control here can be displayed at an associated position of the target object image, and the associated position is used to indicate at least one of the upper position, lower position, left position and right position of the target object image; for example, see Fig. 9 , Fig. 9 is a schematic diagram of an operation control corresponding to a target object image provided in an embodiment of the present application, based on Fig. 9 , 901 indicates a copy control, 902 indicates a rotation control, and 903 indicates a delete control. In actual applications, in response to a trigger operation on a target object image, operation controls as indicated by 901, 902, and 903 are displayed, thereby responding to a trigger operation on a target operation control in at least one operation control, the target editing operation indicated by the target operation control is performed on the target object image.

[0121] In actual implementation, when the editing operation is a copy operation, the target operation control may be a copy control, so that in response to a trigger operation on the target operation control in at least one operation control, the process of executing the target editing operation indicated by the target operation control on the target object image may be, in response to the trigger operation on the copy control, copying the target object image, so that when the target editing operation on the target object image is completed, the process of displaying the edited target editing content may be, displaying the copied target object image; or, when the editing operation is a rotation operation, the target operation control may be a rotation control, so that in response to a trigger operation on the target operation control in at least one operation control, the process of executing the target editing operation indicated by the target operation control on the target object image may be, in response to a drag operation on the rotation control The process of rotating the target object image by a dynamic operation may be displayed; thus, when the target editing operation on the target object image is completed, the process of displaying the target editing content obtained by editing may be, when the rotation process is completed, in response to the release operation on the rotation control, displaying the rotated target object image; or, when the editing operation is a deletion operation, the target operation control may be a deletion control, thus, in response to the triggering operation on the target operation control in at least one operation control, the process of executing the target editing operation indicated by the target operation control on the target object image may be, in response to the triggering operation on the deletion control, deleting the target object image; thus, when the target editing operation on the target object image is completed, the process of displaying the target editing content obtained by editing may be, when the deletion process is completed, canceling the display of the target object image.

[0122] It should be noted that, in the case where the editing operation is a deletion operation and the target operation control is a deletion control, the deletion control here is used to cancel the display of the corresponding target object image, which is different from the adding completion control mentioned above for canceling the display of multiple object images with one click. The deletion control here cancels the display of corresponding object images one by one. Therefore, when there are multiple target object images with the same image content displayed in the content editing area, it is also possible to cancel the display of multiple target object images in the content editing area in response to the triggering operation of the deletion control for each target object image; when each target object image in the content editing area is canceled, the adding completion control is switched to the adding control.

[0123] In some other embodiments, the editing operation may be a drawing operation, specifically, displaying at least one drawing control, and different drawing controls correspond to different drawing tools; thus, in response to an editing operation on a target object image triggered in the content editing area, a process of displaying the edited target editing content may be, in response to a triggering operation on a target drawing control in at least one drawing control, controlling the content editing area to be in a drawing mode; in the drawing mode, in response to a drawing operation triggered based on the drawing tool corresponding to the target drawing control, displaying the drawn target editing content including the target object image.

[0124] It should be noted that the drawing tools include but are not limited to graffiti pens, mosaic pens, erasing pens, spray pots, etc. When a trigger operation for a corresponding target drawing control is received, the control content editing area is placed in a drawing mode corresponding to the target drawing control. For example, when the drawing tool corresponding to the target drawing control is a graffiti pen, the control content editing area is placed in a drawing mode corresponding to the graffiti pen, namely, a graffiti mode. In the graffiti mode, in response to a drawing operation triggered by the graffiti pen, graffiti content including a target object image is displayed; when the drawing tool corresponding to the target drawing control is a spray pot, the control content editing area is placed in a drawing mode corresponding to the spray pot, namely, a spray mode. In the spray mode, in response to a drawing operation triggered by the spray pot, spray content including a target object image is displayed.

[0125] For example, see Fig.10 , Fig.10 is a schematic diagram of a drawing control provided in an embodiment of the present application, based on Fig.10 , 1001 indicates a graffiti control. In response to a trigger operation on the graffiti control indicated by 1001, the content editing area is controlled to be in a drawing mode; in the drawing mode, in response to a drawing operation triggered by a drawing tool corresponding to the target drawing control, the graffiti content including the target object image as shown in FIG. b is displayed.

[0126] In actual implementation, for the above process of displaying the target image associated with the target text in response to the image generation instruction for the target editing content, after receiving the image generation instruction for the target editing content, the target object image and the drawing content in the target editing content can be analyzed separately to obtain the association relationship between the target object image and the drawing content, and then the target editing content including the target object image and the drawing content is combined with the association relationship to generate the target image associated with the target text. Here, the association relationship is used to indicate the operation performed on the target object image based on the drawing content.

[0127] It should be noted that the drawing content here can be either handwriting lines that depict the object image or symbols that have a semantic connection with the object image, such as drawing a plus sign between two object images, and the association relationship is used to indicate that the two objects will be merged in the subsequent image generation process; for example, a curve with an arrow is drawn next to an object, and the association relationship is used to indicate that the object will be moved along the curve trajectory to the end point of the curve in the subsequent image generation process, etc.

[0128] In other embodiments, the editing operation may also be a moving operation such as a single-finger dragging operation, a zooming operation such as a two-finger separation operation, and a zooming operation such as a two-finger pinching operation. Specifically, when the editing operation is a moving operation, in response to the dragging operation on the target object image, the process of moving the target object image is displayed; in response to the release operation on the target object image, the moved target object image is displayed; when the editing operation is a zooming operation, in response to the zooming operation on the target object image, the process of zooming in the target object image is displayed; in response to the release operation on the target object image, the zoomed in target object image is displayed; when the editing operation is a zooming operation, in response to the zooming in operation on the target object image, the process of zooming out the target object image is displayed; in response to the release operation on the target object image, the zoomed out target object image is displayed.

[0129] In some embodiments, the image style of the generated target image can also be set. Specifically, at least one image style control is displayed, and the image style control is used to indicate the image style of the generated target image; in response to a selection operation on a target image style control in at least one image style control, the target image style control is controlled to be in a selected state; thereby, based on the generated object images, a process of determining a target image associated with a target text can be, when the target image type control is in a selected state, in response to an image generation instruction for each generated object image, a target image with a target image style is displayed; wherein the target image style is the image style indicated by the target image style control.

[0130] It should be noted that the image style can be used to indicate the display method of the image content of the target image, such as the same image content can be displayed in an abstract style, a realistic style, or a cartoon style, etc. It can also be used to indicate the display scene of the object included in the target image, such as the display scene for the same object can be rain, snow, or the ocean, etc. At the same time, the image style control here carries the text of the corresponding image style, so that the user can determine the image style corresponding to the corresponding image style control based on the displayed text.

[0131] It should be noted that the selected state and the unselected state here can be reflected by different graphic identifiers. For example, the image style control in the selected state can be marked with a first graphic identifier, such as a check mark, while the image style control that is not in the selected state can be marked with a second graphic identifier, such as a plus sign; or, the selected state and the unselected state can be reflected by different display effects. For example, the image style control in the selected state can be displayed with a first display effect, such as a special effects state, while the image style control that is not in the selected state can be displayed with a second display effect, such as a non-special effects state.

[0132] For example, see Fig.11 , Fig.11 is a schematic diagram of an image style control provided in an embodiment of the present application, based on Fig.11 , indicated in the dotted box 1101 are three image style controls. In response to the selection operation of the image style control corresponding to the abstract style among the three image style controls, the image style control controlling the abstract style is in a selected state; here, the image style control in the selected state is marked with a check mark, while the image style control that is not in the selected state is marked with a plus sign; then in response to the image generation instruction for each generated object image, a target image with an abstract style is displayed.

[0133] It should be noted that multiple image styles can also be selected, so that, in response to the image generation instruction for each generated object image, a target image with two image styles is displayed. Fig.11 , in response to the selection operation of the image style control corresponding to the abstract style and the image style control corresponding to the cartoon style among the three image style controls, the image style control of the abstract style and the image style control corresponding to the cartoon style are controlled to be in a selected state; then in response to the image generation instructions for each generated object image, the target images with abstract style and cartoon style are displayed.

[0134] In some embodiments, the image style can also be edited by oneself. Specifically, in response to a display instruction of the image style editing box, an image style editing box for setting the image style of the target image is displayed; in the image style editing box, the input text is displayed, and the text is used to indicate the target image style of the desired target image; thus, based on the generated object images, a process of determining the target image associated with the target text can be, in response to a determination instruction for the text input in the image style editing box, generating a target image with the target image style based on the generated object images.

[0135] It should be noted that, in response to the display instruction of the image style edit box, before displaying the image style edit box for setting the image style of the target image, an image style customization control may also be displayed, so that in response to the trigger operation for the image style customization control, the display instruction of the image style edit box is received. Then, in response to the determination instruction for the text input in the image style edit box, the process of generating a target image with a target image style based on each generated object image may also be to display a determination control, which is used to determine to generate an image style control carrying the input text; in response to the trigger operation for the determination control, a target image with a target image style is generated based on each generated object image; here, in response to the trigger operation for the determination control, the process of generating a target image with a target image style based on each generated object image may be to generate an image style control carrying the input text in response to the trigger operation for the determination control, and control the image style control to be in a selected state; and then in response to the image generation instruction for each generated object image, a target image with a target image style is generated. In this way, a corresponding image style control is generated based on the text input by the user, so that the user can directly generate a target image of a corresponding style based on the image style control during the next image generation process.

[0136] For example, see Fig.12 , Fig.12 is a schematic diagram of the generation process of the image style control provided in the embodiment of the present application, based on Fig.12 , the fourth custom control in the dotted box 1201 is an image style custom control, so in response to the trigger operation for the image style custom control, an image style editing box for setting the image style of the target image as indicated by 1202 in FIG. b is displayed; then as shown in FIG. c, in the image style editing box, the input text, i.e. "light rain", is displayed, and then in response to the trigger operation for the determination control as indicated by 1203, an image style control carrying the input text as indicated by 1204 in FIG. d is generated, and the image style control is controlled to be in a selected state; then in response to the image generation instructions for each generated object image, a target image with a target image style is generated.

[0137] In some embodiments, during the image generation process, prompt information may also be displayed. Specifically, the process of determining the target image associated with the target text based on the generated object images may also be to display waiting prompt information in response to the image generation instruction for each generated object image; wherein the waiting prompt information is used to indicate that the target image is being generated; when it is determined that the generation of the target image is completed based on the waiting prompt information, the generated target image associated with the target text is displayed. Here, for determining that the generation of the target image is completed based on the waiting prompt information, it may be that when the waiting prompt information disappears, it is determined that the generation of the target image is completed.

[0138] It should be noted that, as mentioned above, the image generation instruction can be triggered by determining the control, and the waiting prompt information here can be displayed in any area of ​​the image editing page, such as being displayed in the content editing area according to a preset setting, and at the same time, the display size and display transparency of the waiting prompt information can also be preset, such as the display size can be the same as the size of the content editing area, and the display transparency can be non-transparent, thereby covering the content editing area. For example, see Fig.13 , Fig.13 This is a schematic diagram of a waiting prompt information provided by an embodiment of the present application, based on Fig.13 In response to the trigger operation for the frame indicated by 1301, an image generation instruction for each object image to be generated is received; in response to the image generation instruction, a waiting prompt information as indicated by the dotted box 1302 in Figure b is displayed; here, the display size of the waiting prompt information is the same as the size of the content editing area, and the display transparency is non-transparent; when it is determined based on the waiting prompt information that the generation of the target image is completed, the generated target image associated with the target text is displayed.

[0139] In some embodiments, for the process of determining the target image associated with the target text based on each generated object image, here, it can be a process of generating the target image associated with the target text based on each generated object image through a trained AI model, such as an image generation model. Specifically, after obtaining each generated object image or the target edited content obtained by editing, the convolutional neural network or recurrent neural network in the image generation model is used to extract features of each object image or the target edited content obtained by editing to obtain a corresponding feature vector, and then the deconvolutional neural network in the image generation model is used to generate a target image associated with the target text based on the corresponding feature vector.

[0140] In actual implementation, for the process of training the image generation model, first, establish the feature association relationship between the image and the object image, and the image and the drawing content in the editing content, that is, segment each image to obtain the object information and drawing information of the image, wherein the object information is used to indicate the object included in the image (such as flowers, grass, cars, tables, refrigerators, etc.), as well as the coordinates, size, deformation information, etc. of the object, and the drawing information is used to indicate the image content that can be drawn in the image. Specifically, first obtain the original data set including multiple images, and then, on the one hand, perform image segmentation such as threshold segmentation, region growing, edge detection, cluster analysis, etc. on the image to obtain the object information in the image;

[0141] On the other hand, the image is gray-processed to obtain a gray-scale image; and the gray-scale image is edge-detected to obtain edge information in the image; then, based on the edge information, the gray-scale image is binarized to obtain a binary image; then, the binary image is subjected to line extraction processing such as line extraction algorithms, such as Hough transform, straight line segment detection, curve fitting, etc., that is, line information is extracted from the binary image to obtain multiple lines in the binary image, and finally, the multiple extracted lines are standardized, such as removing duplicate lines, merging adjacent lines, smoothing lines, etc., to obtain standardized lines, and the standardized lines are determined as the drawing information of the image.

[0142] In actual implementation, after obtaining the object information and drawing information of each image, the image generation model is trained using the object information and drawing information, the gradient is calculated through the back propagation algorithm, and the parameters of the model are updated using the optimization algorithm, that is, the object information and drawing information of each image are feature extracted through the convolutional neural network or recurrent neural network in the image generation model to obtain the corresponding feature vector; then, the final image is generated based on the corresponding feature vector through the deconvolution neural network in the image generation model, and the difference between the generated final image and the image corresponding to the object information and drawing information is obtained, so as to update the model parameters of the image generation model based on the difference.

[0143] In some embodiments, after displaying the object images corresponding to each generated object, it is also possible, in response to a trigger operation on a target object image among the generated object images, to display an information introduction page of the object corresponding to the target object image; wherein the information introduction page includes introduction information, and the introduction information is used to introduce the target object corresponding to the target object image.

[0144] It should be noted that the trigger operation for the target object image among the generated object images can be a click operation, and the information introduction page can be an encyclopedia page of the target object image, that is, the introduction information can include at least one of the name, source, distribution and form of the target object.

[0145] In some embodiments, after the target text is displayed, in response to the image generation instruction for the target text, the generated target image associated with the target text can be directly displayed without user editing or selection, wherein the target image includes at least one object. The embodiment of the present application does not limit whether the object image corresponding to each object is displayed in response to the image generation instruction for the target text, or whether the target image associated with the target text is pre-set.

[0146] In some embodiments, the target image can be determined by user selection. Specifically, the process of determining the target image associated with the target text based on the generated object images can be, in response to the image generation instruction for each generated object image, displaying at least two candidate images associated with the target text; in response to the selection operation of the target candidate image among the at least two candidate images, determining the selected target candidate image as the target image.

[0147] It should be noted that the candidate images are different images that include the same object and have the same image style. The display area of ​​the multiple candidate images can be a content editing area. For example, see Fig.14 , Fig.14 is a schematic diagram of the target image selection process provided by the embodiment of the present application, based on Fig.14 In response to the image generation instruction for each object image generated, four candidate images with the target as indicated by the dotted box 1401 are displayed; in response to the selection operation for the target candidate image among the four candidate images, the selected target candidate image is determined as the target image.

[0148] In some embodiments, after determining the target image associated with the target text based on the generated object images, at least one image operation control associated with the target image can also be displayed, the image operation control is used to indicate the execution of an interactive operation on the target image, and different image operation controls indicate different interactive operations; in response to a trigger operation on a target image operation control in at least one image operation control, the target interactive operation indicated by the target image operation control is executed on the target image. Here, the image operation control includes at least a refresh control, a clear control, etc., and the image operation control can be displayed in any blank area in the content editing area.

[0149] In some embodiments, the image operation control is a refresh control, so that after determining the target image associated with the target text based on the generated object images, a refresh control for the target image can also be displayed; in response to a trigger operation on the refresh control, the target image is switched to a new target image associated with the target text; wherein the new target image is different from the target image, and the new target image includes at least one object.

[0150] It should be noted that the target image and the new target image are generated based on the target content displayed in the content editing area, that is, the target object image or the target editing content including the target object image. The new target image is the same as the target content displayed in the content editing area corresponding to the target image, and the image style is also the same. Specifically, in response to the trigger operation for the refresh control, the target content displayed in the content editing area corresponding to the target image is obtained; then, based on the target content, a new target image is generated. For example, see Fig.15 , Fig.15 is a schematic diagram of an image operation control provided in an embodiment of the present application, based on Fig.15 , 1501 indicates a refresh control for the target image. In response to a trigger operation for the refresh control indicated by 1501, the target image is switched to a new target image associated with the target text.

[0151] In other embodiments, the image operation control is a clear control. Thus, after determining the target image associated with the target text based on the generated object images, a clear control for the target image can be displayed. In response to a trigger operation on the clear control, the target image is canceled from display, and all contents in the content editing area are canceled from display, that is, the content editing area is controlled to be in a blank state.

[0152] It should be noted that the clear control here is used to delete the target image, thereby canceling the display of the target image, and is also used to clear the target content displayed in the content editing area, that is, the target object image or the target editing content including the target object image, so that the content editing area is in a blank state. Fig.15 ,based on Fig.15 , 1502 indicates a clear control for the target image. In response to the triggering operation of the clear control indicated by 1502, the target image is canceled from being displayed, and all contents in the display content editing area are deleted.

[0153] In actual implementation, after determining the target image associated with the target text based on the generated object images, in addition to generating a new target image based on the refresh control, a new target image can also be generated based on the add control and add completion control associated with the object image, as well as the image style control.

[0154] In some embodiments, in response to a triggering operation of adding a control associated with the object image, the target image may be switched to a new target image associated with the target text; wherein the new target image is different from the target image, the number of objects included in the new target image is greater than the number of objects included in the target image, and the objects included in the new target image are increased by the objects corresponding to the adding control compared to the target image; illustratively, see Fig.15 , the target image includes the first three object images indicated in the dotted box 1503. When a trigger operation of adding a control corresponding to the fourth object image indicated in the dotted box 1503 is received, the target image is switched to a new target image associated with the target text, wherein the new target image includes the four object images indicated in the dotted box 1503.

[0155] Alternatively, in response to a triggering operation of an add completion control associated with the object image, the target image is switched to a new target image associated with the target text; wherein the new target image is different from the target image, the number of objects included in the new target image is less than the number of objects included in the target image, and the objects included in the new target image lack the objects corresponding to the add completion control compared to the target image; illustratively, continue to refer to Fig.15 , the target image includes the first three object images indicated in the dotted box 1503. When a trigger operation of adding a completion control corresponding to the second object image indicated in the dotted box 1503 is received, the target image is switched to a new target image associated with the target text, wherein the new target image includes the two object images indicated in the dotted box 1503.

[0156] In other embodiments, it is also possible to switch the target image to a new target image associated with the target text in response to a trigger operation on the image style control in the selected state; wherein the new target image is different from the target image, the new target image is not associated with the image style corresponding to the image style control, the target image is associated with the image style corresponding to the image style control, and the image style associated with the new target image lacks the image style corresponding to the added control compared to the target image; exemplarily, continue to refer to Fig.15 , the target image is associated with the first two image styles indicated in the dotted box 1504. When a trigger operation is received for the third image style control indicated in the dotted box 1504, the target image is switched to a new target image associated with the target text, wherein the new target image is associated with the three image styles indicated in the dotted box 1504.

[0157] Alternatively, in response to a trigger operation on an image style control in a non-selected state, the target image is switched to a new target image associated with the target text; wherein the new target image is different from the target image, the new target image is associated with the image style corresponding to the image style control, the target image is not associated with the image style corresponding to the image style control, and the image style associated with the new target image adds the image style corresponding to the added control compared to the target image; illustratively, continue to refer to Fig.15 , the target image is associated with the first two image styles indicated in the dotted box 1504. When a trigger operation is received for the second image style control indicated in the dotted box 1504, the target image is switched to a new target image associated with the target text, wherein the new target image is associated with the first image style indicated in the dotted box 1504.

[0158] Step 104 , in response to the publishing instruction for the target image, publish the target image, and display image indication information in the associated area of ​​the target text, where the image indication information is used to indicate that there is an image associated with the target text.

[0159] In actual implementation, the publishing instruction for the target image is triggered by a publishing control. Specifically, after determining the target image associated with the target text, the publishing control can also be displayed; then, in response to the triggering operation for the publishing control, the publishing instruction is received. Fig.15 ,based on Fig.15 , 1505 indicates a publishing control, and thus in response to the trigger operation for the publishing control indicated by 505, a publishing instruction is received.

[0160] It should be noted that the target image is published here to indicate that the target image is shared to the browsing platform that displays the target text, so that the user can browse the target image associated with the target text when browsing the target text. Specifically, in response to the publishing instruction for the target image, the process of publishing the target image can be, in response to the publishing instruction for the target image, sending an image publishing request carrying the target image to the server, so that the server responds to the image publishing request, stores the target image, and establishes an association relationship between the target image and the target text, and when receiving a target work acquisition request including the target text sent by other terminals, the target work carrying the target image is sent to other terminals, so that when other terminals display the target text of the target work, the corresponding target image can also be viewed.

[0161] In some embodiments, in response to a publishing instruction for a target image, after the target image is published, a publishing prompt information may also be displayed, wherein the publishing prompt information is used to indicate that the corresponding target image is successfully published. Fig.16 , Fig.16 is a schematic diagram showing the target image and image indication information provided by the embodiment of the present application, based on Fig.16 After the target image is published, the publishing prompt information as indicated by 1601 is displayed.

[0162] In some embodiments, the image indication information includes but is not limited to text information, voice information, character information such as underline, wavy line, etc., and the associated area can be one of the upper area, lower area, left area and right area of ​​the target text. When the type of image indication information is different, the associated area is also different; when the image indication information is character information such as underline, the associated area here can be the lower area of ​​the target text, so the process of displaying the image indication information in the associated area of ​​the target text can be to display the character information below the target text. For example, continue to refer to Fig.16 ,based on Fig.16 , where the character information is underlined, and the associated area is the area below the target text.

[0163] Step 105: When a trigger operation for the image indication information is received, a target image associated with the target text is displayed.

[0164] In actual implementation, the trigger operation for the image indication information can be a pressing operation, a clicking operation, etc. When the trigger operation for the image indication information is received, after the target image associated with the target text is displayed, an interactive operation can also be performed on the target image. Specifically, at least one interactive control for the target image is displayed; wherein the interactive control is used to indicate the execution of an interactive operation on the target image, and different interactive controls indicate different interactive operations; in response to the trigger operation on the target interactive control among at least one interactive control, the target interactive operation indicated by the target interactive control is executed on the target image.

[0165] It should be noted that the interactive control here can be a like control, a comment control or a share control, etc., so that when the target interactive control is a like control, in response to the trigger operation of the like control, the target image is liked, and the number of likes of the target image is controlled to increase by one; when the target interactive control is a comment control, in response to the trigger operation of the comment control, the target image is commented on, and the number of comments on the target image is controlled to increase by one; when the target interactive control is a share control, in response to the trigger operation of the share control, the target image is shared, and the number of shares of the target image is controlled to increase by one. For example, continue to refer to Fig.16 ,based on Fig.16, the image indication information is character information. When a trigger operation for the character information is received, the target image associated with the target text as shown in FIG. b and the like control and the comment control as indicated by the dotted box 1603 are displayed. Thus, in response to the trigger operation for the target interactive control among the two interactive controls as indicated by the dotted box 1603, the target interactive operation indicated by the target interactive control is performed for the target image.

[0166] By applying the above-mentioned embodiment of the present application, a target text including at least one object is first displayed, and then in response to an image generation instruction for the target text, a target image associated with the target text is determined based on the object images corresponding to the objects generated by the display, and then the target image is released, and the image indication information is displayed, so that when a trigger operation for the image indication information is received, the target image associated with the target text is displayed. In this way, by identifying the objects included in the target text, determining the target image and directly releasing the target image, and then after the target image is released, the target image is displayed based on the trigger operation for the image indication information, thereby simplifying the operations in the target image generation process and improving the efficiency and convenience of the image generation process.

[0167] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0168] The inventors found that the application forms of image generation technology in the related art are basically independent web pages or PC (Personal Computer) products, and the generation methods are mainly text-generated images and image-generated images, that is, input a text prompt to generate an image, or generate an image based on a reference image. But it is not combined with the reading scene. If users of the reading app want to generate illustrations for a certain paragraph, they need to go to the relevant products, and then based on this type of product, they need to enter a text prompt to generate the corresponding image. The form is relatively simple and limited. It cannot automatically identify the entity (object) in the prompt and generate independent elements for users to move, scale, and other edits, nor can it allow users to edit the elements in the reference image in the product.

[0169] Based on this, an embodiment of the present application provides a method for self-service generation of illustrations in novels, allowing users to customize the generation of text illustrations. Specifically, the user can first select a paragraph of text, then intelligently identify the entities in the paragraph of text, and determine the entity preview image (object image) of the corresponding entity in combination with the entity, so as to perform moving, scaling and other editing on the entity preview image in the canvas, and supplement the sketch with simple drawings, so as to generate the final finished illustration with the sketch. At the same time, users can also publish illustrations. In this way, through this application, users can easily view popular science diagrams of objects without visual concepts, and easily create novel illustrations, making the reading experience more interesting and enabling users to interact and participate more in the reading community.

[0170] Next, the technical solution of this application is explained from the product side. In actual implementation, the target text selected by the user and the "AI illustration" button (image generation control) are displayed, and then in response to the trigger operation for the button, the entity in the target text is identified and the text corresponding to the entity is highlighted in the target text (first display style), and the entity image corresponding to each entity is displayed at the same time; then in response to the trigger operation for the target entity image in each entity, the encyclopedia page (information introduction page) of the target entity image is displayed.

[0171] In actual implementation, when the entity image corresponding to each entity is displayed, a plus sign-shaped add control is also displayed below each entity image. When a trigger operation is received for a target add control among multiple add controls, the entity image corresponding to the target add control is displayed in the canvas (content editing area) below the entity image, thereby adding the entity image to the canvas. At the same time, the target add control is switched to a check mark-shaped add completion control.

[0172] In actual implementation, the entity image in the canvas can also be edited. Specifically, in response to a drag operation on the entity image, the process of moving the entity image is displayed; or, in response to a zoom operation on the entity image, the size of the entity image is changed; or, in response to a trigger operation on the entity image, an elimination control (delete control), a rotation control, and a copy control corresponding to the entity image are displayed, thereby eliminating, rotating, and copying the entity image in response to the trigger operations on these three controls, respectively.

[0173] In actual implementation, when there are multiple identical entity images displayed in the canvas, it is also possible to cancel the display of multiple entity images in the canvas in response to a trigger operation of a eliminate control corresponding to each entity image; when each entity image in the canvas is canceled, the add completion control is switched to the add control; when a trigger operation of the add completion control below the entity image is received, the corresponding entity image in the canvas is cleared at the same time, and the add completion control below the entity image is switched to the add control.

[0174] In actual implementation, a graffiti button (drawing tool) can also be displayed on the canvas, so that in response to the triggering operation of the graffiti button, the canvas is controlled to be in drawing mode so that drawing can be performed on the canvas. The drawing content here can be either handwriting lines that depict the entity images in the picture, or symbols that have semantic connections with the picture contents. For example, drawing a plus sign between two entities can be intelligently recognized as a fusion of the two entities; for example, drawing a curve with an arrow next to an entity can be intelligently recognized as moving the entity along the curve trajectory to the end point of the curve, etc.

[0175] In actual implementation, a prompt word (image style control) about the style / image description can also be displayed below the canvas, so that after the user completes the canvas editing and clicks the confirmation button, the content on the canvas can be processed according to the prompt word. After the image generation is completed, you can also click to publish the illustration. Alternatively, click the add control below the entity image and the add completion control to add or delete entities, or click the prompt word below the canvas to add or delete prompt words. Here, each click causes the target image to be regenerated once, that is, refreshed in sequence.

[0176] In actual implementation, an eraser button and a refresh button are also displayed in the upper right corner of the canvas. When the eraser button is clicked, all the contents of the canvas are cleared; when the refresh button is clicked, the existing canvas contents are retained and the target image is regenerated.

[0177] In actual implementation, after the illustration is successfully published, an underline is displayed below the target text. When any user clicks on the underline, the illustration corresponding to the target text is displayed, and the illustration can also be liked / commented.

[0178] Next, the technical solution of this application is described from the technical perspective. Fig.17 , Fig.17 is a flow chart of the image generation method provided in the embodiment of the present application, based on Fig.17 , the image generation method provided in the embodiment of the present application is implemented through steps 1701 to 1715. Specifically, the target text selected by the user and the "AI Illustration" button are displayed. After receiving the trigger operation for the "AI Illustration" button, the client sends the target text selected by the user to the server, and then the server first uses the AI ​​language model to segment the text, and then, based on the segmentation results, identifies the entities in the target text and associates them with the encyclopedia page. Subsequently, the name, picture and encyclopedia page link of each entity are returned to the client. Then, the client displays a pop-up window (image editing interface), and the pop-up window content includes: entity words in the target text displayed based on the highlighted text; entity links returned by the server that can be used to jump to the encyclopedia page; canvases that support entity editing, stick figures, undo, redo and other operations; and prompts for multiple painting styles.

[0179] In actual implementation, after the user draws in the pop-up window, when a trigger operation is received for a certain control, the client will send the user's drawing content and the selected prompt to the server. The server uses the AI ​​image model to automatically generate illustrations based on the entities and prompts on the screen and returns them to the client. The client then displays the illustration so that the user can regenerate (refresh), erase and redraw (clear), add or subtract entities, and perform publishing operations on the illustration; after the illustration is published, the server is responsible for storing the illustration and associating it with the target text for all other users to view.

[0180] Next, for the image generation process, see Fig.18 , Fig.18 is a schematic diagram of a process of generating an image based on an image model provided in an embodiment of the present application, based on Fig.18 , this application combines the image entity (entity information) and graffiti sketch (drawing information) selected by the user to generate a picture. Specifically, here, a trained AI model such as an image generation model can be used to generate a target image associated with the target text based on the generated entity images. Specifically, after obtaining the generated target editing content including the entity information and drawing information, the convolutional neural network or recurrent neural network in the image generation model is used to extract features of the target editing content to obtain the corresponding feature vector, and then the deconvolutional neural network in the image generation model is used to generate a target image associated with the target text based on the corresponding feature vector.

[0181] In actual implementation, for the process of training the image generation model, first, establish the feature association relationship between the image and the entity image, and the image and the drawing content in the editing content, that is, segment each image to obtain the entity information and drawing information of the image, wherein the entity information is used to indicate the entity included in the image (such as flowers, grass, cars, tables, refrigerators, etc.), as well as the coordinates, size, deformation information, etc. of the entity, and the drawing information is used to indicate the image content that can be drawn in the image. Specifically, first obtain the original data set including multiple images, and then, on the one hand, perform image segmentation such as threshold segmentation, region growing, edge detection, cluster analysis, etc. on the image to obtain the entity information in the image;

[0182] On the other hand, the image is gray-processed to obtain a gray-scale image; and the gray-scale image is edge-detected to obtain edge information in the image; then, the image obtained by edge detection is binarized to obtain a binary image; then, the binary image is subjected to line extraction processing such as line extraction algorithms, such as Hough transform, straight line segment detection, curve fitting, etc., that is, line information is extracted from the binary image to obtain multiple lines in the binary image, and finally, the multiple extracted lines are standardized, such as removing duplicate lines, merging adjacent lines, smoothing lines, etc., to obtain standardized lines, and the standardized lines are determined as the drawing information of the image.

[0183] In actual implementation, after obtaining the entity information and drawing information of each image, the image generation model is trained using the entity information and drawing information, the gradient is calculated through the back propagation algorithm, and the parameters of the model are updated using the optimization algorithm, that is, the entity information and drawing information of each image are feature extracted through the convolutional neural network or recurrent neural network in the image generation model to obtain the corresponding feature vector; then, the final image is generated based on the corresponding feature vector through the deconvolution neural network in the image generation model, and the difference between the generated final image and the image corresponding to the entity information and drawing information is obtained, so as to update the model parameters of the image generation model based on the difference.

[0184] In this way, after the model training is completed, when the user uses the "AI Drawing" function, the model receives the entity picture selected by the user and the user's drawing information as input, converts it into a set of feature vectors, and then uses the deconvolution neural network to hand over the feature vectors to the trained image model for processing, and the image model converts it into an image.

[0185] In some embodiments, a text editing method can also be provided to allow users to edit the text content in the target text, and then the edited text content is given to the AI ​​image model as part of the prompt for image generation. In this way, the edited prompt text helps to add more feature information, such as the object, scene or color of the image, etc., thereby helping the model to better understand the content and features of the image and generate an image that better meets the requirements.

[0186] In this way, automatically generated illustrations can help readers better understand the plots and scenes in the novel and improve the reading experience. Moreover, through the innovative method of selecting entities and hand-drawing, users can create illustrations to express their ideas and creativity while reading, which increases the user's participation in the novel reading scene. At the same time, the automatic generation of illustrations saves a lot of time and cost required for manual drawing, improves the efficiency of illustration generation, and compared with pure hand-drawn illustrations, this application can quickly optimize and improve the user's hand-drawn stick drawings to improve the quality and aesthetics of the illustrations. In addition, this application helps users transform their ideas and creativity into more expressive and artistic illustrations. Compared with the method of generating images based on text (text-generated images) or generating images based on images (image-generated images) in related technologies, this application combines two inputs, text prompt words and images, which can help the image model better understand image data and task requirements, and improve the performance and effect of the model.

[0187] By applying the above-mentioned embodiment of the present application, a target text including at least one object is first displayed, and then in response to an image generation instruction for the target text, a target image associated with the target text is determined based on the object images corresponding to the objects generated by the display, and then the target image is released, and the image indication information is displayed, so that when a trigger operation for the image indication information is received, the target image associated with the target text is displayed. In this way, by identifying the objects included in the target text, determining the target image and directly releasing the target image, and then after the target image is released, the target image is displayed based on the trigger operation for the image indication information, thereby simplifying the operations in the target image generation process and improving the efficiency and convenience of the image generation process.

[0188] The following is a description of an exemplary structure of the interactive device 455 in the virtual scene provided by the present application as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the interactive device 455 in the virtual scene of the memory 450 may include:

[0189] A first display module 4551, configured to display a target text, wherein the target text includes text corresponding to at least one object;

[0190] A second display module 4552, configured to display the generated object image corresponding to each of the objects in response to the image generation instruction for the target text;

[0191] A determination module 4553, configured to determine a target image associated with the target text based on the generated object images, wherein the target image includes at least one of the objects;

[0192] A publishing module 4554 is used to publish the target image in response to a publishing instruction for the target image, and display image indication information in an associated area of ​​the target text, wherein the image indication information is used to indicate that there is an image associated with the target text;

[0193] The third display module 4555 is used to display the target image associated with the target text when a trigger operation for the image indication information is received.

[0194] In some embodiments, the device also includes a fourth display module, which is used to display a content editing area; the determination module 4553 is also used to display at least one selected target object image in the content editing area in response to a selection operation on at least one of the generated object images; based on at least one of the target object images displayed in the content editing area, in response to an image generation instruction, a generated target image associated with the target text is displayed, and the target image is generated based on the target object image in the content editing area.

[0195] In some embodiments, each of the object images is associated with an add control; the selection operation is a trigger operation for the add control associated with the object image, and the determination module 4553 is further used to display at least one of the added target object images in the content editing area in response to the trigger operations respectively performed on the add controls associated with at least one of the object images.

[0196] In some embodiments, the device also includes a first switching module, which is used to switch the add control associated with the target object image to an add completion control; wherein the add completion control is used to indicate that the target object image has been added to the content editing area; when a trigger operation for the add completion control is received, in the content editing area, the target object image is canceled from being displayed, and the add completion control is switched to the add control.

[0197] In some embodiments, the device also includes a fifth display module, which is used to display a content editing area; the determination module 4553 is also used to display at least one selected target object image in the content editing area in response to a selection operation on at least one of the generated object images; in response to an editing operation on the target object image triggered in the content editing area, display the edited target editing content; in response to an image generation instruction for the target editing content, display the generated target image associated with the target text, wherein the target image is generated based on the target editing content.

[0198] In some embodiments, the determination module 4553 is also used to display at least one operation control in response to a trigger operation on the target object image, wherein the operation control is used to perform an editing operation on the target object image, and different operation controls indicate different editing operations; in response to a trigger operation on a target operation control in the at least one operation control, a target editing operation indicated by the target operation control is performed on the target object image; when the target editing operation on the target object image is completed, the target editing content obtained by editing is displayed.

[0199] In some embodiments, the device also includes a sixth display module, which is used to display at least one drawing control, and different drawing controls correspond to different drawing tools; the determination module 4553 is also used to control the content editing area to be in a drawing mode in response to a triggering operation on a target drawing control in the at least one drawing control; in the drawing mode, in response to a drawing operation triggered by the drawing tool corresponding to the target drawing control, the target editing content obtained by drawing, including the target object image, is displayed.

[0200] In some embodiments, the device also includes a seventh display module, which is used to display at least one image style control, and the image style control is used to indicate the image style of the generated target image; in response to a selection operation of a target image style control in the at least one image style control, the target image style control is controlled to be in a selected state; the determination module 4553 is also used to display the target image with a target image style in response to an image generation instruction for each of the generated object images when the target image type control is in a selected state; wherein the target image style is the image style indicated by the target image style control.

[0201] In some embodiments, the device also includes an eighth display module, which is used to display an image style editing box for setting the image style of the target image in response to a display instruction of the image style editing box; in the image style editing box, the input text is displayed, and the text is used to indicate the target image style that the target image is expected to have; the determination module 4553 is also used to generate the target image with the target image style based on the generated object images in response to a determination instruction for the text entered in the image style editing box.

[0202] In some embodiments, the determination module 4553 is also used to display waiting prompt information in response to an image generation instruction for each of the generated object images; wherein the waiting prompt information is used to indicate that the target image is being generated; when it is determined that the generation of the target image is completed based on the waiting prompt information, the generated target image associated with the target text is displayed.

[0203] In some embodiments, the device also includes a ninth display module, and the ninth display module is used to display an information introduction page of the object corresponding to the target object image in response to a trigger operation for the target object image in each of the generated object images; wherein the information introduction page includes introduction information, and the introduction information is used to introduce the target object corresponding to the target object image.

[0204] In some embodiments, the determination module 4553 is also used to display at least two candidate images associated with the target text in response to an image generation instruction for each of the generated object images; and in response to a selection operation for a target candidate image among the at least two candidate images, determine the selected target candidate image as the target image.

[0205] In some embodiments, the device also includes a tenth display module, which is used to display a refresh control for the target image; in response to a trigger operation on the refresh control, the target image is switched to a new target image associated with the target text; wherein the new target image is different from the target image, and the new target image includes at least one of the objects.

[0206] In some embodiments, the target text belongs to a target work, and the device also includes an eleventh display module, which is used to display the text content included in the target work; in response to a selection operation on text in the text content, the selected text is displayed using a target style; wherein the target style is used to distinguish between selected text and unselected text in the text content; and the selected text is determined as the target text.

[0207] In some embodiments, the device further includes a twelfth display module, and the twelfth display module is used to display an image generation control; in response to a trigger operation on the image generation control, the image generation instruction is received.

[0208] In some embodiments, the device further includes a thirteenth display module, and the thirteenth display module is used to display a publishing control; in response to a trigger operation on the publishing control, the publishing instruction is received.

[0209] In some embodiments, the image indication information is character information, and the publishing module is further used to display the character information below the target text; the third display module is further used to display the target image associated with the target text when a trigger operation for the character information is received.

[0210] In some embodiments, the device also includes a fourteenth display module, which is used to display at least one interactive control for the target image; wherein the interactive control is used to indicate an interactive operation to be performed on the target image, and different interactive controls indicate different interactive operations; in response to a trigger operation on a target interactive control among the at least one interactive control, the target interactive operation indicated by the target interactive control is performed on the target image.

[0211] In some embodiments, the second display module 4552 is also used to display the object image corresponding to each of the objects in the image editing interface in response to an image generation instruction for the target text; the device also includes a fifteenth display module, and the fifteenth display module is used to display the text corresponding to each of the objects in the target text in the image editing interface using a first display style, and to display the text in the target text other than the text corresponding to each of the objects using a second display style; wherein the first display style is different from the second display style.

[0212] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the image generation method described in the embodiment of the present application.

[0213] The present application embodiment provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the image generation method provided by the present application embodiment, for example, Figure 3 The image generation method is shown.

[0214] In some embodiments, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a CD-ROM; or it may be various devices including one or any combination of the above memories.

[0215] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0216] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0217] As an example, the executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0218] In summary, the embodiments of the present application have the following beneficial effects:

[0219] (1) By identifying the objects included in the target text, determining the target image and directly publishing the target image, and then after publishing the target image, displaying the target image based on the trigger operation of the image indication information, thereby simplifying the operations in the target image generation process and improving the efficiency and convenience of the image generation process.

[0220] (2) Automatically generated illustrations can help readers better understand the plots and scenes in the novel and improve the reading experience. Moreover, through the innovative method of selecting objects and hand-drawing, users can create illustrations to express their ideas and creativity while reading, which increases the user's participation in the novel reading scene. At the same time, the automatic generation of illustrations saves a lot of time and cost required for manual drawing, improves the efficiency of illustration generation, and compared with pure hand-drawn illustrations, this application can quickly optimize and improve the user's hand-drawn sketches to improve the quality and aesthetics of the illustrations. In addition, this application helps users transform their ideas and creativity into more expressive and artistic illustrations. Compared with the method of generating images based on text (text-generated images) or generating images based on images (image-generated images) in related technologies, this application combines two inputs, text prompt words and images, which can help the image model better understand image data and task requirements and improve the performance and effect of the model.

[0221] It should be noted that in the embodiments of the present application, it involves obtaining user operation data and other related data. When the embodiments of the present application are applied to specific products or technologies, corresponding permissions or consents need to be obtained, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0222] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. An image generation method, characterized in that: The method comprises: Displaying a target text, wherein the target text includes text corresponding to at least one object; In response to the image generation instruction for the target text, displaying the generated object image corresponding to each of the objects; Determining a target image associated with the target text based on the generated object images, wherein the target image includes at least one of the objects; In response to a publishing instruction for the target image, the target image is published, and image indication information is displayed in an associated area of ​​the target text, wherein the image indication information is used to indicate that there is an image associated with the target text; When a trigger operation for the image indication information is received, the target image associated with the target text is displayed.

2. The method according to claim 1, characterized in that The method further comprises: Display the content editing area; The step of determining a target image associated with the target text based on each of the generated object images includes: In response to a selection operation on at least one of the generated object images, displaying the selected at least one target object image in the content editing area; Based on at least one of the target object images displayed in the content editing area, in response to an image generation instruction, a target image associated with the target text is displayed, the target image being generated based on the target object image in the content editing area.

3. The method according to claim 2, characterized in that Each of the object images is associated with an add control; The selection operation is a trigger operation for adding a control associated with the object image, and in response to the selection operation for at least one of the generated object images, displaying the selected at least one target object image in the content editing area includes: In response to trigger operations respectively performed on the adding controls associated with at least one of the object images, the added at least one target object image is displayed in the content editing area.

4. The method according to claim 3, characterized in that After displaying the added at least one target object image, the method further includes: Switching the adding control associated with the target object image to the adding completion control; Wherein, the adding completion control is used to indicate that the target object image has been added to the content editing area; When a trigger operation for the adding completion control is received, in the content editing area, the target object image is canceled from being displayed, and the adding completion control is switched to the adding control.

5. The method according to claim 1, characterized in that The method further comprises: Display content editing area; The step of determining a target image associated with the target text based on each of the generated object images includes: In response to a selection operation on at least one of the generated object images, displaying the selected at least one target object image in the content editing area; In response to an editing operation triggered in the content editing area and directed to the target object image, displaying target editing content obtained by editing; In response to an image generation instruction for the target edited content, a target image associated with the target text is displayed, the target image being generated based on the target edited content.

6. The method according to claim 5, characterized in that The step of displaying target edited content obtained by editing in response to an edit operation on the target object image triggered in the content editing area comprises: In response to a trigger operation on the target object image, displaying at least one operation control, wherein the operation control is used to perform an editing operation on the target object image, and different editing operations indicated by different operation controls; In response to a trigger operation on a target operation control among the at least one operation control, executing a target editing operation indicated by the target operation control on the target object image; When the target editing operation on the target object image is completed, the target editing content obtained by editing is displayed.

7. The method according to claim 5, characterized in that The method further comprises: Display at least one drawing control, where different drawing controls correspond to different drawing tools; The step of displaying target edited content obtained by editing in response to an edit operation on the target object image triggered in the content editing area comprises: In response to a triggering operation on a target drawing control among the at least one drawing control, controlling the content editing area to be in a drawing mode; In the drawing mode, in response to a drawing operation triggered by a drawing tool corresponding to the target drawing control, the target editing content obtained by drawing and including the target object image is displayed.

8. The method according to claim 1, characterized in that The method further comprises: Displaying at least one image style control, where the image style control is used to indicate the image style of the generated target image; In response to a selection operation on a target image style control in the at least one image style control, controlling the target image style control to be in a selected state; The step of determining a target image associated with the target text based on each of the generated object images includes: When the target image type control is in a selected state, in response to an image generation instruction for each of the generated object images, the target image having the target image style is displayed; The target image style is the image style indicated by the target image style control.

9. The method according to claim 1, characterized in that The method further comprises: In response to an instruction to display an image style editing box, displaying an image style editing box for setting an image style of the target image; In the image style editing box, the input text is displayed, and the text is used to indicate the target image style that the target image is expected to have; The step of determining a target image associated with the target text based on each of the generated object images includes: In response to a determination instruction for the text input in the image style edit box, the target image having the target image style is generated based on each of the generated object images.

10. The method according to claim 1, characterized in that The step of determining a target image associated with the target text based on each of the generated object images includes: In response to the image generation instruction for each of the generated object images, displaying a waiting prompt message; Wherein, the waiting prompt information is used to indicate that the target image is being generated; When it is determined based on the waiting prompt information that the generation of the target image is completed, the generated target image associated with the target text is displayed.

11. The method according to claim 1, characterized in that The method further comprises: In response to a trigger operation on a target object image among the generated object images, displaying an information introduction page of the object corresponding to the target object image; The information introduction page includes introduction information, and the introduction information is used to introduce the target object corresponding to the target object image.

12. The method according to claim 1, characterized in that The step of determining a target image associated with the target text based on each of the generated object images includes: In response to an image generation instruction for each of the generated object images, displaying at least two candidate images associated with the target text; In response to a selection operation on a target candidate image among the at least two candidate images, the selected target candidate image is determined as the target image.

13. The method according to claim 1, characterized in that The method further comprises: Displaying a refresh control for the target image; In response to a trigger operation on the refresh control, switching the target image to a new target image associated with the target text; The new target image is different from the target image, and the new target image includes at least one of the objects.

14. The method according to claim 1, wherein: The target text belongs to a target work, and the method further comprises: Displaying the text content included in the target work; In response to a selection operation on text in the text content, displaying the selected text in a target style; The target style is used to distinguish selected text from unselected text in the text content; The selected text is determined as the target text.

15. The method according to claim 1, wherein: The method further comprises: Display image generation controls; In response to a trigger operation on the image generation control, the image generation instruction is received.

16. The method according to claim 1, wherein: The method further comprises: Display publishing controls; In response to a trigger operation on the publishing control, the publishing instruction is received.

17. The method according to claim 1, wherein: The image indication information is character information, and the displaying of the image indication information in the associated area of ​​the target text includes: Displaying the character information below the target text; When a trigger operation for the image indication information is received, displaying the target image associated with the target text comprises: When a trigger operation for the character information is received, the target image associated with the target text is displayed.

18. The method of claim 1, wherein: After displaying the target image associated with the target text, the method further includes: displaying at least one interactive control for the target image; Wherein, the interactive control is used to instruct to perform an interactive operation on the target image, and different interactive controls indicate different interactive operations; In response to a trigger operation on a target interactive control among the at least one interactive control, a target interactive operation indicated by the target interactive control is performed on the target image.

19. The method according to claim 1, wherein: In response to the image generation instruction for the target text, displaying the object image corresponding to each of the objects comprises: In response to an image generation instruction for the target text, displaying an object image corresponding to each of the objects in an image editing interface; The method further comprises: In the image editing interface, the text corresponding to each of the objects in the target text is displayed in a first display style, and the text other than the text corresponding to each of the objects in the target text is displayed in a second display style; The first display style is different from the second display style.

20. An image generating device, characterized in that: The device comprises: A first display module, used to display a target text, wherein the target text includes a text corresponding to at least one object; A second display module, configured to display the generated object image corresponding to each of the objects in response to the image generation instruction for the target text; A determination module, configured to determine a target image associated with the target text based on the generated object images, wherein the target image includes at least one of the objects; a publishing module, configured to publish the target image in response to a publishing instruction for the target image, and display image indication information in an associated area of ​​the target text, wherein the image indication information is used to indicate that there is an image associated with the target text; The third display module is used to display the target image associated with the target text when a trigger operation for the image indication information is received.

21. An electronic device, characterized in that: include: A memory for storing executable instructions; A processor, configured to implement the image generation method according to any one of claims 1 to 19 when executing the executable instructions stored in the memory.

22. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to cause a processor to execute and implement the image generation method described in any one of claims 1 to 19.

23. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the image generation method according to any one of claims 1 to 19 is implemented.

Images