Image generation method and device, electronic equipment, storage medium and product

Abstract

The invention relates to an image generation method and device, electronic equipment, a storage medium and a product, and relates to the technical field of computers. The image generation method comprises the steps that an image generation instruction sent by a user is displayed, the image generation instruction is used for instructing generation of an illustration of a text, and the text is associated with the image generation instruction; determining a text publishing platform according to the image generation instruction; determining size information of illustrations of the text according to the publishing platform; generating prompt information according to the size information and the text; processing the prompt information by utilizing a machine learning model to generate a illustration of the text, and matching the illustration with the size information; and displaying the illustration of the text.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and particularly to an image generation method, apparatus, electronic device, storage medium, and product. Background Art

[0002] With the development of Internet technologies and Internet applications, users can publish articles on various platforms. The articles can be news blog content or life sharing. To enrich the reading experience of the articles, users will also select corresponding images and insert them into the articles. Summary of the Invention

[0003] According to some embodiments of the present disclosure, there is provided an image generation method, including: displaying an image generation instruction sent by a user, the image generation instruction being used to indicate generating an illustration for text, the text being associated with the image generation instruction; determining a publishing platform of the text according to the image generation instruction; determining size information of the illustration for the text according to the publishing platform; generating a prompt message according to the size information and the text; processing the prompt message by using a machine learning model to generate the illustration for the text, the illustration matching the size information; and displaying the illustration for the text.

[0004] According to some other embodiments of the present disclosure, there is provided an image generation apparatus, including: a first display module configured to display an image generation instruction sent by a user, the image generation instruction being used to indicate generating an illustration for text, the text being associated with the image generation instruction; a first determination module configured to determine a publishing platform of the text according to the image generation instruction; a second determination module configured to determine size information of the illustration for the text according to the publishing platform; a prompt generation module configured to generate a prompt message according to the size information and the text; an image generation module configured to process the prompt message by using a machine learning model to generate the illustration for the text, the illustration matching the size information; and a second display module configured to display the illustration for the text.

[0005] According to some embodiments of the present disclosure, there is provided an electronic device, including: a memory; and a processor coupled to the memory, the processor being configured to execute the image generation method according to any one of the embodiments of the present disclosure based on instructions stored in the memory.

[0006] According to some embodiments of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, it executes the image generation method according to any one of the embodiments of the present disclosure.

[0007] According to some embodiments of the present disclosure, there is provided a computer program product, and when the computer program product runs on a computer, it causes the computer to implement the image generation method according to any one of the embodiments of the present disclosure.

[0008] Other features, aspects, and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Embodiments of the present disclosure will be described below with reference to the accompanying drawings. It should be understood that the drawings in the following description only relate to some embodiments of the present disclosure and do not constitute a limitation on the present disclosure. In the drawings:

[0010] Figure 1 A flowchart showing a method for generating an image according to some embodiments of the present disclosure is shown.

[0011] Figure 2 A flowchart showing a method for determining size information according to some embodiments of the present disclosure is shown.

[0012] Figure 3 A flowchart showing a method for determining size information according to some other embodiments of the present disclosure is shown.

[0013] Figure 4 A flowchart showing a method for generating an image of multi-platform content according to some embodiments of the present disclosure is shown.

[0014] Figure 5 A schematic diagram of an interactive interface according to some embodiments of the present disclosure is shown.

[0015] Figure 6 A schematic diagram of the structure of an image generation device according to some embodiments of the present disclosure is shown.

[0016] Figure 7 A block diagram of an electronic device according to some embodiments of the present disclosure is shown.

[0017] Figure 8 A block diagram of an electronic device according to some other embodiments of the present disclosure is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. It should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein.

[0019] It should be understood that the steps recited in the method embodiments of the present disclosure can be executed in a different order and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard. Unless otherwise specifically stated, the relative arrangements of the components and steps set forth in these embodiments should be construed as merely exemplary and do not limit the scope of the present disclosure.

[0020] The term "comprising" and its variants used in this disclosure mean open terms that include at least the subsequent elements / features, but do not exclude other elements / features, that is, "including but not limited to". The term "based on" means "at least partially based on".

[0021] It should be noted that the concepts such as "first", "second", etc. mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependent relationships. Unless otherwise specified, the concepts such as "first", "second", etc. are not intended to imply that the objects so described must be in a given order in terms of time, space, ranking or any other way.

[0022] It should be noted that the modifications of "one" and "multiple" mentioned in this disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless clearly specified otherwise in the context, it should be understood as "one or more".

[0023] The names of the messages or information exchanged between multiple devices in the embodiments of this disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0024] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. And the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0025] The embodiments of this disclosure will be described in detail below with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. In addition, in one or more embodiments, specific features, structures or characteristics can be combined in any suitable manner that will be clear to those of ordinary skill in the art from this disclosure.

[0026] The authors of some articles are not good at drawing pictures. Therefore, some extra effort is needed to find pictures. Moreover, different publishing platforms have different requirements for the size of images, and users may also need to modify the images to meet the requirements of the platforms. This makes users need to spend extra time processing images during the process of creating and publishing articles, reducing the publishing efficiency.

[0027] Embodiments of the present disclosure provide an image generation method for generating an accompanying image for text to be published by a user. The size of the generated accompanying image can match the size requirements of the publishing platform, thereby efficiently helping the user to generate an accompanying image for the text.

[0028] Figure 1 FIG. shows a schematic flowchart of an image generation method according to some embodiments of the present disclosure. As Figure 1 shown, the image generation method of this embodiment includes steps S11 to S16.

[0029] In step S11, an image generation instruction sent by the user is displayed. The image generation instruction is used to indicate the generation of an accompanying image for the text, and the text is associated with the image generation instruction.

[0030] The user can input the image generation instruction through various types of input controls such as a text input control and a voice input control. Alternatively, the user can also trigger a control in the interaction interface for sending the image generation instruction. The image generation instruction can be sent in the interaction interface between the user and the agent. Of course, it can also be sent in other types of interfaces, and the present disclosure does not limit this.

[0031] The text and the image generation instruction can be sent together. For example, the user can input the text and the image generation instruction at one time through the input control. Alternatively, the text can be attached to the input control in the form of a document. After the user inputs the image generation instruction and triggers the sending function, the document and the generation instruction are sent to the module for generating the image together, for example, sent to the server.

[0032] In a scenario where the user and the agent interact in the form of sending messages, the text and the image generation instruction can also be distributed in different messages. For example, the user can first send the text and then send an image generation instruction such as "I want to publish an article on platform A and generate an accompanying image for the above text". Another example is that the text can be generated by the agent according to the user's previous instructions, and the user's image generation instruction refers to the text, or the user sends an image generation instruction such as "generate an accompanying image for the above text".

[0033] The content of the text can be directly presented in the interaction interface, for example, it is a message sent by the user or the agent; the content of the text can also be carried in a document, for example, it is a document uploaded by the user or created by the agent; the content of the text can also be located on other pages different from the current page, and the user can provide a link to the text so as to obtain the content of the text by accessing the link before processing the text.

[0034] In step S12, according to the image generation instruction, the publishing platform of the text is determined.

[0035] The information of the publishing platform can be obtained by means of semantic analysis or keyword matching. For example, the image generation instruction is input into a machine learning model with natural language understanding ability, and at the same time, the model is instructed to judge the publishing platform carried by the image generation instruction. Then, the information of the publishing platform returned by the model can be obtained. The model can be pre-trained with image generation instructions marked with the publishing platform for training. The image generation instructions for training may directly include the information of the publishing platform or may not include the information of the publishing platform.

[0036] The information of the publishing platform of the text may be directly included in the image generation instruction. For example, in the image generation instruction "I want to post an article on Platform A and match pictures for the above text", the publishing platform can be clearly determined as Platform A.

[0037] The information of the publishing platform may not be explicitly included in the image generation instruction. For example, the image generation instruction is "Match pictures for the article in www.aaa.bb". In this case, the publishing platform can be determined according to the website or application corresponding to the link in the image generation instruction. Another example is that the image generation instruction is "Match pictures for the above movie review. I want to post a note", then the publishing platform that may support posting movie reviews and notes can be determined according to the keywords "movie review" and "note" therein.

[0038] Several ways to determine the publishing platform are as follows. In response to the image generation instruction including the description information of the publishing platform, the publishing platform of the text is determined according to the description information. The description information can directly indicate the publishing platform and can be determined by means of keyword matching. Or, in response to the image generation instruction being associated with a link, the publishing platform of the text is determined according to the link, such as understanding the domain name information contained in the link itself or by accessing the link. Or, semantic understanding of the text is performed to determine the type of the text, and the publishing platform of the text is determined according to the type of the text. The process of semantic understanding can be implemented by using a machine learning model.

[0039] In step S13, according to the publishing platform, the size information of the pictures matching the text is determined.

[0040] For example, for each of multiple publishing platforms, the size requirements for the pictures can be pre-configured. The size requirements may be the aspect ratio information of the image, the image file size information (i.e., the space occupied on the storage device), the resolution information, or the pixel number information in the length or width direction. Each of the above information can be one or more specific values or one or more value ranges.

[0041] By querying the size requirements of the publishing platform, the size information of the text pictures can be determined. The determined size information needs to meet the size requirements of the platform.

[0042] In step S14, according to the size information and the text, a prompt message is generated.

[0043] The prompt message includes the size information to precisely constrain the size of the image to be generated. Part or all of the original text can be directly included in the prompt message to make the generated image have a higher matching degree with the text. Or, after processing the text, such as extracting the main information, theme, type, keywords, etc. of the text, the processed content can be included in the prompt message. In this way, the content volume of the prompt message can be constrained, and the processing pressure during the image generation process can be reduced.

[0044] In step S15, a machine learning model is used to process the prompt message to generate an illustration for the text, and the illustration matches the size information.

[0045] The machine learning model is, for example, an image generation model. This model can be a model for generating images based on text, such as a base model, a large language model, etc. The prompt message can be directly input into the machine learning model, or, after processing the prompt message (such as format conversion, natural language to vector, etc.), it can be input into the machine learning model.

[0046] An example of the prompt message is "Generate an image based on the following content:.... The ratio of the generated image is..., and the length does not exceed...". Thus, the image output by the machine learning model can match the content of the text, and the size of the image meets the requirements in the prompt message.

[0047] This method can directly generate an image whose size meets the platform requirements, that is, directly generate an image with a suitable size from the prompt message, without the user having to modify the size again, nor do other models or business logics need to modify the size. In this way, on the one hand, the operation efficiency of the user can be improved; on the other hand, directly generating an image that meets the size requirements can also make the elements in the generated image, that is, the elements that match the description of the text, be completely presented in the final image, without losing information due to secondary modification, nor affecting the aesthetic feeling and presentation effect of the image.

[0048] In step S16, the illustration of the text is displayed.

[0049] The generated illustration of the text can include one or more. The generated image can be presented to the user separately without accompanying text. For example, the agent can send a response message to the user, and the response message includes the generated image. Or, the generated image can be inserted into the text, and the text with the inserted illustration is presented to the user so that the user can directly view the effect of the text after viewing the illustration.

[0050] Based on the image generation instruction, the above embodiments automatically determine the publishing platform of the text, and then generate an image that matches the text content and meets the platform requirements for the text, thereby being able to efficiently assist users in automatically matching pictures for the text and improving the user's creation and text publishing efficiency.

[0051] The following refers to Figure 2 and Figure 3 to exemplarily describe several embodiments for determining the size information of the pictures matching the text.

[0052] Figure 2 FIG. shows a flowchart of a method for determining size information according to some embodiments of the present disclosure. As Figure 2 shown, the method for determining size information in this embodiment includes steps S1311 to S1314.

[0053] In step S1311, query the size information of the pictures corresponding to the publishing platform of the text.

[0054] For example, a correspondence table, or a correspondence file, or a correspondence database between the publishing platform and the picture size information can be established in advance.

[0055] In response to being able to query the size information of the pictures corresponding to the text publishing platform, determine the queried size information as the size information of the pictures matching the text.

[0056] In step S1312, in response to not querying the size information of the pictures corresponding to the text publishing platform, determine the type of the text according to at least one of the publishing platform and the content of the text.

[0057] Some publishing platforms belong to the blog category, some belong to the news category, some belong to the life sharing category, and so on. The types of texts in the same type of publishing platforms are relatively close, so the type of the text can be determined based on the publishing platform.

[0058] The key information such as keywords, themes, and abstracts in the content of the text itself can also reflect the type of the text. The type of the text can be determined according to the preset correspondence table between the key information and the type, and the key information of the content of this text. Or, a natural language processing model can also be used to process the text to obtain the classification result of the text.

[0059] When determining the type of the text, prompt information including at least one of the information of the publishing platform (such as name, introduction), the text or the key information of the text can also be constructed, and a machine learning model is used to process the prompt information to determine the type of the text.

[0060] In step S1313, according to the type of the text, determine the reference publishing platform matching the text.

[0061] According to the type of the text, the matching degree between the text and each reference publishing platform can be determined, and this matching degree reflects the probability that the text will be published on the reference publishing platform. The reference publishing platform that matches the text can be the reference publishing platform with the highest matching degree.

[0062] The reference publishing platform can be a platform that already has the size information of the corresponding illustration.

[0063] In step S1314, according to the size information of the illustration corresponding to the reference publishing platform, the size information of the illustration of the text is determined.

[0064] For example, the size information of the illustration corresponding to the reference publishing platform can be directly determined as the size information of the illustration of the text. Or, on the basis of the size information of the illustration corresponding to the reference publishing platform, fine-tuning can also be performed to obtain the size information of the illustration of the text.

[0065] In the above embodiments, when the size information of the illustration of the publishing platform of the text cannot be directly obtained, the reference publishing platform with the requirement of the size information of the illustration is determined according to the type of the text. Thus, in the case of the emergence of a new platform, images can also be efficiently generated for users, and the size of the images can meet the requirements of the publishing platform with a high probability.

[0066] Considering that some texts may need to be illustrated at multiple places, and the size requirements of the illustrations at different positions may be different, some embodiments of the present disclosure can more accurately generate images that meet the requirements according to the position of the illustration.

[0067] Figure 3 A flowchart showing a method for determining size information according to other embodiments of the present disclosure is shown. As Figure 3 shown, the determination method of this embodiment includes steps S1321 to S1323.

[0068] In step S1321, semantic understanding of the text is performed to determine the target content in the text.

[0069] The target content in the text refers to the content that needs to be illustrated at the position where the text is located (such as before, in, or after the text). The target content can be part of the text or the whole text.

[0070] For example, text can be input into a machine learning model, which can identify and output the target content. The target content can be parts with a high degree of importance, or parts that are likely to cause reading fatigue during the reading process by the reader, etc. The machine learning model can be pre-trained with training data being text marked with the positions of the accompanying images. Through iterative training, the machine learning model can identify the positions in the text where accompanying images are needed, i.e., the target content.

[0071] In step S1322, determine the type of the accompanying image of the text based on at least one of the position of the target content in the text and the type of the target content.

[0072] Some publishing platforms have different requirements for accompanying images at different positions in the text. For example, the size requirements for the front cover or the back cover image are more stringent and need to conform to a specified aspect ratio, while the size requirements for the illustrations are relatively loose. And the type of the target content can determine the richness of the content of the accompanying image for it, and thus determine the size of the image.

[0073] An example of determining the type of the accompanying image based on the position of the target content in the text is as follows. In response to the target content being the full text of the text, determine that the type of the accompanying image of the text is at least one of the front cover image and the back cover image; in response to the target content being some paragraphs in the text, determine that the type of the accompanying image of the text is an illustration. The front cover image is the image located after the title and before the full text, or it can also be the image located before the title. The back cover image is the image located after the full text. The illustration is the image other than the front cover image and the back cover image, that is, the image located between the texts.

[0074] In some embodiments, in response to the type of the target content being visual or an opinion, determine that the type of the accompanying image of the text is a descriptive image; in response to the type of the target content being a transition or a turn, determine that the type of the accompanying image of the text is a decorative image, where the size of the descriptive image is larger than that of the decorative image. The descriptive image is an image that includes the elements in the target content. For example, if the target content mentions a description of a scene, the descriptive image can reflect the scene. The decorative image is an image that may not include practical meaning, and the degree of overlap between its content and the text content is relatively limited, mainly serving for aesthetics to relieve the user's reading fatigue, and the size occupied by such an image is smaller than that of the descriptive image.

[0075] In step S1323, determine the size information corresponding to this type from the size information of the images corresponding to the publishing platform as the size information of the accompanying image of the text.

[0076] The size information of the images corresponding to the publishing platform can include one or more types. For example, for a certain publishing platform, the size information of the accompanying images of its various types can be pre-stored.

[0077] Through the above embodiments, when generating text illustrations, the target content that needs to be illustrated can be determined first, and then based on the position or type of the target content, the type of the image that matches the illustration position can be determined. Thus, the size of the image can be determined more accurately, making the generated illustration better integrated with the text, saving the user's operations, and improving the efficiency of content creation.

[0078] In addition, if the text comes from graphic content, the existing images in the graphic content can be used as a reference to determine the size information of the text illustration to be generated. In some embodiments, determining the size information of the text illustration according to the publishing platform includes: determining the size information of the text illustration according to the publishing platform and the size of the images in the graphic content, and the difference between the size of the images in the graphic content and the size information of the text illustration is within a specified range. The specified range can be set in advance. In this way, the generated text illustration can be close to the size of the existing images in the graphic content. On the one hand, it can meet the image size requirements of the platform, and on the other hand, it can make the generated image consistent with the existing images as a whole, with stronger usability and improved content creation efficiency.

[0079] With the booming development of Internet applications, some users maintain accounts on multiple platforms, and after creating a piece of content, they will publish it synchronously on multiple platforms. The embodiments of the present disclosure support generating illustrations for the published content of the same text on multiple platforms at one time.

[0080] Figure 4 The flowchart shows a method for generating images of multi-platform content according to some embodiments of the present disclosure. As Figure 4 shown, the image generation method of this embodiment includes steps S41 to S46.

[0081] In step S41, an image generation instruction sent by the user is displayed, and the image generation instruction is used to indicate generating illustrations for the text on multiple platforms, and the text is associated with the image generation instruction.

[0082] In step S42, according to the image generation instruction, multiple publishing platforms of the text are determined.

[0083] In step S43, for each of the multiple publishing platforms, according to the publishing platform, the size information of the illustration of the text in each platform is determined.

[0084] In step S44, according to the size information of the illustration of the text in each platform and the text, prompt information corresponding to each platform is generated.

[0085] In step S45, each prompt message is processed using a machine learning model to generate illustrations for the text on multiple platforms, and the illustration for each platform matches the size information corresponding to that platform.

[0086] In some embodiments, step S44 can be implemented in the following manner: according to the text, a first prompt message is generated; and, according to the first prompt message and the size information of the illustrations for each platform, second prompt messages corresponding to each platform are respectively generated. Step S45 can be implemented in the following manner: the second prompt messages corresponding to each platform are respectively processed using a machine learning model to generate illustrations of the text corresponding to each platform.

[0087] That is, it can be understood that the first prompt message corresponds to the content of the text, and it is used to describe the content in the image to be generated. The first prompt message can correspond to each platform. Based on the same first prompt message, different second prompt messages are generated through the size information of the illustrations corresponding to each platform. The second prompt message corresponding to each platform can include the size information of the illustration corresponding to that platform.

[0088] In this way, before generating the illustrations, the sizes of the illustrations for each platform can be restricted by the second prompt message, so that the images generated by the machine learning model can directly meet the requirements of each platform.

[0089] In step S46, the illustrations of the text on multiple platforms are displayed.

[0090] Through the above embodiments, illustrations can be generated at one time for the content to be published of the same text on multiple platforms, thereby efficiently assisting users in multi-platform creation. And, compared with the method of first generating the illustration of the text and then modifying its size to meet the requirements of multiple platforms, the above embodiments of the present disclosure do not require secondary modification and have higher efficiency. Also, information loss will not occur due to the size modification process, improving the quality of content creation.

[0091] Next, taking the text being in a link as an example, an application scenario of the embodiments of the present disclosure is described.

[0092] In some embodiments, the image generation instruction is in the message sent by the user, and the message further includes a link to the text; the image generation method further includes: based on the link, obtaining the content corresponding to the link; and, obtaining the text from the content corresponding to the link; the displaying the illustration of the text includes: displaying the text including the illustration on a preview interface.

[0093] Figure 5 Shows a schematic diagram of an interaction interface according to some embodiments of the present disclosure. As Figure 5As shown, the interaction interface 5 of this embodiment includes a dialogue area 51 and a preview area 52. In the dialogue area 51, the user sends a message 511 to the intelligent agent: "Help me generate illustrations for the article in www.aa.bb". Based on the link in the message 511, access the link and obtain the text in the link. For example, send an HTTP request to the link, obtain the returned data, and based on the page structure information in the returned data, obtain the text; or, open the link to display the interface of the web page, and extract the content of the web page from the image by means of screenshot and image recognition. Determine the platform corresponding to the link and the requirements of the platform for the size information of the illustrations. Then, according to the text and the requirements, generate a prompt message, and then generate illustrations for the text based on the prompt message.

[0094] After generating the illustrations, the effect of the text with the inserted illustrations can be displayed in the preview area 52, so as to facilitate the user to confirm whether to make further modifications. For example, Image 1 is inserted as the cover image after the title and before the first paragraph of the article, as shown in 521; Image 2 is inserted as an illustration between two paragraphs of the text, as shown in 522.

[0095] Optionally, the intelligent agent can respond to the user by sending a message 512 in the dialogue area 51: "Okay, the cover image and illustrations for the article on Platform A have been generated for you". In addition, Image 1 can be provided through message 513 and Image 2 can be provided through message 514 in the dialogue interface.

[0096] Furthermore, the user can send an instruction through the input control 515 to adjust the generated image.

[0097] Through the method of this embodiment, it is possible to support generating illustrations for the content in an external link, so that the user can efficiently preview the effect after inserting the illustrations, improving the user's operation efficiency.

[0098] The methods of the various embodiments of the present disclosure have been introduced above. Next, a device for executing the methods of the above various embodiments will be described.

[0099] Figure 6 The structural schematic diagram of an image generation device according to some embodiments of the present disclosure is shown. As Figure 6As shown in the figure, the image generation device 6 of this embodiment includes: a first display module 61 configured to display an image generation instruction sent by a user, where the image generation instruction is used to indicate generating an illustration for text, and the text is associated with the image generation instruction; a first determination module 62 configured to determine the publishing platform of the text according to the image generation instruction; a second determination module 63 configured to determine the size information of the illustration for the text according to the publishing platform; a prompt generation module 64 configured to generate a prompt message according to the size information and the text; an image generation module 65 configured to process the prompt message by using a machine learning model to generate an illustration for the text, and the illustration matches the size information; a second display module 66 configured to display the illustration for the text.

[0100] Based on the image generation instruction, the above-mentioned embodiment automatically determines the publishing platform of the text, and then generates an image that matches the text content and meets the platform requirements for the text, so as to efficiently assist the user in automatically matching an illustration for the text and improve the user's creation and text publishing efficiency.

[0101] In some embodiments, the second determination module 63 is further configured to: query the size information of the illustration corresponding to the publishing platform of the text; in response to not querying the size information of the illustration corresponding to the text publishing platform, determine the type of the text according to at least one of the publishing platform and the content of the text; determine a reference publishing platform that matches the text according to the type of the text; determine the size information of the illustration for the text according to the size information of the illustration corresponding to the reference publishing platform.

[0102] In some embodiments, the second determination module 63 is further configured to: perform semantic understanding on the text to determine the target content in the text; determine the type of the illustration for the text according to at least one of the position of the target content in the text and the type of the target content; determine the size information corresponding to the type from the size information of the images corresponding to the publishing platform as the size information of the illustration for the text.

[0103] In some embodiments, the second determination module 63 is further configured to: in response to the target content being the full text of the text, determine that the type of the illustration for the text is at least one of a head image and a tail image; in response to the target content being some paragraphs in the text, determine that the type of the illustration for the text is an illustration.

[0104] In some embodiments, the second determination module 63 is further configured to: in response to the type of the target content being vision or view, determine that the type of the illustration for the text is a descriptive image; in response to the type of the target content being transition or turning point, determine that the type of the illustration for the text is a decorative image, where the size of the descriptive image is larger than the size of the decorative image.

[0105] In some embodiments, the text is from graphic content, and the second determination module 63 is further configured to: determine the size information of the illustration for the text according to the publishing platform and the size of the image in the graphic content, and the difference between the size of the image in the graphic content and the size information of the illustration for the text is within a specified range.

[0106] In some embodiments, there are multiple publishing platforms for the text, and the second determination module 63 is further configured to: for each of the multiple publishing platforms, determine the size information of the illustration for the text in each platform according to the publishing platform.

[0107] In some embodiments, the prompt generation module 64 is further configured to: generate a first prompt message according to the text; and generate second prompt messages corresponding to each platform respectively according to the first prompt message and the size information of the illustration for each platform; the image generation module 65 is further configured to: use a machine learning model to process the second prompt messages corresponding to each platform respectively to generate the illustrations for the text corresponding to each platform.

[0108] In some embodiments, the image generation instruction is in the message sent by the user, and the message further includes a link to the text; the image generation device further includes a text acquisition module 66, which is configured to: obtain the content corresponding to the link based on the link; and acquire the text from the content corresponding to the link; the second display module 66 is further configured to: display the text including the illustration on the preview interface.

[0109] In some embodiments, the first determination module 62 is further configured to: in response to the image generation instruction including the description information of the publishing platform, determine the publishing platform of the text according to the description information; or, in response to the image generation instruction being associated with the link, determine the publishing platform of the text according to the link; or, perform semantic understanding on the text to determine the type of the text, and determine the publishing platform of the text according to the type of the text.

[0110] Figure 7 The block diagram of an electronic device according to some embodiments of the present disclosure is shown.

[0111] The memory 71 is used to store one or more computer-readable instructions. The memory 71 may include any combination of various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory, including but not limited to random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), flash memory. The memory 71 may store, for example, an operating system, application programs, a boot loader (BootLoader), a database, and other programs, and may also store various application programs and various data, etc.

[0112] The processor 72 is used to run computer-readable instructions to implement the method described in any of the foregoing embodiments. For the specific implementation of each step of the method, reference may be made to the foregoing embodiments, and repeated parts will not be elaborated here.

[0113] The processor 72 may be configured to execute the steps of the foregoing embodiments. The processor 72 may be embodied as various processing devices, such as a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The central processing unit (CPU) may be of the X76 or ARM architecture, etc.

[0114] The processor 72 and the memory 71 may communicate with each other directly or indirectly. For example, the processor 72 and the memory 71 may communicate through a network. The network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network. The processor 72 and the memory 71 may also communicate with each other through a system bus, and the present disclosure does not limit this.

[0115] It should be noted that Figure 7 The components of the electronic device 7 shown are exemplary and not restrictive. According to actual application needs, the electronic device 7 may also have other components. The processor 72 may control other components in the electronic device 7 to perform desired functions.

[0116] The electronic device 7 may be implemented in a software, firmware, and / or hardware manner and may be integrated in a device installed with relevant application programs.

[0117] Based on the image generation indication, the foregoing embodiments automatically determine the publishing platform of the text, and then generate an image that matches the text content and meets the platform requirements for the text, thereby being able to efficiently assist the user in automatically matching pictures for the text and improving the user's creation and text publishing efficiency.

[0118] Figure 8 A block diagram of an electronic device according to some other embodiments of the present disclosure is shown.

[0119] Figure 8 The electronic device 8 shown may be a computer system with a dedicated hardware structure and can perform corresponding functions when installed with relevant application programs.

[0120] The electronic device includes, but is not limited to, mobile terminals such as smart phones, laptop computers, personal digital assistants (PDAs), tablet personal computers (Tablet PCs), portable multimedia players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable devices, etc., and fixed terminals such as digital TVs, desktop computers, etc.

[0121] As Figure 8 shown, the central processing unit (CPU) 81 executes various processes according to the programs stored in the read-only memory (ROM) 82 or the programs loaded from the storage section 88 into the random access memory (RAM) 83. In the RAM 83, data required when the CPU 81 executes various processes, etc. is stored as needed. The central processing unit is merely exemplary and can also be other types of processors, such as the various processors described above. The ROM 82, RAM 83, and storage section 88 can be various forms of computer-readable storage media. It should be noted that although Figure 8 the ROM 82, RAM 83, and storage section 88 are shown separately, one or more of them can be combined or located in the same or different memories or storage modules.

[0122] The CPU 81, ROM 82, and RAM 83 are connected to each other via a bus 84. The input / output interface 85 is also connected to the bus 84.

[0123] The following components are connected to the input / output interface 85: an input section 86, such as a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; an output section 87, including a display, such as a cathode ray tube (CRT), liquid crystal display (LCD), speaker, vibrator, etc.; a storage section 88, including a hard disk, magnetic tape, etc.; and a communication section 89, including a network interface card such as a LAN card, modem, etc. The communication section 89 allows communication processing to be performed via a network such as the Internet. It is easy to understand that although Figure 8 some parts in the electronic device 8 are shown to communicate via the bus 84, they can also communicate via a network or other means, where the network can include a wireless network, a wired network, and / or any combination of a wireless network and a wired network.

[0124] As needed, a drive 810 is also connected to the input / output interface 85. A removable medium 811, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is mounted on the drive 810 as needed, so that the computer program read therefrom is installed into the storage section 88 as needed.

[0125] In the case where the above series of processes are implemented by software, a program constituting the software can be installed from a network such as the Internet or a storage medium such as a removable medium 811.

[0126] According to an embodiment of the present disclosure, the processes described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product that, when run on a computer, causes the computer to implement the method described in any of the foregoing embodiments. The computer program product includes computer instructions carried on a computer-readable medium and includes program code for executing the method shown in the flowchart. In such an embodiment, the computer instructions can be downloaded and installed from a network through the communication section 89, or installed from the storage section 88, or installed from the ROM 82. When the computer program is executed by the CPU 81, the method of the embodiment of the present disclosure is executed.

[0127] It should be noted that, in the context of the present disclosure, a computer-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0128] A computer-readable medium can be a computer-readable storage medium, a computer-readable signal medium, or any combination of the two.

[0129] Computer-readable storage media include, but are not limited to: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. Computer instructions are stored on the computer-readable storage medium, and when the instructions are executed by a processor, the method described in any of the foregoing embodiments is implemented.

[0130] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable signal medium may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

[0131] Based on the automatically determined text publishing platform indicated by the image generation, the foregoing embodiments can generate an image that matches the text content and meets the platform requirements for the text, thereby efficiently assisting the user in automatically matching pictures for the text and improving the user's creation and text publishing efficiency.

[0132] The foregoing computer-readable medium may be included in the foregoing electronic device; or may exist separately without being assembled into the electronic device.

[0133] In some embodiments, a computer program is also provided, including: instructions that, when executed by a processor, cause the processor to execute the method described in any of the foregoing embodiments. For example, the instructions may be embodied as computer program code.

[0134] In embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The foregoing programming languages include but are not limited to object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network (including a local area network (LAN) or a wide area network (WAN)), or may be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0136] The functions described above can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chip (SOCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0137] According to some embodiments of the present disclosure, there is provided an image generation method, including: displaying an image generation instruction sent by a user, the image generation instruction being used to indicate generating an illustration for text, and the text being associated with the image generation instruction; determining a publishing platform of the text according to the image generation instruction; determining size information of the illustration for the text according to the publishing platform; generating a prompt message according to the size information and the text; processing the prompt message by using a machine learning model to generate an illustration for the text, the illustration matching the size information; and displaying the illustration for the text.

[0138] In some embodiments, determining size information of the illustration for the text according to the publishing platform includes: querying size information of the illustration corresponding to the publishing platform of the text; in response to not querying size information of the illustration corresponding to the publishing platform of the text, determining a type of the text according to at least one of the publishing platform and the content of the text; determining a reference publishing platform matching the text according to the type of the text; and determining size information of the illustration for the text according to the size information of the illustration corresponding to the reference publishing platform.

[0139] In some embodiments, determining the size information of the accompanying image for the text according to the publishing platform includes: performing semantic understanding on the text to determine the target content in the text; determining the type of the accompanying image for the text according to at least one of the position of the target content in the text and the type of the target content; and determining the size information corresponding to the type from the size information of the images corresponding to the publishing platform as the size information of the accompanying image for the text.

[0140] In some embodiments, determining the type of the accompanying image for the text according to at least one of the position of the target content in the text and the type of the target content includes: in response to the target content being the full text of the text, determining that the type of the accompanying image for the text is at least one of a leading image and a trailing image; and in response to the target content being a partial paragraph in the text, determining that the type of the accompanying image for the text is an illustration.

[0141] In some embodiments, determining the type of the accompanying image for the text according to at least one of the position of the target content in the text and the type of the target content includes: in response to the type of the target content being a vision or an opinion, determining that the type of the accompanying image for the text is a descriptive image; and in response to the type of the target content being a transition or a turn, determining that the type of the accompanying image for the text is a decorative image, wherein the size of the descriptive image is larger than the size of the decorative image.

[0142] In some embodiments, the text is from graphic content. Determining the size information of the accompanying image for the text according to the publishing platform includes: determining the size information of the accompanying image for the text according to the publishing platform and the size of the image in the graphic content, and the difference between the size of the image in the graphic content and the size information of the accompanying image for the text is within a specified range.

[0143] In some embodiments, there are multiple publishing platforms for the text. Determining the size information of the accompanying image for the text according to the publishing platform includes: for each of the multiple publishing platforms, determining the size information of the accompanying image for the text in each platform according to the publishing platform.

[0144] In some embodiments, generating a prompt message according to the size information and the text includes: generating a first prompt message according to the text; and respectively generating a second prompt message corresponding to each platform according to the first prompt message and the size information of the accompanying image for each platform; using a machine learning model to process the prompt message to generate the accompanying image for the text includes: using the machine learning model to process the second prompt message corresponding to each platform respectively to generate the accompanying image for the text corresponding to each platform.

[0145] In some embodiments, an image generation instruction is located in a message sent by a user, and the message further includes a link to text; the image generation method further includes: obtaining content corresponding to the link based on the link; and obtaining the text from the content corresponding to the link; displaying an illustration of the text includes: displaying the text including the illustration on a preview interface.

[0146] In some embodiments, determining the publishing platform of the text according to the image generation instruction includes: in response to the image generation instruction including description information of the publishing platform, determining the publishing platform of the text according to the description information; or, in response to the image generation instruction being associated with a link, determining the publishing platform of the text according to the link; or, performing semantic understanding on the text to determine the type of the text, and determining the publishing platform of the text according to the type of the text.

[0147] According to some embodiments of the present disclosure, an image generation device is provided, including: a first display module configured to display an image generation instruction sent by a user, the image generation instruction being used to indicate generating an illustration for text, and the text being associated with the image generation instruction; a first determination module configured to determine the publishing platform of the text according to the image generation instruction; a second determination module configured to determine the size information of the illustration of the text according to the publishing platform; a prompt generation module configured to generate a prompt information according to the size information and the text; an image generation module configured to process the prompt information by using a machine learning model to generate an illustration of the text, and the illustration matching the size information; a second display module configured to display the illustration of the text.

[0148] According to some embodiments of the present disclosure, an electronic device is provided, including: a memory; and a processor coupled to the memory, the processor being configured to execute the image generation method according to any embodiment of the present disclosure based on instructions stored in the memory.

[0149] According to some embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the image generation method according to any embodiment of the present disclosure is implemented.

[0150] According to some embodiments of the present disclosure, a computer program product is provided, and when the computer program product runs on a computer, the computer is enabled to implement the image generation method according to any embodiment of the present disclosure.

[0151] Although some specific embodiments of the present disclosure have been described in detail by way of examples, those skilled in the art should understand that the above examples are only for illustration and not for limiting the scope of the present disclosure. Those skilled in the art should understand that the above embodiments can be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. An image generation method, comprising: Displaying an image generation instruction sent by a user, where the image generation instruction is used to indicate generating an illustration for text, and the text is associated with the image generation instruction; Determining a publishing platform of the text according to the image generation instruction; Determining size information of the illustration for the text according to the publishing platform; Generating a prompt message according to the size information and the text; Processing the prompt message by using a machine learning model to generate the illustration for the text, and the illustration matches the size information; Displaying the illustration for the text.

2. The image generation method according to claim 1, wherein, The determining the size information of the illustration for the text according to the publishing platform includes: Querying the size information of the illustration corresponding to the publishing platform of the text; In response to not querying the size information of the illustration corresponding to the publishing platform of the text, determining the type of the text according to at least one of the publishing platform and the content of the text; Determining a reference publishing platform matching the text according to the type of the text; Determining the size information of the illustration for the text according to the size information of the illustration corresponding to the reference publishing platform.

3. The image generation method according to claim 1, wherein, The determining the size information of the illustration for the text according to the publishing platform includes: Performing semantic understanding on the text to determine target content in the text; Determining the type of the illustration for the text according to at least one of the position of the target content in the text and the type of the target content; Determining the size information corresponding to the type from the size information of the images corresponding to the publishing platform as the size information of the illustration for the text.

4. The image generation method according to claim 3, wherein, The determining the type of the illustration for the text according to at least one of the position of the target content in the text and the type of the target content includes: In response to the target content being the full text of the text, determining that the type of the illustration for the text is at least one of a front cover image and a back cover image; In response to the target content being some paragraphs in the text, determining that the type of the illustration for the text is an illustration.

5. The image generation method according to claim 3, wherein, The determining the type of the illustration for the text according to at least one of the position of the target content in the text and the type of the target content includes: In response to the type of the target content being visual or an opinion, determining that the type of the illustration for the text is a descriptive image; In response to the type of the target content being a transition or a turning point, determining that the type of the illustration for the text is a decorative image, where the size of the descriptive image is larger than the size of the decorative image.

6. The image generation method according to claim 1, wherein, The text comes from graphic content, and the determining the size information of the illustration for the text according to the publishing platform includes: Determining the size information of the illustration for the text according to the publishing platform and the size of the image in the graphic content, and the difference between the size of the image in the graphic content and the size information of the illustration for the text is within a specified range.

7. The image generation method according to claim 1, wherein, There are multiple publishing platforms of the text, and the determining the size information of the illustration for the text according to the publishing platform includes: For each of the multiple publishing platforms, determine, according to the publishing platform, the size information of the illustration for the text in each platform.

8. The image generation method according to claim 7, wherein: The generating the prompt information according to the size information and the text includes: generating first prompt information according to the text; and generating second prompt information corresponding to each platform respectively according to the first prompt information and the size information of the illustration for each platform; The using the machine learning model to process the prompt information to generate the illustration for the text includes: using the machine learning model to process the second prompt information corresponding to each platform respectively to generate the illustration for the text corresponding to each platform.

9. The image generation method according to any one of claims 1 to 8, wherein: The image generation instruction is located in the message sent by the user, and the message further includes a link to the text; The image generation method further includes: obtaining the content corresponding to the link based on the link; and obtaining the text from the content corresponding to the link; The displaying the illustration for the text includes: displaying the text including the illustration on a preview interface.

10. The image generation method according to any one of claims 1 to 8, wherein, The determining the publishing platform of the text according to the image generation instruction includes: In response to the image generation instruction including the description information of the publishing platform, determining the publishing platform of the text according to the description information; or, In response to the image generation instruction being associated with a link, determining the publishing platform of the text according to the link; or, Performing semantic understanding on the text to determine the type of the text, and determining the publishing platform of the text according to the type of the text.

11. An image generation device, comprising: A first display module configured to display an image generation instruction sent by a user, the image generation instruction being used to indicate generating an illustration for a text, and the text being associated with the image generation instruction; A first determination module configured to determine the publishing platform of the text according to the image generation instruction; A second determination module configured to determine the size information of the illustration for the text according to the publishing platform; A prompt generation module configured to generate prompt information according to the size information and the text; An image generation module configured to use a machine learning model to process the prompt information to generate the illustration for the text, and the illustration matching the size information; A second display module configured to display the illustration for the text.

12. An electronic device, comprising: A memory; And A processor coupled to the memory, the processor being configured to execute the image generation method according to any one of claims 1 to 10 based on instructions stored in the memory.

13. A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the image generation method according to any one of claims 1 to 10 is implemented.

14. A computer program product, when the computer program product runs on a computer, causes the computer to implement the image generation method according to any one of claims 1 to 10.

Images