Poster generation methods, devices, electronic equipment and storage media

Abstract

This disclosure provides a poster generation method, apparatus, electronic device, and storage medium, relating to the field of graphics processing technology, and particularly to the fields of artificial intelligence and computer vision technology. The specific implementation scheme includes: acquiring poster material images and poster description information; determining feature vector representations and a mask image of the poster material images based on the poster material images and poster description information; determining poster layout information; generating candidate posters based on the poster layout information and feature vector representations; and combining the candidate posters according to the poster material images and corresponding mask images to generate the target poster, achieving more efficient, aesthetically pleasing, and intelligent poster design.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, specifically to the fields of artificial intelligence and computer vision technology, and particularly to a poster generation method, apparatus, electronic device, and storage medium. Background Technology

[0002] Current poster generation methods mainly include template-based automated layout, image stitching, and deep learning-based image generation. These methods offer limited flexibility and produce somewhat unnatural results. Summary of the Invention

[0003] This disclosure provides a poster generation method, apparatus, electronic device, and storage medium.

[0004] According to one aspect of this disclosure, a poster generation method is provided, comprising:

[0005] Obtain poster image materials and poster description information;

[0006] Based on the poster material image and the poster description information, determine the feature vector representation and the mask image of the poster material image;

[0007] Determine the poster layout information, and generate candidate posters based on the poster layout information and feature vector representation;

[0008] Based on the poster material images and corresponding mask images, the candidate posters are combined to generate the target poster.

[0009] According to another aspect of this disclosure, a poster generating apparatus is provided, comprising:

[0010] The first acquisition module is used to acquire poster material images and poster description information;

[0011] The second acquisition module is used to determine the feature vector representation and the mask image of the poster material image based on the poster material image and the poster description information.

[0012] The first generation module is used to determine the poster layout information and generate candidate posters based on the poster layout information and feature vector representation.

[0013] The second generation module is used to combine the candidate posters based on the poster material images and corresponding mask images to generate the target poster.

[0014] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0018] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method described in the first aspect.

[0019] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0022] Figure 1 This is a schematic diagram of a poster generation method provided in an embodiment of this disclosure;

[0023] Figure 2 This is a schematic diagram of another poster generation method provided in this embodiment of the disclosure;

[0024] Figure 3 This is a schematic diagram of another poster generation method provided in this embodiment of the disclosure;

[0025] Figure 4 This is a schematic diagram of the structure of a poster generation device provided in an embodiment of this disclosure;

[0026] Figure 5 A schematic block diagram of an electronic device used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0027] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0028] Image processing is the technology of using computers to analyze images to achieve desired results. It generally refers to digital image processing. A digital image is a large two-dimensional array obtained by taking pictures with equipment such as industrial cameras, video cameras, and scanners. The elements of this array are called pixels, and their values ​​are called gray values.

[0029] Artificial intelligence (AI) is a new technological science that studies and develops theories, methods, technologies, and application systems to simulate, extend, and expand human intelligence. It attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence.

[0030] Computer vision is a science that studies how to make machines "see". More specifically, it refers to machine vision that uses cameras and computers to replace human eyes to identify, track and measure targets, and then performs image processing to make the computer-processed images more suitable for human observation or transmission to instruments for detection.

[0031] Figure 1 This is a schematic diagram of a poster generation method provided in an embodiment of this disclosure. Figure 1 As shown, the method includes:

[0032] S101, Obtain poster material images and poster description information.

[0033] In some embodiments, the poster material image may be one or more images uploaded or entered by the user, which are used to generate the target poster.

[0034] In some embodiments, the poster description information may be user-inputted descriptive information about the target poster, such as the poster's text theme or design style description, to help generate a more accurate target poster.

[0035] S102, Based on the poster material image and poster description information, determine the feature vector representation and the mask image of the poster material image.

[0036] In some embodiments, semantic parsing of poster material images can be performed based on a pre-trained multimodal visual language model to extract multimodal information such as main object categories, compositional features, and color distribution.

[0037] Furthermore, multimodal information and poster description information can be rewritten and optimized based on a large language model to obtain semantically consistent feature vector representations, and image features can be fused with text description features to provide high-quality input for poster generation.

[0038] A mask image is an image or matrix used to specify an operation area or control data processing. It is usually a single-channel image, and the pixel values ​​only represent the category or operation instruction. Optionally, the mask image of the poster material image can be obtained based on a pre-trained recognition model. In this embodiment, the white area in the mask image is the poster material image.

[0039] S103, determine the poster layout information, and generate candidate posters based on the poster layout information and feature vector representation.

[0040] In some embodiments, poster layout information can be determined based on user requirements, which may include constraints such as poster size, number of images, and poster theme, and the layout of the source images can be based on these constraints.

[0041] Optionally, the poster layout information may include information such as the position, size, rotation angle, and front / back hierarchy of each poster image within the poster.

[0042] Furthermore, after determining the poster layout information, based on the poster layout information and feature vector representation, a candidate poster is generated by combining the feature vector representation that integrates the poster material image and the poster description information with the poster layout information including the position and size of the poster material image. This candidate poster is more in line with the theme and style required by the user and has a higher generation efficiency.

[0043] It should be noted that the candidate poster is only a poster framework and other derivative materials generated based on the poster layout information and feature vector representation. Other derivative materials can be stylized materials generated based on the theme features in the feature vector representation, and do not include specific poster material images or other content.

[0044] S104: Combine the candidate posters based on the poster material images and corresponding mask images to generate the target poster.

[0045] Optionally, the position of each poster material image can be determined in the candidate poster, and the mask image corresponding to the poster material image can be filled into the corresponding position in the candidate poster. The poster material image is filled according to the position and shape of the mask image to obtain the target poster, ensuring that the poster material image is accurately presented in the poster.

[0046] In some embodiments, the poster material images and corresponding mask images are combined with the candidate posters to generate a poster in Scalable Vector Graphics (SVG) format, which is then displayed to the user. The user can adjust and replace the poster material images in the poster according to actual needs to generate the final target poster that matches the user's needs.

[0047] In this embodiment, the poster material image and poster description information input by the user are obtained, and feature extraction is performed based on the poster material image and poster description information to obtain a feature vector representation combining the image and text. Poster layout information is obtained, and candidate posters with poster structure are obtained based on the poster layout information and feature vector representation. The mask image of the poster material image is obtained, and the mask image, poster material image and candidate poster are combined to achieve the separation of template and content. This not only improves the overall design consistency of the poster, but also supports flexible replacement of materials, and the poster generation effect is more in line with user needs.

[0048] Figure 2 This is a schematic diagram of another poster generation method provided in this disclosure embodiment. For example... Figure 2 As shown, the method includes:

[0049] S201, Obtain poster material images and poster description information.

[0050] In this embodiment of the disclosure, the method for implementing step S201 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.

[0051] S202, Based on the poster material image and poster description information, determine the feature vector representation and the mask image of the poster material image.

[0052] Optionally, semantic parsing of poster material images can be performed based on a visual language model to extract information such as main object categories, compositional features, and color distribution, thereby obtaining multimodal descriptive information of the poster material images; optimized prompt words can be generated based on the poster description information; the multimodal descriptive information can be rewritten and optimized based on the optimized prompt words by a large model, and a feature vector representation can be output. The image information is converted into textual multimodal descriptive information based on the visual language model, and the multimodal descriptive information is rewritten and optimized based on the optimized prompt words generated by the large model based on the poster description information, resulting in a more accurate, richer, and semantically consistent feature vector representation.

[0053] Optionally, the application scenario of the poster can be determined based on the poster description information; the mask image of the poster material image can be determined based on the application scenario. For example, the application scenario of the poster can include, but is not limited to, e-commerce, social media, administrative office, education, and life and entertainment scenarios. The main features of the mask image are different for different application scenarios. For example, in the e-commerce scenario, the mask image needs to highlight the main product in the poster material image; in the social media scenario, the mask image focuses on the main person in the poster material image; in the education scenario, the mask image focuses on the knowledge elements in the poster material image; or the shape of the white area in the mask image can be set to the shape of scene-related elements. For example, in the education scenario, the shape is mainly the shape of related elements such as books, to ensure that the generated poster effect is more in line with the application scenario and improve the poster generation effect.

[0054] Optionally, a mask image of the poster material image can be generated based on the pre-trained generative model. During model training, the training image set can be labeled, such as a single poster material image, two poster material images, or multiple poster material images. The application scenarios suitable for the training image can also be labeled. Thus, the generative model can be trained based on the labeled training image set, and the mask image can be generated based on the trained generative model, thereby improving the efficiency and accuracy of mask image generation.

[0055] S203, determine the poster layout information, and generate candidate posters based on the poster layout information and feature vector representation.

[0056] In some embodiments, poster size information, the number of images in the poster, and theme information can be determined as poster constraint information, and poster layout information can be generated based on the poster constraint information; wherein the poster size information can be determined based on user requirements, the number of images in the poster can be determined based on the poster material images input by the user or defined by the user, and the theme information can be determined based on the user's poster description information.

[0057] Optionally, the hierarchical relationship between poster material images can be determined based on the number of images and thematic information within the poster. The hierarchical relationship can be a ranking of the importance of the materials in the poster material images or a spatial stacking relationship. For example, poster material image A can be used as the background image of poster material image B, or poster material image C contains the main focus of the product. Compared with poster material image D, which contains a part of the product, poster material image C is highlighted to achieve better visual information communication.

[0058] Based on the size information and the number of images in the poster, the layout information of the poster material images is determined. The layout information is a spatial planning scheme generated within the poster based on the size information and the number of images in the poster to prevent situations such as missing images or image stacking. In this embodiment, the layout information includes at least the image position, image size, and rotation angle of the poster material images.

[0059] Based on the hierarchical relationship between images and the layout information of the images, poster layout information is generated to enhance the spatial visual effect of the poster.

[0060] Optionally, it can also receive a poster type selection instruction and determine the poster constraint information based on the poster type selection instruction. The selection instruction can be a user-initiated specified instruction or a user-specified art style. For example, if the user instructs a certain poster material image to be placed in a certain fixed position, then the poster layout information is generated based on the selection instruction as constraint information. The poster layout information must satisfy the constraint that the user-specified poster material image is placed in a certain fixed position. If the selection instruction is a user-specified art style, then the poster layout information needs to be as close as possible to the layout of that art style.

[0061] Alternatively, the application scenario of the poster can be determined, and the constraint information of the poster can be determined based on the application scenario. For example, if the application scenario of the poster is an administrative office scenario, the application scenario of the poster can be used as the constraint information to make the poster layout information mainly simple and standardized. If the application scenario of the poster is a life and entertainment scenario, the application scenario of the poster can be used as the constraint information to make the poster layout information highlight the characteristics of diversity and creativity.

[0062] After determining the poster layout information, the poster theme and copy information can be determined based on the feature vector representation; the poster is then generated based on the poster layout information, poster theme and copy information to obtain candidate posters.

[0063] For example, the poster theme may include themes such as event invitation, holiday greetings, product promotion, or brand promotion; the copy information may include text information such as promotional copy or blessing copy according to user needs, and may also include descriptive copy of the user's description of the poster generation. Based on the poster layout information, poster theme, and copy information, candidate posters are generated to ensure that the generated candidate posters match user needs, and to ensure that the generated effect matches the required theme and scene, avoiding style deviation.

[0064] In some embodiments, environmental information of the poster application scenario can also be determined, and at least one of the brightness, contrast and color saturation of the candidate poster can be adaptively adjusted based on the application environment information. For example, assuming that the environmental information of the poster application scenario is an outdoor advertising projection with good lighting conditions, the contrast and color saturation of the candidate poster can be appropriately increased so that the candidate poster can be displayed more clearly in the current environment.

[0065] In some embodiments, the style transfer instructions from the user can also be obtained, and the style of the candidate poster can be adjusted according to the style transfer instructions. For example, based on a pre-trained image editing model, the style of the candidate poster can be fused and transferred according to the artistic style, business theme or artistic atmosphere specified by the user, so that the candidate poster matches the user's needs.

[0066] In some embodiments, a visual language model can be used to compare the consistency of visual and linguistic information. If the theme information of the candidate poster is inconsistent with the theme information indicated by the poster description information, the theme content of the candidate poster can be optimized according to the indicated theme information to ensure that the generated candidate poster matches the poster description information or poster theme input by the user.

[0067] S204. Based on the mask image corresponding to the poster material image, perform shape modification on the poster material image to obtain the target material image.

[0068] Mask images can be used to precisely control the editing range of poster material images. For example, a mask image can locate the key areas of the main part of the poster material image. Therefore, the white area in the mask image may only be a part of the content of the poster material image. Based on the shape and size of the mask image, the shape of the poster material image can be changed, such as by cropping, to obtain the target material image that matches the current user's needs.

[0069] S205, Based on the layout information associated with the poster material images, fill the candidate posters with the target material images to generate the target poster.

[0070] It is understandable that the layout information associated with the poster material image includes the position, size, and rotation angle of the poster material image. Based on the layout information, the target material image corresponding to the poster material image is filled into the candidate poster. That is, the target material image is filled into the candidate poster that only includes the frame according to the corresponding position, size, and rotation angle to obtain the final target poster.

[0071] In this embodiment, after acquiring the poster material images and poster description information, the poster material images and poster description information are fused based on the model to obtain accurate and rich feature vector representations. Poster layout information is generated using poster size information, the number of images in the poster, and theme information as constraints. Candidate posters are represented according to the poster layout information and feature vectors. A mask image of the poster material images is generated according to the model. The poster material images are adjusted according to the mask image to obtain target material images that better match the current poster generation scenario. The target material images are filled into the candidate posters according to the layout information to achieve separation of the frame and content. This ensures that the target poster matches the user's needs while supporting flexible adjustment of the material images, making poster generation more flexible.

[0072] Figure 3 This is a schematic diagram of another poster generation method provided in this embodiment of the disclosure. Figure 3 As shown, the method includes:

[0073] S301, Obtain poster material images and poster description information.

[0074] In this embodiment of the disclosure, the method for implementing step S301 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.

[0075] S302, Based on the poster material image and poster description information, determine the feature vector representation and the mask image of the poster material image.

[0076] In this embodiment of the disclosure, the method for implementing step S302 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.

[0077] S303, determine the poster layout information, and generate candidate posters based on the poster layout information and feature vector representation.

[0078] In this embodiment of the disclosure, the method for implementing step S303 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.

[0079] S304, Based on the mask image corresponding to the poster material image, perform shape modification on the poster material image to obtain the target material image.

[0080] In this embodiment of the disclosure, the method for implementing step S304 can be implemented in any of the various embodiments of the disclosure, and no limitation is made here, nor will it be described in detail.

[0081] S305, Based on the layout information associated with the poster material images, fill the candidate posters with the target material images to generate the target poster.

[0082] In this embodiment of the disclosure, the method for implementing step S305 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0083] S306, Determine the application scenario for the poster.

[0084] In some embodiments, the poster application scenario can be provided by the user based on the actual usage scenario.

[0085] S307 adjusts the resolution of the target poster according to the poster application scenario.

[0086] Optionally, a first resolution required by the poster application scenario can be determined; a second resolution of the target poster can be determined, which is the current resolution of the target poster; and the resolution of the target poster can be adjusted based on the first and second resolutions, that is, the second resolution can be adjusted with the first resolution as a reference until it is the same as the first resolution, so that the target poster matches the resolution of the current poster application scenario and improves the display effect of the target poster.

[0087] S308, Adjust the layout structure of the target poster according to the poster application scenario.

[0088] Optionally, the first layout structure required by the poster application scenario can be determined; the second layout structure of the target poster can be determined, which is the current layout structure of the target poster. The layout structure can be the distribution of text, images, and elements in the poster. The editing path for adjusting the second layout structure to the first layout structure can be determined. Based on the editing path, the layout structure of the target poster can be adjusted so that the layout structure of the target poster matches the current poster application scenario, clarifying the hierarchy of different elements such as the main body, copywriting, and images in the poster, and making the presentation of the content in the target poster clearer.

[0089] In this embodiment, poster material images and poster description information input by the user are acquired. Based on the model, the poster material images and poster description information are fused to obtain accurate and rich feature vector representations. Poster layout information is generated using poster size information, the number of images in the poster, and theme information as constraints. Candidate posters are represented according to the poster layout information and feature vectors. A mask image of the poster material images is generated according to the model. The poster material images are adjusted according to the mask image to obtain target material images that better match the current poster generation scenario. The target material images are filled into the candidate posters according to the layout information to achieve separation of the frame and content. This ensures that the target poster meets the user's needs while supporting flexible adjustment of the material images. After the target poster is generated, the resolution and layout structure of the target poster are adjusted according to the poster application scenario, so that the clarity of the target poster meets the scenario requirements in actual application and the presentation of the content in the poster is clearer and more concise, and the poster generation is more flexible.

[0090] Figure 4 This is a schematic diagram of the structure of a poster generation device provided in an embodiment of this disclosure. Figure 4 As shown, the poster generating device 400 includes:

[0091] The first acquisition module 401 is used to acquire poster material images and poster description information;

[0092] The second acquisition module 402 is used to determine the feature vector representation and the mask image of the poster material image based on the poster material image and the poster description information.

[0093] The first generation module 403 is used to determine the poster layout information and generate candidate posters based on the poster layout information and feature vector representation.

[0094] The second generation module 404 is used to combine candidate posters based on poster material images and corresponding mask images to generate target posters.

[0095] In some embodiments, the second generation module 404 is configured to:

[0096] Based on the mask image corresponding to the poster material image, the shape of the poster material image is modified to obtain the target material image;

[0097] Based on the layout information associated with the poster material images, the target material image is filled into the candidate poster to generate the target poster.

[0098] In some embodiments, the first generation module 403 is configured to:

[0099] Based on the feature vector representation, determine the poster theme and copy information;

[0100] The poster is generated based on the poster layout information, poster theme, and copy information, resulting in candidate posters.

[0101] In some embodiments, the second acquisition module 402 is configured to:

[0102] Semantic analysis of poster material images is performed based on a visual language model to obtain multimodal descriptive information of the poster material images;

[0103] Generate optimized prompts based on the poster description information;

[0104] The multimodal description information is rewritten and optimized based on the optimized prompt words by a large model, and the output feature vector representation is generated.

[0105] In some embodiments, the second acquisition module 402 is configured to:

[0106] Determine the application scenario of the poster based on the poster description information;

[0107] Determine the mask image for the poster material based on the poster's application scenario.

[0108] In some embodiments, the first generation module 403 is configured to:

[0109] Determine the poster size, number of images within the poster, and theme information as poster constraints.

[0110] Determine the hierarchical relationship between the poster's images based on the number of images and the theme information within the poster;

[0111] Based on the size information and the number of images in the poster, determine the layout information of the poster's source images;

[0112] Based on the hierarchical relationship between images and the layout information of the images, generate poster layout information.

[0113] In some embodiments, the first generation module 403 is further configured to:

[0114] Receive the poster type selection instruction, and determine the poster constraint information based on the poster type selection instruction; or,

[0115] Determine the application scenario for the poster, and based on the application scenario, determine the poster's constraint information.

[0116] In some embodiments, the first generation module 403 is further configured to:

[0117] Determine the environmental information of the poster application scenario, and based on the application environment information, adaptively adjust at least one of the brightness, contrast and color saturation of the candidate posters;

[0118] In response to the discrepancy between the theme information of the candidate poster and the theme information indicated in the poster description, the theme content of the candidate poster is optimized according to the indicated theme information.

[0119] Obtain style transfer instructions and adjust the style of the candidate posters accordingly.

[0120] In some embodiments, the second generation module 404 is further configured to:

[0121] Determine the application scenario for the poster;

[0122] Adjust the resolution of the target poster according to the poster's application scenario;

[0123] Adjust the layout structure of the target poster according to the application scenario.

[0124] In some embodiments, the second generation module 404 is configured to:

[0125] Determine the primary resolution required for the poster's application scenario;

[0126] Determine the second resolution of the target poster;

[0127] The resolution of the target poster is adjusted based on the first and second resolutions.

[0128] In some embodiments, the second generation module 404 is configured to:

[0129] Determine the primary layout structure required by the poster's application scenario;

[0130] Determine the second layout structure of the target poster;

[0131] Determine the editing path for adjusting the second layout structure to the first layout structure;

[0132] Adjust the layout structure of the target poster according to the editing path.

[0133] In this embodiment, poster material images and poster description information input by the user are acquired. Based on the model, the poster material images and poster description information are fused to obtain accurate and rich feature vector representations. Poster layout information is generated using poster size information, the number of images in the poster, and theme information as constraints. Candidate posters are represented according to the poster layout information and feature vectors. A mask image of the poster material images is generated according to the model. The poster material images are adjusted according to the mask image to obtain target material images that better match the current poster generation scenario. The target material images are filled into the candidate posters according to the layout information to achieve separation of the frame and content. This ensures that the target poster meets the user's needs while supporting flexible adjustment of the material images. After the target poster is generated, the resolution and layout structure of the target poster are adjusted according to the poster application scenario, so that the clarity of the target poster meets the scenario requirements in actual application and the presentation of the content in the poster is clearer and more concise, and the poster generation is more flexible.

[0134] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0135] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0136] Figure 5 A schematic block diagram of an electronic device for implementing embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0137] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0138] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0139] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the poster generation method. For example, in some embodiments, the poster generation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the poster generation method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the poster generation method by any other suitable means (e.g., by means of firmware).

[0140] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0141] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0142] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0143] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0144] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0145] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0146] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0147] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A poster generation method, wherein, The method includes: Obtain poster images and poster description information; Based on the poster material image and the poster description information, a feature vector representation is determined, and based on the poster description information, the poster application scenario is determined. Based on the poster application scenario, the mask image of the poster material image is determined. Determine the poster layout information, and generate candidate posters based on the poster layout information and feature vector representation, wherein the poster layout information is obtained based on the hierarchical relationship between the poster material images and the layout information of the poster material images; Based on the poster material images and corresponding mask images, the candidate posters are combined to generate the target poster; The process of generating the target poster includes: modifying the shape of the poster material image based on the mask image corresponding to the poster material image to obtain a target material image; and filling the candidate poster with the target material image based on the layout information associated with the poster material image to generate the target poster.

2. The method according to claim 1, wherein, The step of generating candidate posters based on the poster layout information and feature vector representation includes: Based on the feature vector representation, the poster theme and copy information are determined; The poster is generated based on the poster layout information, the poster theme, and the copywriting information to obtain the candidate poster.

3. The method according to claim 1, wherein, The process of determining the feature vector representation includes: The poster material images are semantically parsed based on a visual language model to obtain multimodal description information of the poster material images; Optimized prompts are generated based on the poster description information; The multimodal description information is rewritten and optimized based on the optimized prompt words using a large model, and the feature vector representation is output.

4. The method according to claim 1, wherein, The method for obtaining the hierarchical relationship between the poster material images and the layout information of the poster material images includes: Determine the poster size, number of images within the poster, and theme information as poster constraints. The hierarchical relationship between the poster material images is determined based on the number of images in the poster and the theme information; Based on the size information and the number of images in the poster, the layout information of the poster material images is determined.

5. The method according to claim 4, wherein, The method further includes at least one of the following operations: Receive a poster type selection instruction, and determine the poster constraint information based on the poster type selection instruction; or, Determine the application scenario of the poster, and based on the application scenario, determine the poster constraint information.

6. The method according to any one of claims 1-5, wherein, After determining the poster layout information and generating candidate posters based on the poster layout information and feature vector representation, the process further includes: Determine the environmental information of the poster application scenario, and based on the environmental information, adaptively adjust at least one of the brightness, contrast, and color saturation of the candidate poster; In response to the discrepancy between the theme information of the candidate poster and the theme information indicated by the poster description information, the theme content of the candidate poster is optimized according to the indicated theme information. Obtain a style transfer instruction, and adjust the style of the candidate poster according to the style transfer instruction.

7. The method according to any one of claims 1-5, wherein, After generating the target poster, at least one of the following operations is also included: Determine the application scenario for the poster; The resolution of the target poster is adjusted according to the poster's application scenario. The layout structure of the target poster is adjusted according to the application scenario of the poster.

8. The method according to claim 7, wherein, The step of adjusting the resolution of the target poster according to the poster application scenario includes: Determine the first resolution required for the poster application scenario; Determine the second resolution of the target poster; The resolution of the target poster is adjusted based on the first resolution and the second resolution.

9. The method according to claim 7, wherein, The step of adjusting the layout structure of the target poster according to the poster application scenario includes: Determine the first layout structure required by the poster application scenario; Determine the second layout structure of the target poster; Determine the editing path from which the second layout structure is adjusted to the first layout structure; The layout structure of the target poster is adjusted according to the editing path.

10. A poster generating device, comprising: The first acquisition module is used to acquire poster material images and poster description information; The second acquisition module is used to determine a feature vector representation based on the poster material image and the poster description information, determine the poster application scenario based on the poster description information, and determine the mask image of the poster material image based on the poster application scenario. The first generation module is used to determine the poster layout information and generate candidate posters based on the poster layout information and feature vector representation, wherein the poster layout information is obtained based on the hierarchical relationship between the poster material images and the layout information of the poster material images; The second generation module is used to combine the candidate posters based on the poster material images and the corresponding mask images to generate the target poster; The generation of the target poster includes: Based on the mask image corresponding to the poster material image, the shape of the poster material image is modified to obtain the target material image; Based on the layout information associated with the poster material image, the target material image is filled into the candidate poster to generate the target poster.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method of any one of claims 1-9.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-9.

13. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-9.

Images