Method for automatically generating picture and electronic equipment

Abstract

The embodiment of the invention discloses a method for automatically generating a picture and electronic equipment. The method comprises the following steps: determining generation demand information of a user on the picture; generating a semantic tag for the main object image through an artificial intelligence AI large model, so as to obtain a description text for describing the features of the main object image according to the semantic tag, and converting the description text into a first vector; according to a first vector corresponding to the subject object image and second vectors respectively corresponding to various different types of elements in a pre-established element asset library, determining various types of elements respectively matched with the subject object image; and organizing the main object image and the matched elements according to a preset picture description protocol to generate a target picture. According to the embodiment of the invention, the picture generation efficiency can be improved, and the aesthetic value, specialty and other aspects of design practice of the automatically generated picture can be guaranteed.

Description

Technical Field

[0001] The present application relates to the technical field of picture generation, and in particular to a method and electronic device for automatically generating pictures. Background Art

[0002] In the commodity information service system, there is often a need to design marketing materials. Such marketing materials usually exist in the form of pictures, and usually need to present product pictures, brand information, value point information, etc. in the pictures. For example, the operator in the system needs to put an advertisement about the system outside the site, and will need to reflect the product information of the high-quality products in the system in the advertising link. At this time, it is necessary to design marketing material pictures based on the information of the product for off-site delivery. Alternatively, some merchants may need to carry out brand promotion, and may also need to design corresponding marketing material pictures, which also need to include brand-related product pictures, value points and other information. Alternatively, some designers may also need to design some "posters" and other marketing material pictures for promotion, and so on.

[0003] In the traditional way, it is usually the operators, merchants, etc. who put forward the requirements, and then the designers are responsible for the design of the copy, background, font, layout, etc., before the above marketing material pictures can be produced. Alternatively, in order to improve efficiency, some templates can be pre-configured, and the designers can produce marketing material pictures by replacing the products, copy, etc. in the template. However, in this way, the produced pictures will be relatively solidified and have poor mobility. In addition, since the designer is still required to manually replace the products, copy, etc. in the template, the verification still relies on the manual operation of the designer. However, in actual applications, especially in scenarios such as operators conducting on-site and off-site delivery, the delivery rhythm is usually very fast, and hundreds of pictures may need to be produced every day for delivery. Therefore, multiple designers may need to work continuously to keep up with the delivery rhythm, but this makes the labor cost required to be invested very high. In addition, during the delivery process, it may also involve the adjustment of the delivered pictures. For example, after a batch of material pictures are delivered in large quantities one day, it is found that the delivery effect of this batch of materials is not ideal, and a batch of products may need to be replaced, or the value points presented in the materials may not be attractive enough, and the copy needs to be rewritten, etc. These adjustments also need to be done manually, so there will be a problem of slow response speed. In addition, each adjustment also means additional labor costs.

[0004] In short, the existing method of producing marketing material images based on pre-designed templates is difficult to meet the needs of rapid response delivery, and there is also the problem of high labor costs. Summary of the invention

[0005] The present application provides a method and electronic device for automatically generating images, which can improve the efficiency of image generation and ensure the aesthetic value and professionalism of automatically generated images in design practice.

[0006] This application provides the following solutions:

[0007] A method for generating an image, comprising:

[0008] Determining the user's generation requirement information for the image, wherein the generation requirement information at least includes a subject object image;

[0009] Generate a semantic label for the subject object image through an artificial intelligence AI big model, so as to obtain a description text for describing the features of the subject object image according to the semantic label, and convert the description text into a first vector;

[0010] Determine elements of multiple types that match the subject object image, respectively, based on the first vector corresponding to the subject object image and the second vectors corresponding to multiple different types of elements in a pre-established element asset library; wherein the elements are elements required in the process of generating pictures, and the second vectors are generated by converting the description text after a semantic label is generated for the element by an AI big model and a description text for describing the characteristics of the element is obtained according to the semantic label;

[0011] The subject object image and the matched elements are organized according to a preset picture description protocol to generate a target picture.

[0012] Wherein, when generating semantic tags for the subject object image, the generated semantic tags include: category, color tendency, suitable occasion, style and / or crowd of the subject object.

[0013] When generating a semantic tag for the element, the generated semantic tag includes: the scene, style, crowd, category, aspect ratio and / or quantity of the main object to which the element is suitable.

[0014] The elements in the element asset library include: layout scheme elements, content elements, style elements, and background elements. The content elements include multiple subtypes, including image, text, or component. The style elements are used to describe the style configuration information of text and / or components. The style includes fonts and / or colors.

[0015] Among them, the layout scheme class elements are used to define the skeleton structure of the canvas layout format, and define the relative position of the content class elements in the canvas through a structured relative positioning design method, so that the matched content class elements can be filled into the corresponding positions in the canvas according to the definition of the matched target layout scheme class elements.

[0016] Among them, the layout scheme class element is used to divide the canvas into multiple information areas, and define the subtype of the content class elements that need to be filled in each information area and the alignment information of the content class elements in the information area, so as to fill the corresponding subtype of the content class elements into the corresponding information area according to the alignment method.

[0017] The alignment information of the content element in the information area includes:

[0018] The alignment of the content elements in the main axis direction and / or the minor axis direction of the information area, the main axis direction is the arrangement direction of the content elements in the information area, and the minor axis direction is the perpendicular direction to the main axis direction.

[0019] Among them, in the process of generating pictures according to the generation requirement information, for the content of the component class, first determine the size of the background picture frame required to carry the text content based on the text content and font size information to be displayed in the component, and then draw the background picture frame of the component, so that the size of the background picture frame of the component can adaptively change with the text displayed in the component.

[0020] Wherein, a plurality of breakpoints are pre-set in the dimension of size and / or aspect ratio, and corresponding layout scheme elements are configured in the element asset library for the plurality of breakpoints;

[0021] When generating an image according to the generation requirement information, determine the target size and / or aspect ratio of the image to be generated. If the target size and / or aspect ratio does not hit the breakpoint, select a layout scheme class element with the closest size and / or aspect ratio, and adaptively determine the position of the content class element according to the structured relative positioning design information defined by the layout scheme class element.

[0022] In the process of determining the matching elements, after the matching background elements are determined, the color matching elements are determined by programmatic generation.

[0023] Among them, when determining the color matching elements by programmatic generation, it includes:

[0024] Determine the main color of the matched background element, and define a set of offsets based on the main color as the starting point. The offsets make the generated main color system undergo specified changes in hue, S saturation, and L lightness, and make the generated color plate color meet the defined requirements.

[0025] The generation requirement information also includes text content, which includes text content to be added to the generated image and / or text information describing the required scene, style, and target group.

[0026] The main object image includes a product image, and the generated target image includes a marketing material image used to participate in a marketing activity.

[0027] A method for generating an image, comprising:

[0028] Receiving user's generation requirement information for pictures, wherein the generation requirement information at least includes a subject object image;

[0029] Submit the generation requirement information to the server, and the server is used to generate a semantic label for the main object image through the AI ​​big model, so as to obtain a description text for describing the characteristics of the main object image according to the semantic label, and then convert the description text into a first vector; determine the elements that match the main object image under multiple types according to the first vector corresponding to the main object image and the second vectors corresponding to multiple different types of elements in the pre-established element asset library, and organize the main object image and the matching elements according to a preset picture description protocol to generate a target picture; wherein the element is an element required in the process of generating a picture, and the second vector is generated by converting the description text after generating a semantic label for the element through the AI ​​big model and obtaining a description text for describing the characteristics of the element according to the semantic label;

[0030] Receive the generated target image and display it.

[0031] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the methods described above.

[0032] An electronic device, comprising:

[0033] one or more processors; and

[0034] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of any of the methods described above.

[0035] A computer program product comprises a computer program / computer executable instructions, wherein the computer program / computer executable instructions implement the steps of any of the aforementioned methods when executed by a processor in an electronic device.

[0036] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0037] Through the embodiments of the present application, in response to the needs of users to generate pictures, an element asset library can be pre-established, which may include a variety of different types of design elements. For each specific element instance, a semantic label can be generated for the element through the AI ​​big model to obtain a description text for describing the characteristics of the element, and then it can be converted into a second vector for mathematical expression. After the user inputs a specific generation requirement, the AI ​​big model can also be used to generate a semantic label for the subject object image specified by the user to obtain a description text for describing the characteristics of the subject object image, and the description text is converted into a first vector. In this way, according to the first vector corresponding to the subject object image and the second vectors corresponding to the various different types of elements, the elements that match the subject object image under various types can be determined. After that, the subject object image and the matching elements can be organized according to the preset picture description protocol to generate a target picture. In this way, not only the automatic generation of images is achieved, but also the semantic analysis of sensory engineering can be applied to the intelligent typesetting scenario of images. This method produces an intelligent visual matching organizational logic, which can ensure the aesthetic value and professionalism of the automatically generated images in design practice, thereby helping to enhance the visual appeal of images to users, and even improve indicators such as conversion rate, providing strong support for the successful implementation of marketing strategies.

[0038] In the preferred implementation mode, based on the semantic annotation of the multimodal language model and the matching of sensory engineering elements to realize the automated production of design materials, it can also be combined with the structured layout method of flexible layout (Flex), responsive design components, etc., to realize the design rule set summarized based on the design method, and interspersed with the logical thinking ability of the AI ​​large model, which is conducive to the realization of intelligent dynamic adjustment at various sizes, ensuring that users can provide a consistent visual experience in a variety of devices and scenarios.

[0039] Of course, any product implementing the present application does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 It is a schematic diagram of the system architecture provided by the embodiment of the present application;

[0042] Figure 2 is a flow chart of the method provided in the embodiment of the present application;

[0043] Figure 3 It is a schematic diagram of a method for dividing canvas information areas provided in an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of the display effect of content elements in an information area under an alignment method provided in an embodiment of the present application;

[0045] Figure 5 is a schematic diagram of the display effect of content elements in the information area under another alignment method provided in an embodiment of the present application;

[0046] Figure 6 is a schematic diagram of a vector matching process provided in an embodiment of the present application;

[0047] Figure 7 is a schematic diagram of various picture generation results provided in the embodiments of the present application;

[0048] Figure 8 is a flowchart of a client-side method provided in an embodiment of the present application;

[0049] Fig. 9 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0051] In the embodiments of the present application, in response to the demand for generating pictures (including the demand for generating marketing material pictures in e-commerce scenarios, or the demand for generating pictures in other scenarios), the ability of AI large-scale parameter models (referred to as "AI large models") can be used to automatically generate pictures to improve efficiency and reduce or reduce dependence on designers and labor costs. The so-called AI large model refers to a deep learning model containing massive parameters. This AI large model is large in size and can store and process a large amount of information, thereby achieving higher performance in various tasks. It has strong natural language text understanding and logical thinking capabilities, and can usually also realize the ability to process multimodal information and content generation capabilities.

[0052] In specific implementation, the application of using AI big model for content generation has been quite extensive. In the scenario of the embodiment of the present application, a relatively simple AI big model application method can be to directly organize the user's demand information into prompt text (prompt), and input it into the AI ​​big model, and the AI ​​big model generates pictures. However, the quality of pictures generated in this way is usually uncontrollable. If it is an ordinary leisure and entertainment scene, it is acceptable. However, in scenes such as marketing material delivery, there are usually very high requirements for the professionalism of the generated pictures. Therefore, the above-mentioned method of using AI big model to generate pictures makes it difficult for the generated pictures to be directly used for on-site delivery in marketing scenes.

[0053] Therefore, in order to more effectively utilize the generation capability of the AI ​​large model to apply it to scenarios with higher professional requirements, the corresponding solution provided by the embodiment of the present application. In this solution, first, an element asset library can be provided, in which a variety of different types of elements can be stored. These elements can be the various elements required in the image generation process. For example, the type of element may include background, font, component, color, copywriting, layout, etc. For each type, a plurality of different elements can be provided respectively and saved in the element asset library. For example, it may include multiple background images, multiple font screenshots, multiple color schemes, multiple layout schemes, and so on. Among them, the elements in these element asset libraries can be universal assets preset by the system, which can be shared by multiple different users, or the user can also configure his own dedicated element asset library for the user to use when generating images.

[0054] For each specific element in the element asset library, the AI ​​big model can also generate semantic tags for the specific elements to obtain descriptive text for describing the characteristics of the elements. In addition, the descriptive text can be converted into a vector for subsequent matching calculations. In other words, every time a new element is added to the specific element asset library, it can be labeled by the AI ​​big model, wherein structured text can be used to complete the above labeling process, thereby generating a structured descriptive text for describing the characteristics of the specific elements. After that, the descriptive text can be converted into a vector for mathematical expression. Among them, the element label may include the scene, style, crowd, category of the main object, aspect ratio and / or quantity, etc. that the element is suitable for.

[0055] In the case where the above-mentioned element asset library is established, the user can initiate a specific image generation demand. When initiating the demand, at least a specific subject object image needs to be specified. The subject object image is the image that needs to be displayed as the subject in the generated image. For example, in a commodity marketing scenario, the specific subject object image is usually a commodity image, which can be a commodity image of one or more commodities, and so on. After receiving the user's demand information, the subject object image can also be labeled by the AI ​​big model, that is, semantic tags can be added. Specific semantic tags may include the category of the subject object, color tendency, suitable occasions, style and / or crowd, etc. After completing the addition of semantic tags, multiple semantic tags corresponding to the subject object image can be organized together to obtain a description text for describing the characteristics of the element, and then, the description text can also be converted into a vector of mathematical expression.

[0056] It should be noted that, whether it is labeling an element or labeling a subject object image input by a user, structured text can be used to complete the specific labeling task. That is to say, each semantic tag can correspond to a structured text, for example, it can be a key-value pair structure, such as (suitable style: warm), etc. In this way, for the same element or subject object image, it can be combined into a structured description text according to the semantic tags of this structured expression. For example, a subject object image has the following tags (suitable style: warm), (product color tendency: orange), (suitable background type: hand-painted flowers), etc., then the description text of the subject object image can be {(suitable style: warm), (product color tendency: orange), (suitable background type: hand-painted flowers)...}, and then the structured description text is converted into a corresponding vector, and so on.

[0057] In this way, the similarity between the subject object image and each element can be calculated by performing distance calculation on the vector corresponding to the subject object image and the vectors corresponding to various different types of elements in the element asset library, and then the elements that match the subject object image in various types can be determined. After that, the subject object image and the matched elements can be organized according to a preset image description protocol to generate a target image.

[0058] In this way, the main AI model is used to label the subject object image input by the user and various elements in the asset library, and then convert them into vectors of mathematical expression. Then, the matching elements are calculated by calculating the distance between vectors, and finally, the subject object image and various types of matching elements are organized into a specific target image. This ensures the quality of the generated image. Of course, in order to further improve the quality, the AI ​​model can also be controlled to label the subject object image or elements within a certain range. That is, the labels used for labeling are not randomly generated, but are selected by AI from multiple given labels, and so on. The above method is conducive to improving the professionalism of specific target images, and thus increasing the probability of their direct application in professional scenarios such as marketing material delivery.

[0059] In addition, in order to further improve the quality or professionalism of the generated target image, the embodiment of the present application also improves the rendering method in the specific process of generating the image. In the traditional picture design system, it is usually described in layers, and the coordinate position of the specific content in the canvas is defined, etc. However, due to the variety of sizes of the main object images in different user needs, the sizes of the specific texts, components, etc. required to be displayed will also be different. Therefore, it may be difficult to ensure the beauty and harmony of the picture by directly describing it according to this absolute coordinate method.

[0060] In view of the above situation, the embodiment of the present application adopts the definition of the specific layout scheme class element as a container, and defines the alignment method of the specific content class element in the container. That is to say, in the embodiment of the present application, the specific content class element is not placed in the absolute position of the canvas according to the coordinate information, but the specific content class element can be filled into the container according to the predefined alignment method. This method belongs to the structured relative positioning method, and can also achieve the decoupling between layout, content and style, so as to enhance the beauty of the specific generated picture, the harmony of the overall picture, etc., and further enhance its professionalism. More specific implementation methods will be introduced in detail later.

[0061] From the perspective of system architecture, Figure 1As shown, in the embodiment of the present application, an element asset library may be first included, which may include layout, background, font, component, copywriting, color matching and other different types of elements, each type may include multiple different element instances. In the specific implementation, a layout editor and a responsive component editor may also be provided to design specific layout and component elements, and save them in the element asset library. In addition, a matcher may also be included, which can mainly utilize the capabilities of the AI ​​big model to semantically mark the subject object image submitted by the user and each element in the element asset library in a multimodal manner (including content of multiple modalities such as images and texts), and convert them into vectors (Embedding) expressed in mathematics, and then the various types of elements matching the subject object image may be determined by calculating the distance between the vectors. Among them, regarding color matching elements, programmatic generation can be performed after completing the matching of other elements. Then, the renderer may organize the subject object image and the elements matched by each item according to the preset image description protocol to generate a specific target image.

[0062] The specific implementation scheme provided in the embodiments of the present application is described in detail below.

[0063] Embodiment 1

[0064] First, the present invention provides a method for generating a picture. Figure 2 , the method may include:

[0065] S201: Determine the user's generation requirement information for the image, wherein the generation requirement information at least includes a main object image.

[0066] The solution provided in the embodiment of the present application can be applied to professional image generation scenarios, and specific users can include marketers in the system, and can also include merchants and designers. Of course, in practical applications, the solution can also be applied to ordinary life scenarios, and in this case, specific users can also be ordinary consumer users, etc.

[0067] An operation portal for initiating a request for image generation may be provided to the user, and the user may initiate a specific generation request through such an operation portal and submit specific generation requirement information, wherein in the scenario of the embodiment of the present application, the specific generation requirement may at least include a subject object image. The subject object image may also have different definitions in different scenarios. For example, in the scenario of marketing material image generation, the specific subject object image may generally be a product image, and in other scenarios, the specific subject object image may also be a person image, an object image, an animal image, and so on.

[0068] Among them, in the image generation scenario related to marketing materials, the user can directly upload a product image when initiating a request, or, if it is a product that has been released in the current product information service system, the user can also specify the product ID or product details page URL, etc., and the system will select a product image from the product's graphic and text details to generate the image, etc.

[0069] In addition, in an optional manner, the specific generation demand information may also include copy information, which may include copy that needs to be reflected in the target image, such as information on the main value points of the product, etc. Alternatively, if the user has a clearer demand, the specific demand may be described in more detail through specific copy, such as a description of the specific scene, style, color scheme, etc. Alternatively, in the marketing material delivery scenario, the specific delivery channel information may also be described through copy. Since some common delivery channels usually have official standards for the size of the marketing material images delivered, the generated target image may be more in line with the specific delivery demand when this information is known, which is also one of the professional manifestations of the generated image.

[0070] S202: Generate semantic tags for the subject object image through an artificial intelligence AI big model, so as to obtain description text for describing the features of the subject object image according to the semantic tags, and convert the description text into a first vector.

[0071] After receiving the user's demand information, the AI ​​big model can be used to generate semantic labels for the main object image to obtain a descriptive text for describing the characteristics of the main object image. Afterwards, the descriptive text can be converted into a first vector of mathematical expression.

[0072] In the specific implementation, in order to improve the subsequent matching accuracy, the labels output by the AI ​​big model can be controlled. For example, label instances on multiple dimensions can be enumerated in advance, and prompt text (prompt) for interacting with the AI ​​big model can be constructed. Then, the subject object image required by the user and the above prompt text can be input into the AI ​​big model, and the AI ​​big model will perform labeling.

[0073] For example, in a specific implementation, in a scenario where a marketing material image is generated, the specific subject object image is a product image of a certain product. When the AI ​​big model labels the product image, the prompt text input to the AI ​​big model may include the following:

[0074] “Product category: What category is the product in the current picture? You can refer to but not limited to: laptops, hair dryers, eye shadows...;

[0075] Background style suitable for the product: What background style is suitable for the current product? It can be, but is not limited to: calm, warm, minimalist, complex, fashionable, warm...;

[0076] Suitable background type for the product: What is the background type that the current product is suitable for? You can choose from the following but are not limited to: white, black, gray, clean monochrome, monochrome shading, shading with gradient color, clean gradient color, dark gradient color, vector geometric graphics, vector geometric color block segmentation, vector curve color block segmentation, curve lines, hand-painted plants, hand-painted flowers, hand-painted animals, hand-painted scenes, light-sensitive lines, grid space, pure color light and shadow, plant shadow, blue sky and clouds, 3D geometric graphics, 3D flowers, 3D spheres, 3D rendered abstract graphics...;

[0077] Suitable background content for the product: Please use your imagination to think about what kind of background the current product is suitable for, and give a detailed text description;

[0078] Product color preference: What is the color of the product in the current product image? References include but are not limited to: yellow-green, black-and-white, blue-green, and pink-purple;

[0079] Color sorting: What are the three colors that account for the largest proportion in the current product image? Give an array of three main colors and sort them from largest to smallest according to the proportion of these three colors in the product image;

[0080] Recommended background: If the current product image is combined with other products of the same type to form a poster design, what should the background that matches the product look like? Please provide a specific description text;

[0081] Suitable festivals for the product: What festivals or promotional events are suitable for the current product? Please give a detailed text description. If there is no description, please give a ' / ' directly.

[0082] Product target group: What is the target group of the current product? What are the preferences of this group for the product background? Please give a detailed text description.

[0083] The above prompt text can be used to control the AI ​​big model to generate more standardized labels for specific subject object images. Of course, if the listed label instances are not suitable, the AI ​​big model can also generate other labels.

[0084] S203: Determine elements of multiple types that match the main object image according to the first vector corresponding to the main object image and the second vectors corresponding to multiple different types of elements in a pre-established element asset library; wherein the elements are elements required in the process of generating pictures, and the second vectors are generated by converting a description text for describing the characteristics of the elements after a semantic label is generated for the elements through an AI big model and the description text is obtained according to the semantic label.

[0085] After obtaining the first vector corresponding to the main object image, the distance calculation can be performed with the second vectors corresponding to various different types of elements in the pre-established element asset library. Among them, as mentioned above, the specific elements can be various types of elements required in the process of generating pictures, including layout, background, fonts, components, copywriting, color matching, etc. Among them, layout is used to define the skeleton structure of the layout of the picture; the background is the picture with the largest proportion and the bottom layer in the picture; the component refers to the relatively secondary level copywriting in the picture except for the title, etc., including operation buttons, etc., such as: promotion, logistics, discounts, after-sales, action points, selling point functions and other information.

[0086] For the various types of elements mentioned above, the AI ​​big model can also be used to generate semantic labels for the elements to obtain descriptive text for describing the characteristics of the elements. After that, the descriptive text can be converted to obtain the corresponding second vector of mathematical expression. In other words, each element instance in the element asset library can also correspond to its own semantic label and the corresponding second vector obtained by conversion. Specifically, a management entrance to the element asset library can be provided in the background. Designers or merchant users in the system can add new element instances to the element asset library. Each time an element instance is added, the AI ​​big model can add a label to it and convert it into a corresponding vector. The specific method of adding labels will be introduced in detail later.

[0087] Among them, as mentioned above, in order to further improve the professional level of the generated pictures, the design and expression of layout class elements are also provided in the embodiment of the present application. Specifically, the various types of elements in the element asset library can be classified first. Specifically, since layout is usually used to define the skeleton structure of the canvas layout, components, copywriting and main object images input by users usually belong to the foreground content that needs to be filled into the canvas, and fonts, colors, etc. are mainly used to express the style of specific content. Therefore, the above elements can be divided into three categories: layout, content and style. Among them, content class elements can also include multiple subtypes, for example, they can include image class, copywriting class, component class, etc. Style class elements can specifically include fonts, colors and other elements. In addition, since the background can usually play a role in laying the tone of the entire picture, the background can usually be determined first, and then the specific fonts, component colors, etc. can be determined. Therefore, the background can also be used as a separate type of element. In this way, the various types of elements in the specific element asset library can be divided into four categories: layout, content, style and background. In the embodiment of the present application, the mutual decoupling between the above-mentioned elements can be achieved through the structured relative positioning design of the layout elements, so that each part can be dynamically adjusted, and at the same time, when each part changes, the generated pictures can be guaranteed in terms of aesthetics, picture harmony, etc.

[0088] Among them, the structured relative positioning design of the layout in the embodiment of the present application means that the relative position of the content elements in the canvas can be defined by the layout elements, so that when the renderer subsequently organizes various matched elements, the matched content elements can be filled into the corresponding positions in the canvas according to the definition of the matched target layout scheme elements.

[0089] Specifically, the layout element can be used to divide the canvas into multiple information areas, and define the subtype of the content elements to be filled in each information area and the alignment information of the content elements in the information area, so as to fill the corresponding subtype of the content elements into the corresponding information area according to the alignment. That is to say, in an embodiment of the present application, the specific content elements are filled into a certain information area in the specific canvas according to a predefined alignment, for example, centered in a certain information area, or aligned to the left or right, etc. In this way, no matter how the size of the specific content elements themselves changes, the same alignment can be maintained in the corresponding information area, so that the overall harmony of the picture is guaranteed.

[0090] For example, in one way, the canvas can be divided into three information areas: the main information area, the secondary information area, and the main content area. That is, the screen is divided into three parts. In this way, a total of six main layouts can be enumerated. These six layouts are the basic layouts of three-stage structured positioning. Figure 3 Of course, the Figure 3 The division method of different information areas is only given as an example. In actual implementation, the size and proportion of the specific canvas can also be adjusted. For example, it can be lengthened or shortened by a certain proportion in the horizontal or vertical direction, etc.

[0091] In an optional implementation, the above-mentioned structural design paradigm based on the basic specifications and principles of typesetting can also be integrated with a flexible layout (flex) mode, and the definition of the layout structure can be realized by defining the main and auxiliary axes and forced alignment, aiming to achieve a deep integration with the proximity principle, similarity principle and continuity principle in the Gestalt theory of design. In other words, the alignment information of specific content elements in the information area may include: the alignment of content elements in the main axis direction and / or auxiliary axis direction of the information area, wherein the main axis direction is the arrangement direction of the content elements in the information area, and the auxiliary axis direction is the perpendicular direction of the main axis direction.

[0092] In the above way, each information area in the layout can be controlled by the following parameters:

[0093] 1. axis: the main axis direction, that is, the arrangement direction of content elements in the information area. For example, if three copy contents need to be displayed in a certain information area, whether these three copy contents are arranged horizontally or vertically can be determined by this parameter. Specifically, the value of this parameter can be "x" or "y", "x" means that the main axis direction of the area is horizontal, and "y" means that the main axis direction of the area is vertical.

[0094] 2. justify: The main axis alignment of the information area. The value of this parameter can include: "center", "start", "end", "between", and "around".

[0095] "center" means centered in the main axis direction. Assuming the main axis direction is horizontal, when the alignment is "center", the display effect of the content element in the information area can be as follows: Figure 4 (A) As shown in the figure, the dotted rectangle represents an information area in the layout. Three texts need to be displayed in the information area, for example, "One", "Two", and "Three has extra text". Figure 4In the example shown in (A), the third copy will wrap at the end of each word, which is controlled by the content element. When the above three copies are displayed in a certain information area of ​​the canvas, the specific position of the three copies in the information area can be determined by the parameter values ​​of parameters such as axis and justify. Among them, when the parameter value of axis is "x" and the parameter value of justify is "center", the display effect of the above three copies in the information area can be as follows: Figure 4 (A) shows that the three pieces of text are arranged horizontally from left to right, and the three pieces of text are displayed in the center of the area.

[0096] "start" means the head is aligned in the main axis direction. Assuming that the current main axis direction is horizontal, when the alignment is "start", all content elements are aligned to the left in the information area. For example, the specific effect can be as follows: Figure 4 (B) shown.

[0097] "end" means the tail is aligned in the main axis direction. Assuming that the current main axis direction is horizontal, when the alignment is "end", all content elements are aligned to the right in the information area. For example, the specific effect can be as follows: Figure 4 (C) shown.

[0098] "between" means aligning the elements in the main axis direction. If the main axis direction is horizontal, when the alignment is "between", the content element will fill the entire area in the information area. For example, the specific effect can be as follows: Figure 4 (D) as shown.

[0099] "Around" means even distribution in the main axis direction. Assuming that the current main axis direction is horizontal, when the alignment is "around", the content elements will be evenly distributed in the information area. For example, the specific effect can be as follows Figure 4 (E) as shown.

[0100] 3. align: the alignment method of the information area along the secondary axis. In other words, specific content elements can be aligned in the direction of the primary axis of the information area, and can also be aligned in a certain way along the secondary axis. The secondary axis is the vertical direction of the primary axis. For example, if the primary axis is the horizontal direction, that is, the x direction, then the vertical direction, that is, the y direction, is the secondary axis, and so on. The parameter values ​​of the secondary axis alignment method can also include "center", "start", "end", "between", and "around", which respectively represent center alignment, head alignment, tail alignment, head-to-tail alignment, and average distribution.

[0101] For example, assuming that the current main axis direction is horizontal, the main axis alignment direction is "center", that is, center alignment, and the minor axis alignment methods are "center", "start", "end", and "between", the corresponding display effects can be as follows: Figure 5 As shown in (A), (B), (C), and (D).

[0102] In addition, the parameters of the specific information area may also include:

[0103] 4. "pl": This parameter indicates how much space is left on the left side of the current area;

[0104] 5. "pt": This parameter indicates how much space is left at the top of the current area;

[0105] 6. "pb": This parameter indicates how much space is left at the bottom of the current area;

[0106] 7. "pr": This parameter indicates how much space is left on the right side of the current area;

[0107] 8. "gap": This parameter indicates the spacing between elements in the current area;

[0108] 9. "percent": This parameter indicates the percentage of the current area in the total area;

[0109] 10. "background": This parameter indicates whether the current area needs to be filled with background color.

[0110] The above is a method of defining parameters in each information area. The content part may include product images, texts (including titles, value points, brand logos, etc.), components, and background elements. Each element can be freely placed in each information area. The information area defined by the above parameters is of a container nature. Specific content elements can be automatically aligned in the information area using the above alignment rules.

[0111] In this way, with a simple set of rules as the core, it is not only suitable for typesetting needs in a wide range of marketing material generation scenarios, but also can effectively reduce costs, achieve rapid generation of intelligent marketing materials, and have strong scalability, and can be seamlessly connected with various AI back-end technologies (for example, Stable Diffusion raw image technology, large language models, etc.). In addition, since the mathematical coordinate calculations have been removed, only the placement and alignment of elements need to be considered during the layout process. Therefore, the design logic of absolute layout typesetting is simplified, which is more suitable for the logical operation scenarios of language models. Moreover, this strict alignment method is more in line with rigorous design specifications and design aesthetics. In addition, through structured layout, elements can naturally dynamically adapt according to the size of the canvas, and the size adjustment is flexible.

[0112] The dynamic adaptation of the canvas size is mentioned above, and it will be introduced in detail here. In an embodiment of the present application, multiple specific elements can be saved in the element asset library in advance, including layout type elements, and the definition of layout type elements is as described above. Among them, each specific layout type element instance will have information such as size, aspect ratio, etc. For example, the aspect ratio of a certain layout type element instance is 1:1, and the aspect ratio of another layout type element instance is 3:4. There may be 16:9, and so on. In principle, when generating pictures based on user needs, if a certain aspect ratio of the picture is required, you can choose to use a layout element with a certain aspect ratio to generate it, which is the most ideal state. However, in actual applications, the aspect ratios and sizes of the pictures generated corresponding to different user needs may be varied, and some may be uncommon sizes, for example, 3.5:4, and so on. This makes it difficult to enumerate all possible sizes required by users in the element asset library, and design corresponding layout type element instances one by one. On the other hand, since the aforementioned structured layout design method is adopted for the layout type element in the embodiment of the present application, this element can naturally perform dynamic adaptation according to the canvas size. That is to say, as long as the actual required size deviates too much from the size of a certain layout type element itself, a specific image can be generated based on the parameters of the layout type element, and the position of the content-type elements in the generated image can usually still be maintained within a reasonable range. Of course, if the actual required size deviates too much from the size of a certain layout type element itself, it is no longer suitable to generate the image through the layout type element. At this time, other layout type elements with closer sizes can be selected for image generation.

[0113] Therefore, based on the above situation, in the specific implementation, multiple breakpoints can be set in advance in the dimension of size and / or aspect ratio, and corresponding layout scheme class elements can be configured in the element asset library for the multiple breakpoints. For example, in the dimension of aspect ratio, the specific breakpoint positions can include 1:1, 3:4, 16:9, etc. Of course, some more extreme situations can also be set, such as very narrow horizontal or vertical strip layout elements, etc. For other non-breakpoint positions, it is not necessary to configure the corresponding layout elements in the element asset library. Specifically, when generating images according to the generation requirement information, the target size and / or aspect ratio of the image to be generated can be determined first. If the target size and / or aspect ratio does not hit the breakpoint, the layout scheme class element with the closest size and / or aspect ratio can be selected, and the position of the content class element can be adaptively determined according to the structured relative positioning design information defined by the layout scheme class element. It can be seen that through the solution provided by the embodiment of the present application, intelligent dynamic adjustment of multiple different sizes can be achieved to ensure that users can provide a consistent visual experience in a variety of devices and scenarios.

[0114] Among them, the size and / or aspect ratio of the image to be generated in the specific generation requirements can be determined in a variety of ways. For example, in one way, the user can specify the specific required size and / or aspect ratio when entering the requirement information, that is, the input copy requirement information can include the copy description content about the specific required size and / or aspect ratio. Or, in another way, for the marketing material delivery scenario, the user can also specify the specific required delivery channel in the input requirements. Since the specific delivery channel usually puts forward standard official requirements for the size and / or aspect ratio of the specific marketing material, this standard information can be collected by querying the public network, etc. Therefore, the size and / or aspect ratio of the image to be generated by the user can also be determined based on the delivery channel information specified by the user.

[0115] The above describes the definition of layout type elements. In addition, the embodiment of the present application also makes improvements to component type elements. In traditional component design, the background picture frame of the component is usually drawn first (for example, the component is a button, and the button needs a rounded rectangular background picture frame. In addition, some text needs to be displayed on the button to express the function of the button, such as action point copy such as "Go and have a look"), and then the copy is filled in the next background picture frame. However, the following situation often occurs: if the copy that needs to be filled is relatively long, it may exceed the background picture frame, so that the copy cannot be fully displayed, or affect the beauty of the entire picture. In view of this situation, the solution provided by the embodiment of the present application is: first, according to the text content and font size information required to be displayed in the component, determine the size of the background picture frame required to carry the text content, and then draw the background picture frame of the component, so that the size of the background picture frame of the component can be adaptively changed with the text displayed in the component.

[0116] In addition, the style of responsive components can also be standardized in the form of parameters. Taking selling points as an example, the component style can be controlled by the following variables: serial number, stroke line type, outer stroke line type, stroke thickness, outer stroke thickness, rounded corners, projection position, projection color, stroke color, outer stroke color, font color, font component serial number, background color, main color tendency, etc. This implementation method of standardizing the style of responsive components in the form of parameters can better combine the operational advantages, aesthetic advantages, and expansion advantages of the structured positioning paradigm of layout type elements, and its style can be more flexible to adapt to the canvas ratio and the number of words in the copy.

[0117] For the background, you can preset the background image library based on multiple dimensions such as country, industry, holiday nodes, delivery platform, etc., to better adapt to the customized delivery needs of different countries, groups, platforms, and festivals.

[0118] The above introduces the design principles of various types of elements. Based on the above principles, you can add various specific element instances to the element asset library. For example, you can add specific layout type element instances, or add a background image, or add a screenshot of a font, a color scheme, and so on.

[0119] After adding a specific element instance to the element asset library, a semantic label can be generated for it through the AI ​​big model to obtain a structured description text for describing the characteristics of the element.

[0120] In a specific implementation, a specific element instance can be input into the AI ​​big model so that the AI ​​big model can add semantic labels to the specific elements. Among them, in order to make the labels added by the AI ​​big model better serve the subsequent vector distance calculation, some exemplary labels can also be provided to the AI ​​big model. These exemplary labels can be written in the prompt text to control the output of the AI ​​big model. For example, for elements of the layout class, when adding an element instance to the asset library, in addition to the structured information of the specific element, including the division method of each information area, etc., it can also carry a picture example based on the layout element. At this time, the picture example and the above-constructed prompt text can be input into the AI ​​big model, and the AI ​​big model adds specific semantic labels to the element instance. For example, in a specific implementation method, the prompt text constructed for the layout element can be:

[0121] “Scenario specification: Which marketing delivery scenarios the current layout is applicable to, such as marketing channels A, B, C…

[0122] Applicable groups: What kind of groups are the current layout applicable to? What are the preferences of this group for product background? Please give a detailed text description;

[0123] Applicable style: Which of the following is the current layout suitable for: highlighting products, highlighting discount information, highlighting title information, or highlighting usage scenarios? Please give a detailed text description;

[0124] Product aspect ratio: What ratio of products is the current layout suitable for, such as square products, long strip products, and horizontal products;

[0125] Quantity of goods: The current layout is suitable for marketing diagrams of the quantity of several goods.

[0126] In this way, the AI ​​big model can give corresponding labels in the above-mentioned dimensions such as scenario specifications, applicable population, applicable style, product aspect ratio, and product quantity.

[0127] Similarly, for background elements, the specific background element instance is usually a specific background image. Therefore, the background image and the pre-designed prompt text can be input into the AI ​​big model, and the AI ​​big model will mark the background instance. For example, in the scenario of generating marketing material images, the prompt text constructed for the background element can be:

[0128] “Products Type (productsType): Give a product type: what category the displayed product belongs to, you can refer to but not limited to: laptops, hair dryers, eye shadows...

[0129] Background Style (backgroundStyle): What is the visual style of the background of the current marketing image? It can be, but is not limited to: calm, warm, minimalist, complex, fashionable, and warm.

[0130] Background Type (backgroundType): What type of background part of the current marketing image is, you can choose from the following but not limited to: white, black, gray, clean monochrome, monochrome shading, shading with gradient color, clean gradient color, dark gradient color, vector geometric graphics, vector geometric color block segmentation, vector curve color block segmentation, curve lines, hand-painted plants, hand-painted flowers, hand-painted animals, hand-painted scenes, light lines, grid space, pure light and shadow, plant shadow, blue sky and clouds...

[0131] Background Content: What does the current marketing material background look like visually? Please give a detailed text description;

[0132] Background Color Tendency: What is the color of the current marketing materials? You can refer to but not limited to: yellow-green, black-and-white, blue-green, and pink-purple.

[0133] Color sorting (backgroundColorSorting): What are the three colors that account for the largest proportion in the background of the current marketing material? Give an array of HEX of the three main colors and sort them from large to small according to their proportion in the marketing image;

[0134] Matching Products: If the background of the current marketing material is used for poster design, what products should be matched with it? Please give several options.

[0135] Festival Matching: What festival or promotional event is the background of the current marketing material suitable for? Please provide specific option texts. If there is none, please directly provide " / ";

[0136] User: If you use the background of the current marketing material to design other marketing graphics, then what kind of user profile is this background suitable for? Please provide a detailed text description of the visual preference characteristics of this user profile.

[0137] The AI ​​big model can give corresponding labels for specific background elements in the above-mentioned dimensions such as displayed product categories, background types, background content, background color preferences, color sorting, matching products, suitable festivals, and crowds.

[0138] Regarding style and font elements, they are usually used to describe the style and font of text content such as titles, or the style and font of components, etc. Therefore, in the embodiments of the present application, in order to be able to mark specific styles and fonts, they can usually be expressed through screenshots of text content and components to which the styles and fonts are applied. In this way, the specific screenshots can not only express the fonts and styles, but also reflect the text content and components of specific applications. That is, it can be known for what kind of text content and components the fonts and styles are used, which can facilitate the marking of specific font and style elements. After uploading the specific screenshot, the screenshot and the prompt text can be input into the AI ​​big model, so that the style and font elements can be marked. For example, in the scenario of generating marketing material pictures, the prompt text constructed for the style and font elements of the title text content can be:

[0139] "Main and subtitle content: What is the main and subtitle information of the current marketing picture? Please provide specific text information. If there is no information, please directly provide " / ";

[0140] Position of the main and subtitles on the screen: Where are the main and subtitles of the current marketing image located on the screen? The options are, but not limited to, the following: center of the screen, left side of the screen, right side of the screen, upper left of the screen, upper right of the screen. If there is no option, please indicate " / " directly.

[0141] Main title and subtitle fonts: What are the main title and subtitle fonts of the current marketing image? Select from the following options: bold, bold italic, serif, handwriting. If there is no font, please directly give " / ";

[0142] Main title and subtitle font style: What is the font style of the main title and subtitle of the current marketing image? Select from the following options: with stroke, with shadow, with stroke and shadow. If there is none, please directly give " / ";

[0143] Main title and subtitle font color: What are the main title and subtitle font colors of the current marketing diagram? Please provide a specific HEX array. If there is no main title in the picture, please directly provide " / ";

[0144] Target audience of the title content: What is the target audience of the main and subtitle content of the current marketing diagram? Please describe it in detail in a paragraph of text.

[0145] Tips for component style and font elements can include:

[0146] “Price: What is the price of the product given in the current marketing materials? Please provide the specific text. If there is no text, please fill in “ / ”

[0147] Price Font: What is the font used to display prices in current marketing materials? Please select one from the following options: Bold, Bold Italic, Serif, Handwritten. If none, please write " / "

[0148] Price Font Color (priceFontColor): What is the font color of the price in the current marketing image? Please give the specific HEX value

[0149] Price Font Style (priceFontStyle): What is the price font style displayed in the current marketing materials? Please select one from the following options: with stroke, with shadow, with stroke and shadow. If none, please fill in " / "

[0150] Price block color (priceFontBlockColor): What is the color of the block below the price font in the current marketing material? Please provide a specific HEX value. If there is a background below the price font, please directly provide " / "

[0151] Price block shape (priceFontBlockShape): What is the shape of the block below the price font in the current marketing material? Please give a specific shape description. If there is a background below the price font, please directly give “ / ”

[0152] Price color block shape style (priceFontBlockStyle): What is the color block shape style below the price font displayed in the current marketing material? Please select one from the following options: solid color block, gradient color, solid color block with stroke, solid color block with projection, solid color block with stroke and projection, gradient color with stroke, gradient color with projection, gradient color with stroke and projection. If none, please fill in " / ";

[0153] Price audience (priceFontUser): With price content, price color block, price color block shape and price color block shape style as price tags, what is the audience profile of the current price tag? Please give a specific description text

[0154] Price label screen ratio (priceScreenRatio): The price label is based on the price content, price color block, price color block shape and price color block shape style. The current price label ratio in the screen can be referenced but not limited to: 10%, 20%;

[0155] ..."

[0156] Regarding the style and font elements of components, in addition to the above dimensions, multiple other dimensions can be included when marking, such as price tag screen position, discount, discount font, discount font color, discount font style, discount color block color, discount color block shape, discount color block shape style, discount target group, discount label screen ratio, discount label screen position, product selling point, product selling point font, product selling point font color, etc., which are not listed here one by one.

[0157] In short, elements of types such as layout, background, style, font, etc. can be marked separately in the above manner, and then the obtained description text can be converted into a second vector of mathematical expression for calculation with the first vector corresponding to the image of the main object input by the user. For example, in a specific implementation method, the distance between the first vector corresponding to the main object image (which can be called Embedding1) and the second vector corresponding to each type of element (which can be called Embedding2) can be calculated according to the cosine similarity, so as to obtain the similarity calculation results between the two and sort them, and then obtain the semantically closest matching result. The matching process can be as follows Figure 6 shown.

[0158] This method of semantically marking the main object image and various design elements through the AI ​​big model, and then converting them into vectors of mathematical expression for distance calculation to match elements, is to apply the semantic analysis of sensory engineering to the field of intelligent typesetting generated by images. Based on the sensory engineering theory of semantic vectors, the AI ​​big model can be used to perform sensory descriptions of different types of elements, and combined with objective parameters, it can achieve intelligent matching of visual elements of different styles, industry categories, and target groups' preferences. This method produces an intelligent visual matching organizational logic, which can not only enhance the aesthetic value of design practice, but also significantly improve the visual appeal and conversion rate of images to users (design precipitation + business data), providing strong support for the successful implementation of marketing strategies.

[0159] In addition, regarding color matching elements, after the background and other elements are matched, the matching process can be completed using programmed matching rules. Specifically, the automatic color matching logic can be based on the HSL (Hue Saturation Lightness, a method of representing points in the RGB color model in a cylindrical coordinate system) color value rule with the background main color as the starting point. For example, the specific logic can be as follows:

[0160] First, define the color palette, which can include:

[0161] primary: primary color

[0162] Primary Contanier: Primary color light color

[0163] primary Dark: primary dark color

[0164] similar: primary color adjacent to the color

[0165] Similar Contanier: adjacent color lighter color

[0166] similar Dark: adjacent dark color

[0167] contrast: primary color contrast

[0168] Contrast Contanier: Contrasting light color

[0169] contrast Dark: Contrast color dark

[0170] black: black

[0171] white:white

[0172] Then, initialize the main color: assume that the main color of the input background element (that is, the background element that successfully matches the current input main object image) is mainColor;

[0173] Finally, generate the main color system: define a set of HSL offsets for each color value in the color palette. This offset can cause the mainColor to undergo specified changes in H (hue), S (saturation), and L (lightness), so that the colors of each color palette meet the defined requirements.

[0174] For example, in one example, the specific code mainly generates a series of color variants based on a given color (color), including the main color, contrasting color, and similar color and their corresponding container color and dark color. For example, it may specifically include:

[0175] Generate primary colors:

[0176] primary: Set the saturation (s) of mainColor to 0.9 and the brightness (l) to 0.6 to get a primary color with high saturation and medium brightness.

[0177] primaryContainer: Increases the brightness of the primary color by 3 units. Useful for creating containers or background colors that appear brighter and contrast with the primary color.

[0178] primaryDark: Decreases the brightness of the primary color by 2.5 units, creating a darker version for use in dark themes or where emphasis is needed.

[0179] Generate similar color schemes:

[0180] Similar: Subtract 40 from the hue (h) of mainColor and set the saturation and brightness to 0.9 and 0.6 respectively, creating a color that is similar to the main color but different.

[0181] The generation logic of similarContainer and similarDark is the same as that of the main color system, but they are generated based on similar colors.

[0182] Generate contrasting colors:

[0183] contrast: Increase the hue of mainColor by 180 degrees, so that the resulting color is opposite to the primary color on the color wheel, creating a sharp contrast. The saturation and brightness are also set to 0.9 and 0.6. The generation logic of contrastContainer and contrastDark is the same as the previous two, but they are generated based on the contrast color.

[0184] S204: Organizing the subject object image and the matched elements according to a preset image description protocol to generate a target image.

[0185] After matching multiple types of elements, the subject object image and the matched elements can be organized according to the preset image description protocol to generate the target image. In other words, the user inputs the required subject object image, and the intelligent image generation engine automatically generates the image by calling the layout, background, copy, font, component, color scheme and other types of design elements that match the subject object image. Among them, in the way of using the layout element of the aforementioned structured design, the image description protocol can be composed of five sub-modules, namely:

[0186] a. Layout module: mainly used to describe the basic layout framework.

[0187] b. Content module: mainly used to describe the specific content filled between the information areas of the canvas.

[0188] c. Background module: mainly used to describe background information.

[0189] d. Style module: mainly used to describe the configuration information of each responsive component.

[0190] e. Color matching module: mainly used to record the generated color matching table.

[0191] The subject object image, the matched elements of various types, and the image description protocol input by the user can be provided to the renderer, which organizes the subject object image and the matched elements of various types according to the image description protocol to generate the final image for output. For example, the specific generated image can include a marketing material image, which can be provided to the operator for delivery to a specific delivery channel, or it can also be some design posters, etc.

[0192] In summary, through the embodiments of the present application, in response to the needs of users to generate pictures, an element asset library can be pre-established, which may include a variety of different types of design elements. For each specific element instance, the AI ​​big model can be used to generate semantic tags for the element to obtain descriptive text for describing the characteristics of the element, and then it can be converted into a second vector for mathematical expression. After the user inputs a specific generation requirement, the AI ​​big model can also be used to generate semantic tags for the subject object image specified by the user to obtain a descriptive text for describing the characteristics of the subject object image, and the descriptive text is converted into a first vector. In this way, according to the first vector corresponding to the subject object image and the second vectors corresponding to the various different types of elements, the elements that match the subject object image under various types can be determined. After that, the subject object image and the matching elements can be organized according to the preset picture description protocol to generate a target picture. In this way, not only the automatic generation of images is achieved, but also the semantic analysis of sensory engineering can be applied to the intelligent typesetting scenario of images. This method produces an intelligent visual matching organizational logic, which can ensure the aesthetic value and professionalism of the automatically generated images in design practice, thereby helping to enhance the visual appeal of images to users, and even improve indicators such as conversion rate, providing strong support for the successful implementation of marketing strategies.

[0193] In a preferred implementation, based on the semantic annotation of the multimodal language model and the matching of sensory engineering elements to realize the automated production of design materials, it can also be combined with the structured layout method of flexible layout (Flex), responsive design components, etc., to realize a set of design rules summarized based on the design method, and interspersed with the logical thinking ability of the AI ​​large model, which is conducive to the realization of intelligent dynamic adjustment at various sizes, ensuring that users can provide a consistent visual experience in a variety of devices and scenarios. For example, assuming that the main object image input by the user is a picture of a bear toy, in various cases such as different sizes of the picture and different sizes / aspect ratios of the required generated pictures, the generated pictures can be as follows: Figure 7As shown, 7 image generation results of different sizes / aspect ratios are given. Although the positions of the main object images, texts, components and other contents, the arrangement of different contents, background images, fonts, colors, etc. are all different in each image, the aesthetics and overall coordination of the picture are all reasonable, and they are professional-level image generation results.

[0194] Embodiment 2

[0195] The second embodiment corresponds to the first embodiment and provides a method for generating a picture from the perspective of the client. Figure 8 , the method may include:

[0196] S801: receiving user's generation requirement information for pictures, wherein the generation requirement information at least includes a subject object image;

[0197] S802: Submitting the generation requirement information to the server, the server is used to generate a semantic tag for the main object image through the AI ​​big model, so as to obtain a description text for describing the characteristics of the main object image according to the semantic tag, and then convert the description text into a first vector; according to the first vector corresponding to the main object image and the second vectors corresponding to the various different types of elements in the pre-established element asset library, determine the elements that match the main object image under various types, and organize the main object image and the matching elements according to a preset picture description protocol to generate a target picture; wherein the element is an element required in the process of generating a picture, and the second vector is generated by converting the description text after the semantic tag is generated for the element through the AI ​​big model to obtain the description text for describing the characteristics of the element according to the semantic tag;

[0198] S803: Receive the generated target image and display it.

[0199] It should be noted that the embodiments of the present application may involve the use of user data. In actual applications, user-specific personal data can be used in the scheme described herein within the scope permitted by applicable laws and regulations, subject to the requirements of applicable laws and regulations of the country where the user is located (for example, with the user's explicit consent, effective notification to the user, etc.).

[0200] Corresponding to the aforementioned method embodiment 1, the embodiment of the present application further provides a device for generating a picture, which may include:

[0201] A demand determination unit, used to determine the user's generation demand information for the picture, wherein the generation demand information at least includes a subject object image;

[0202] A first vector generating unit, configured to generate a semantic label for the subject object image through an artificial intelligence AI big model, so as to obtain a description text for describing the features of the subject object image according to the semantic label, and convert the description text into a first vector;

[0203] A vector calculation unit, for determining elements of multiple types that match the subject object image respectively according to a first vector corresponding to the subject object image and second vectors corresponding to multiple different types of elements in a pre-established element asset library; wherein the elements are elements required in the process of generating a picture, and the second vectors are obtained by converting a description text used to describe the characteristics of the element after a semantic label is generated for the element by an AI big model and the description text is obtained according to the semantic label;

[0204] The image generation unit is used to organize the main object image and the matching elements according to a preset image description protocol to generate a target image.

[0205] Wherein, when generating semantic tags for the subject object image, the generated semantic tags include: category, color tendency, suitable occasion, style and / or crowd of the subject object.

[0206] When generating a semantic tag for the element, the generated semantic tag includes: the scene, style, crowd, category, aspect ratio and / or quantity of the main object to which the element is suitable.

[0207] Specifically, the elements in the element asset library include: layout scheme elements, content elements, style elements, and background elements, wherein the content elements include multiple subtypes, including image, text, or component types, and the style elements are used to describe the style configuration information of text and / or components, and the style includes fonts and / or colors;

[0208] Among them, the layout scheme class elements are used to define the skeleton structure of the canvas layout format, and define the relative position of the content class elements in the canvas through a structured relative positioning design method, so that the matched content class elements can be filled into the corresponding positions in the canvas according to the definition of the matched target layout scheme class elements.

[0209] Specifically, the layout scheme class element is used to divide the canvas into multiple information areas, and to define the subtype of the content class elements that need to be filled in each information area and the alignment information of the content class elements in the information area, so as to fill the corresponding subtype of the content class elements into the corresponding information area according to the alignment.

[0210] The alignment information of the content element in the information area includes:

[0211] The alignment of the content elements in the main axis direction and / or the minor axis direction of the information area, the main axis direction is the arrangement direction of the content elements in the information area, and the minor axis direction is the perpendicular direction to the main axis direction.

[0212] In specific implementation, in the process of generating images according to the generation requirement information, for the content of the component class, you can first determine the size of the background picture frame required to carry the text content based on the text content and its font size information to be displayed in the component, and then draw the background picture frame of the component, so that the size of the background picture frame of the component can adaptively change with the text displayed in the component.

[0213] In addition, multiple breakpoints may be set in advance in terms of size and / or aspect ratio, and corresponding layout scheme elements may be configured in the element asset library for the multiple breakpoints;

[0214] When generating an image according to the generation requirement information, the target size and / or aspect ratio of the image to be generated can be determined. If the target size and / or aspect ratio does not hit the breakpoint, the layout scheme class element with the closest size and / or aspect ratio is selected, and the position of the content class element is adaptively determined according to the structured relative positioning design information defined by the layout scheme class element.

[0215] In the process of determining the matching elements, after the matching background elements are determined, the color matching elements can be determined by programmatic generation.

[0216] When determining color matching elements through programmatic generation, the main color of the matched background elements can be determined, and a set of offsets can be defined with the main color as the starting point. The offsets can make the generated main color system undergo specified changes in hue, S saturation, and L brightness, and make the generated color plate color meet the defined requirements.

[0217] In addition, the generation demand information may also include text content, which includes text content to be added to the generated image and / or text information describing the required scene, style, and target audience.

[0218] The main object image includes a commodity image, and the generated target image includes a marketing material image used to participate in a marketing activity.

[0219] Corresponding to the second embodiment, the embodiment of the present application further provides a device for generating a picture, which may include:

[0220] A generation requirement receiving unit, used to receive generation requirement information of a user for a picture, wherein the generation requirement information at least includes a subject object image;

[0221] A demand submission unit, used to submit the generation demand information to a server, and the server is used to generate a semantic tag for the subject object image through an AI big model, so as to obtain a description text for describing the characteristics of the subject object image according to the semantic tag, and then convert the description text into a first vector; according to the first vector corresponding to the subject object image and the second vectors corresponding to the various different types of elements in the pre-established element asset library, determine the elements that match the subject object image under various types, and organize the subject object image and the matching elements according to a preset picture description protocol to generate a target picture; wherein the element is an element required in the process of generating a picture, and the second vector is generated by converting the description text after the semantic tag is generated for the element through the AI ​​big model to obtain the description text for describing the characteristics of the element according to the semantic tag;

[0222] The image receiving unit is used to receive the generated target image and display it.

[0223] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods in the aforementioned method embodiments are implemented.

[0224] And an electronic device, comprising:

[0225] one or more processors; and

[0226] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of the aforementioned method embodiments.

[0227] A computer program product includes a computer program / computer executable instructions, which, when executed by a processor in an electronic device, implement the steps of the method described in the aforementioned method embodiment.

[0228] in, Fig. 9The architecture of the electronic device is shown as an example, which may include a processor 910, a video display adapter 911, a disk drive 912, an input / output interface 913, a network interface 914, and a memory 920. The processor 910, the video display adapter 911, the disk drive 912, the input / output interface 913, the network interface 914, and the memory 920 may be communicatively connected via a communication bus 930.

[0229] Among them, the processor 910 can be implemented by a general-purpose CPU (Central Processing Unit, processor), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided in this application.

[0230] The memory 920 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 920 can store an operating system 921 for controlling the operation of the electronic device 900, and a basic input and output system (BIOS) for controlling the low-level operation of the electronic device 900. In addition, a web browser 923, a data storage management system 924, and a generated image processing system 925, etc. can also be stored. The above-mentioned generated image processing system 925 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided in the present application is implemented by software or firmware, the relevant program code is stored in the memory 920 and is called and executed by the processor 910.

[0231] The input / output interface 913 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0232] The network interface 914 is used to connect to a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0233] The bus 930 comprises a pathway for transmitting information between the various components of the device (eg, the processor 910, the video display adapter 911, the disk drive 912, the input / output interface 913, the network interface 914, and the memory 920).

[0234] It should be noted that, although the above device only shows a processor 910, a video display adapter 911, a disk drive 912, an input / output interface 913, a network interface 914, a memory 920, a bus 930, etc., in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include components necessary for implementing the solution of the present application, and does not necessarily include all the components shown in the figure.

[0235] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0236] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0237] The above is a detailed introduction to the method and electronic device for generating pictures provided by the present application. The principles and implementation methods of the present application are described in detail using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A method for generating a picture, characterized in that: include: Determining the user's generation requirement information for the image, wherein the generation requirement information at least includes a subject object image; Generate a semantic label for the subject object image through an artificial intelligence AI big model, so as to obtain a description text for describing the features of the subject object image according to the semantic label, and convert the description text into a first vector; Determine elements of multiple types that match the subject object image, respectively, based on the first vector corresponding to the subject object image and the second vectors corresponding to multiple different types of elements in a pre-established element asset library; wherein the elements are elements required in the process of generating pictures, and the second vectors are generated by converting the description text after a semantic label is generated for the element by an AI big model and a description text for describing the characteristics of the element is obtained according to the semantic label; The subject object image and the matched elements are organized according to a preset picture description protocol to generate a target picture.

2. The method according to claim 1, characterized in that When generating semantic tags for the subject object image, the generated semantic tags include: category, color tendency, suitable occasion, style and / or crowd of the subject object.

3. The method according to claim 1, characterized in that When generating a semantic tag for the element, the generated semantic tag includes: the scene, style, crowd, category, aspect ratio and / or quantity of the main object to which the element is suitable.

4. The method according to claim 1, characterized in that: The elements in the element asset library include: layout scheme elements, content elements, style elements, and background elements, wherein the content elements include multiple subtypes, including image, text, or component types, and the style elements are used to describe the style configuration information of text and / or components, and the style includes fonts and / or colors; Among them, the layout scheme class elements are used to define the skeleton structure of the canvas layout format, and define the relative position of the content class elements in the canvas through a structured relative positioning design method, so that the matched content class elements can be filled into the corresponding positions in the canvas according to the definition of the matched target layout scheme class elements.

5. The method according to claim 4, characterized in that The layout scheme class element is used to divide the canvas into multiple information areas, and to define the subtype of the content class elements that need to be filled in each information area and the alignment information of the content class elements in the information area, so as to fill the corresponding subtype of the content class elements into the corresponding information area according to the alignment.

6. The method according to claim 5, characterized in that The alignment information of the content element in the information area includes: The alignment of the content elements in the main axis direction and / or the minor axis direction of the information area, the main axis direction is the arrangement direction of the content elements in the information area, and the minor axis direction is the perpendicular direction to the main axis direction.

7. The method according to claim 4, characterized in that In the process of generating images according to the generation requirement information, for the content of the component class, first determine the size of the background picture frame required to carry the text content based on the text content and font size information to be displayed in the component, and then draw the background picture frame of the component, so that the size of the background picture frame of the component can adaptively change with the text displayed in the component.

8. The method according to claim 4, characterized in that Pre-setting a plurality of breakpoints in size and / or aspect ratio dimensions, and configuring corresponding layout scheme elements in the element asset library for the plurality of breakpoints; When generating an image according to the generation requirement information, determine the target size and / or aspect ratio of the image to be generated. If the target size and / or aspect ratio does not hit the breakpoint, select a layout scheme class element with the closest size and / or aspect ratio, and adaptively determine the position of the content class element according to the structured relative positioning design information defined by the layout scheme class element.

9. The method according to claim 4, characterized in that In the process of determining the matching elements, after the matching background elements are determined, the color matching elements are determined by programmatic generation.

10. The method according to claim 9, characterized in that When determining color matching elements through programmatic generation, it includes: Determine the main color of the matched background element, and define a set of offsets based on the main color as the starting point. The offsets make the generated main color system undergo specified changes in hue, S saturation, and L lightness, and make the generated color plate color meet the defined requirements.

11. The method according to any one of claims 1 to 10, characterized in that: The generation requirement information also includes text content, which includes text content to be added to the generated image and / or text information describing the required scene, style, and target group.

12. The method according to any one of claims 1 to 10, characterized in that: The main object image includes a commodity image, and the generated target image includes a marketing material image used to participate in a marketing activity.

13. A method for generating a picture, characterized in that: include: Receiving user's generation requirement information for pictures, wherein the generation requirement information at least includes a subject object image; Submit the generation requirement information to the server, and the server is used to generate a semantic label for the main object image through the AI ​​big model, so as to obtain a description text for describing the characteristics of the main object image according to the semantic label, and then convert the description text into a first vector; determine the elements matching the main object image under multiple types according to the first vector corresponding to the main object image and the second vectors corresponding to multiple different types of elements in the pre-established element asset library, and organize the main object image and the matching elements according to a preset picture description protocol to generate a target picture; wherein the element is an element required in the process of generating a picture, and the second vector is generated by converting the description text after generating a semantic label for the element through the AI ​​big model and obtaining a description text for describing the characteristics of the element according to the semantic label; Receive the generated target image and display it.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 13 are implemented.

15. An electronic device, characterized in that: include: one or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of claims 1 to 13.

16. A computer program product comprising a computer program / computer executable instructions, characterized in that: When the computer program / computer executable instructions are executed by a processor in an electronic device, the steps of the method according to any one of claims 1 to 13 are implemented.

Images