Design draft generation method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides a design draft generation method and apparatus, an electronic device, and a storage medium. The method may comprise: acquiring design requirement information inputted by a user and basic materials for generating a design draft; on the basis of the design requirement information, determining at least one design dimension that matches the design requirement information; and inputting feature information of elements in the basic materials and the at least one design dimension into a design draft generation model to obtain a target design draft that matches the design requirement information.

Description

Design draft generation method, device, electronic device and storage medium

[0001] This application claims priority to Chinese Patent Application No. 202311253995.8 filed on September 26, 2023, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0002] The present disclosure relates to a design draft generation method, a design draft generation device, an electronic device, and a computer-readable storage medium. Background Art

[0003] In many fields such as e-commerce, film and television, and education, product promotion, film and television publicity, or book publishing are usually required. Therefore, the demand for various types of design drafts in various industries is gradually increasing.

[0004] However, when producing design drafts, professional designers are usually required to manually produce them, and there are certain technical requirements for the design level (such as composition, typesetting, etc.). The design process also takes a lot of time, resulting in low efficiency in generating design drafts.

[0005] Summary of the Invention

[0006] The embodiments of the present disclosure at least provide a design draft generation method, device, electronic device, and storage medium, which can realize the automatic generation of design drafts.

[0007] The present disclosure provides a method for generating a design draft, including:

[0008] Obtaining design requirement information input by the user and basic materials for generating the design draft;

[0009] Based on the design requirement information, determining at least one design dimension matching the design requirement information;

[0010] The characteristic information of the elements in the basic material and the at least one design dimension are input into a design draft generation model to obtain a target design draft that matches the design requirement information.

[0011] In one possible implementation, basic materials for generating the design draft are obtained according to at least one of the following methods:

[0012] Obtaining the basic material uploaded by the user;

[0013] Searching the basic material library to obtain basic materials that match the design requirement information;

[0014] A material generation tool is used to generate basic materials that match the design requirement information.

[0015] In a possible implementation, determining, based on the design requirement information, at least one design dimension that matches the design requirement information includes:

[0016] performing intent recognition on the design requirement information using an intent recognition model to determine at least one design dimension predicted by the design requirement information; and / or,

[0017] According to the design draft type indicated by the design requirement information, at least one key design dimension matching the design draft type is determined, and the at least one key design dimension is used as the design dimension matching the design requirement information.

[0018] In a possible implementation, characteristic information of elements in the basic material is determined according to the following steps:

[0019] Elements are extracted from the basic material, and feature information of the elements under a plurality of preset feature dimensions is determined.

[0020] In one possible implementation, the at least one design dimension includes at least one of the following:

[0021] Illustration design dimension, text design dimension, attribute design dimension; the attribute design dimension includes multiple sub-dimensions, and each sub-dimension corresponds to a type of attribute information.

[0022] In one possible implementation, the design draft generation model is trained according to the following steps:

[0023] Acquire a first training sample set; the first training sample set includes a plurality of first material samples;

[0024] By performing random prediction processing on feature information of elements in the first material sample under multiple feature dimensions, a design draft generation model of the first training stage is trained.

[0025] In an optional embodiment, the training of the design draft generation model in the first training phase by performing random prediction processing on feature information of elements in the first material sample under multiple feature dimensions includes:

[0026] Obtaining a first feature matrix corresponding to the first material sample by determining feature information of elements in the first material sample in multiple feature dimensions; wherein the feature information of any element in any feature dimension may be empty;

[0027] Using each feature information in the first feature matrix as a minimum mask unit, performing random masking on the first feature matrix to obtain a second feature matrix after masking;

[0028] The second feature matrix is ​​used as input information and the first feature matrix is ​​used as supervision information to train a design draft generation model in the first training stage.

[0029] In an optional embodiment, after the training obtains the design draft generation model of the first training stage, the method further includes:

[0030] Acquire a second training sample set; the second training sample set includes a plurality of second material samples;

[0031] With respect to the target design dimension, feature information of the elements in the second material sample in at least one feature dimension corresponding to the target design dimension is predicted and processed, and a design draft generation model of the second training phase is obtained through training.

[0032] In an optional embodiment, predicting and processing the feature information of the elements in the second material sample in at least one feature dimension corresponding to the target design dimension to train and obtain a design draft generation model in the second training phase includes:

[0033] determining feature information of elements in the second material sample in multiple feature dimensions to obtain a third feature matrix corresponding to the second material sample;

[0034] Using each feature information in the third feature matrix as a minimum mask unit, masking the feature information in the third feature matrix under the target design dimension to obtain a masked fourth feature matrix;

[0035] The fourth feature matrix is ​​used as input information and the third feature matrix is ​​used as supervision information to train a design draft generation model in the second training stage.

[0036] The present disclosure provides a design draft generation device, comprising:

[0037] The material acquisition module is used to obtain the design requirement information input by the user and the basic materials used to generate the design draft;

[0038] a dimension determination module, configured to determine, based on the design requirement information, at least one design dimension that matches the design requirement information;

[0039] The design draft generation module is used to input the feature information of the elements in the basic material and the at least one design dimension into the design draft generation model to obtain a target design draft that matches the design requirement information.

[0040] In a possible implementation, the material acquisition module is specifically configured to:

[0041] Obtain the basic material uploaded by the user; or,

[0042] Based on the design requirement information, searching the basic material library to obtain basic materials that match the design requirement information; or,

[0043] A material generation tool is used to generate basic materials that match the design requirement information.

[0044] In a possible implementation, the dimension determination module is specifically configured to:

[0045] performing intent recognition on the design requirement information using an intent recognition model to determine at least one design dimension predicted by the design requirement information; and / or,

[0046] According to the design draft type indicated by the design requirement information, at least one key design dimension matching the design draft type is determined, and the at least one key design dimension is used as the design dimension matching the design requirement information.

[0047] In a possible implementation, the device further includes a feature extraction module, wherein the feature extraction module is configured to:

[0048] Elements are extracted from the basic material, and feature information of the elements under a plurality of preset feature dimensions is determined.

[0049] In one possible implementation, the at least one design dimension includes at least one of the following:

[0050] Illustration design dimension, text design dimension, attribute design dimension; the attribute design dimension includes multiple sub-dimensions, and each sub-dimension corresponds to a type of attribute information.

[0051] In a possible implementation, the device further includes a model training module, wherein the model training module is configured to:

[0052] Acquire a first training sample set; the first training sample set includes a plurality of first material samples;

[0053] By performing random prediction processing on feature information of elements in the first material sample under multiple feature dimensions, a design draft generation model of the first training stage is trained.

[0054] In a possible implementation, the model training module is specifically used to:

[0055] Obtaining a first feature matrix corresponding to the first material sample by determining feature information of elements in the first material sample in multiple feature dimensions; wherein the feature information of any element in any feature dimension may be empty;

[0056] Using each feature information in the first feature matrix as a minimum mask unit, performing random masking on the first feature matrix to obtain a second feature matrix after masking;

[0057] The second feature matrix is ​​used as input information and the first feature matrix is ​​used as supervision information to train a design draft generation model in the first training stage.

[0058] In one possible implementation, the model training module is further configured to:

[0059] Acquire a second training sample set; the second training sample set includes a plurality of second material samples;

[0060] With respect to the target design dimension, feature information of the elements in the second material sample in at least one feature dimension corresponding to the target design dimension is predicted and processed, and a design draft generation model of the second training phase is obtained through training.

[0061] In a possible implementation, the model training module is specifically used to:

[0062] determining feature information of elements in the second material sample in multiple feature dimensions to obtain a third feature matrix corresponding to the second material sample;

[0063] Using each feature information in the third feature matrix as a minimum mask unit, masking the feature information in the third feature matrix under the target design dimension to obtain a masked fourth feature matrix;

[0064] The fourth feature matrix is ​​used as input information and the third feature matrix is ​​used as supervision information to train a design draft generation model in the second training stage.

[0065] An embodiment of the present disclosure also provides an electronic device, including: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the design draft generation method described in any one of the above possible implementation modes are performed.

[0066] An embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the design draft generation method described in any one of the possible implementation modes described above are executed.

[0067] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0069] FIG1 shows a flowchart of a design draft generation method provided by some embodiments of the present disclosure;

[0070] FIG2 shows a schematic diagram of basic materials provided by some embodiments of the present disclosure;

[0071] FIG3 shows a schematic diagram of a target design draft provided by some embodiments of the present disclosure;

[0072] FIG4 is a schematic diagram showing characteristic information of elements in basic materials provided by some embodiments of the present disclosure;

[0073] FIG5 shows a flowchart of a model training method in the first training phase provided by some embodiments of the present disclosure;

[0074] FIG6 shows a schematic diagram of a first feature matrix before random mask processing provided by some embodiments of the present disclosure;

[0075] FIG7 shows a schematic diagram of a second feature matrix after random mask processing provided by some embodiments of the present disclosure;

[0076] FIG8 shows a flowchart of a model training method in the second training phase provided by some embodiments of the present disclosure;

[0077] FIG9 shows a schematic diagram of a third characteristic matrix before masking for a target design dimension provided by some embodiments of the present disclosure;

[0078] FIG10 is a schematic diagram showing a fourth characteristic matrix after mask processing for a target design dimension provided by some embodiments of the present disclosure;

[0079] FIG11 shows a schematic diagram of a design draft generating device provided by some embodiments of the present disclosure;

[0080] FIG12 is a schematic diagram showing a design draft generating device provided by other embodiments of the present disclosure;

[0081] FIG13 shows a schematic diagram of an electronic device provided by some embodiments of the present disclosure. DETAILED DESCRIPTION

[0082] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the disclosure for which protection is sought, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0083] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0084] The term "and / or" herein simply describes an association relationship, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, and the existence of B alone. In addition, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0085] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0086] In addition, the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0087] Research has found that with the rapid development of computer vision computing, there is a great demand for design drafts (such as posters, illustrations, advertisements, book covers, etc.) in many fields such as e-commerce, film and television production, and education. Professional designers are usually required to manually design and produce them according to design requirements, which consumes a lot of labor costs and design time, resulting in low efficiency in generating design drafts.

[0088] Based on the above research, the present disclosure provides a design draft generation method, device, electronic device, and storage medium. These methods can obtain design requirement information input by a user and the basic materials for generating a design draft. Based on the design requirement information, at least one design dimension that matches the design requirement information can be determined. This design dimension reflects the user's demand direction. The feature information of the elements in the basic materials and the at least one determined design dimension are then input into a design draft generation model, thereby generating a target design draft that meets the design requirements. In this way, the design draft generation model can be used to automatically generate a design draft without the need for manual design and production, which helps save labor costs and design time and improve design efficiency.

[0089] To facilitate understanding of this embodiment, the execution subject of the design draft generation method provided by the embodiment of the present disclosure is first introduced in detail. The execution subject of the design draft generation method provided by the embodiment of the present disclosure is an electronic device, and the electronic device may include a terminal device or a server. For example, the electronic device is a server, and the server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, big data and artificial intelligence platforms. In other embodiments, if the electronic device is a terminal device, the terminal device may be a mobile device, a user terminal, a handheld device, a computing device, a wearable device, etc. In addition, the design draft generation method can also be implemented by a processor calling computer-readable instructions stored in a memory.

[0090] The following is a detailed description of the design draft generation method provided by the embodiment of the present application in conjunction with the accompanying drawings. Referring to FIG1 , which is a flowchart of the design draft generation method provided by the embodiment of the present disclosure, the method includes steps S101 to S103, wherein:

[0091] S101, obtaining design requirement information input by a user, and obtaining basic materials for generating the design draft.

[0092] Here, after the user opens the design draft generation interface, the design requirement information entered by the user and the basic materials uploaded or selected by the user can be obtained. If the user only enters the design requirement information, the basic materials matching the design requirement information can also be obtained through automatic generation.

[0093] In a specific implementation, the user can input design requirement information in a preset design interface (and can also upload basic materials simultaneously), and trigger the design button in the design interface to initiate a design draft generation request.

[0094] Among them, the design requirement information may include a piece of text information or audio and video information, which is used to represent the user's design requirements for the design draft. The design requirement information may be information related to content (text, image), layout, style, color, etc. For example, the design requirement information may be "I want to generate a poster about movie A", "Generate a simple product advertising picture", etc., which is not limited here.

[0095] The basic material may refer to a material containing at least one element (such as a poster, an image, or a paragraph of text, etc.), and the at least one element may include a text element and / or an image element, wherein the number of elements may be one or more, and is not limited here. For example, the basic material may include three paragraphs of text and an image.

[0096] Optionally, the basic material used to generate the design draft may be the basic material uploaded by the user. For example, the design interface described above may include a material upload mark, a requirement editing box, and a design draft generation mark. The user may upload the basic material by triggering the material upload mark, and may input the design requirement information into the requirement editing box, and then generate a design draft generation request by triggering the design draft generation mark.

[0097] Optionally, the basic material used to generate the design draft may also be obtained by searching a basic material library based on the design requirement information to obtain basic materials that match the design requirement information.

[0098] Here, the basic material library usually includes materials of different styles and types. By searching and obtaining basic materials in the basic material library, the efficiency of obtaining materials can be improved.

[0099] For example, if the design requirement information is "generate a poster for movie A", then the basic material library can be searched for basic materials related to "movie A", for example, it can be a line from movie A, or an image of the leading actor of movie A, or a still from movie A, etc.; for another example, if the design requirement information is "I want to generate a science fiction illustration", then the basic material library can be searched for images related to "science fiction", for example, it can be an image of a "spaceship", an image of an "alien", etc.

[0100] Optionally, the basic material used to generate the design draft can be a basic material that matches the design requirement information and is generated by a material generation tool based on the design requirement information.

[0101] Here, the material generation tools can include image generation tools (capable of generating images, textures, icons, etc.) and text generation tools (capable of generating text with different fonts and colors). Based on the design requirements, these tools can generate corresponding basic materials. For example, if the design requirements are for a "car show poster," the material generation tools can generate basic materials related to "cars," such as images of sports cars. In practice, the material generation tools typically provide basic templates, from which highly specialized materials can be generated.

[0102] S102: Based on the design requirement information, determine at least one design dimension that matches the design requirement information.

[0103] Here, by determining the design dimensions that match the design requirements, the key design direction of the design draft can be determined, thereby improving the degree of match between the target design draft and the user's intention.

[0104] For example, the design dimensions may include: picture design dimension, text design dimension, and attribute design dimension.

[0105] The illustration design dimensions may include image filling and image replacement, wherein the image may be a background image, a foreground image, a decorative image, etc., which is not limited here.

[0106] The text design dimensions may include text filling and text replacement, wherein the text may refer to title text, content text, decorative text, etc., which are not limited here.

[0107] The attribute design dimension includes multiple sub-dimensions, each sub-dimension corresponds to a type of attribute information. Here, the attribute information may include position information (i.e., typesetting), color information, font information, etc. Therefore, the multiple sub-dimensions may include position (typesetting) design dimension, color design dimension, and font design dimension, etc. In this embodiment, design is performed based on fine-grained attribute information, so that the target design draft has stronger details.

[0108] For example, if the design requirement information is "add a background image to text A", it can be determined that at least one design dimension that matches the design requirement information can be a picture design dimension and an attribute design dimension. In this way, in subsequent steps, a suitable background image can be added to text A based on the picture design dimension and the attribute design dimension, and text A can be placed in a suitable position in the background image, thereby completing the design of the basic material.

[0109] In some embodiments, when determining at least one design dimension that matches the design requirement information, an intent recognition model may be used to perform intent recognition on the design requirement information, thereby determining at least one design dimension that is predicted according to the design requirement information.

[0110] Among them, intent recognition model is one of the key tasks in the field of natural language processing (NLP), and is often used to classify text or speech to determine the user's intention or behavioral intention.

[0111] Here, if the design requirement information is voice information, the voice information can be recognized to determine the design requirement information in the natural language corresponding to the voice information. Then, the design requirement information in the natural language can be encoded based on the text encoder to obtain the design requirement features corresponding to the design requirement information. The design requirement features are then input into the intent recognition model so that the intent recognition model can perform intent recognition on the design requirement information, thereby determining at least one design dimension that matches the design requirement information.

[0112] Common intent recognition models include any one of machine learning-based classification models (such as support vector machines, naive Bayes, random forests, etc.) and deep learning models (such as recurrent neural networks, convolutional neural networks, etc.), and are not specifically limited in the embodiments of this disclosure.

[0113] Optionally, the intent recognition model can be obtained by pre-training the intent recognition model to be trained. Specifically, a training sample set for intent recognition can be pre-constructed, and the intent recognition model to be trained can be trained based on the training sample set to generate the intent recognition model.

[0114] In other embodiments, when determining at least one design dimension that matches the design requirement information, it is also possible to determine at least one key design dimension that matches the design draft type based on the design draft type indicated by the design requirement information, and use the at least one key design dimension as the design dimension that matches the design requirement information.

[0115] The design draft types may include e-commerce types (such as product images and flyers), film and television types (such as stills and movie posters), publication types (such as illustrations, newspapers, and book covers), etc. In other embodiments, the design draft types may also include other types, which are not limited here. The at least one key design dimension may include at least one of an illustration design dimension, a text design dimension, and an attribute design dimension (position design dimension, font design dimension, and color design dimension).

[0116] In this implementation, for each design draft type, key design dimensions related to the design draft type can be determined. For example, e-commerce type design drafts focus more on product copy information, and their corresponding key design dimensions may include text design dimensions. Film and television type design drafts focus more on movie scenes, actors, release time and other related introductions, and their corresponding key design dimensions may include illustration design dimensions and font design dimensions that highlight related introductions. Publication type design drafts focus more on typesetting, and their corresponding key design dimensions may include typesetting design dimensions.

[0117] For example, if the design requirement information indicates designing a movie poster, the corresponding design draft type is determined to be "film and television type". For another example, if the design requirement information indicates "designing a brochure about building B", the corresponding design draft type can be determined to be "publication type".

[0118] It should be noted that the key design dimensions that match the design draft type here are only the areas that the model needs to focus on generating during design. For other design dimensions, the model can still use the capabilities acquired during the training phase to perform a comprehensive design layout for the design draft, rather than completely ignoring these design dimensions in the design draft. For example, for a publication-type design draft, users generally provide a book cover and publication introduction in the basic materials. Therefore, when generating the design draft, while focusing on the book cover and publication introduction in the typographic design dimension, the font and color attributes of the publication introduction will also be designed accordingly.

[0119] The above-mentioned method of determining design dimensions through intent recognition based on an intent recognition model can be implemented in conjunction with the method of determining design dimensions based on the type of design draft. For example, the design requirement information entered by the user indicates the need for illustrations for a movie poster, and the basic materials uploaded by the user also include text information indicating the movie's release date and actors. In this case, although the user did not indicate a need for font attributes in the design requirement information, because font attributes are a key design dimension of movie-related design drafts, in addition to the illustration design dimension, the font attribute design dimension will also be a key design direction.

[0120] It should be noted that the design dimensions determined here to match the design requirement information can be design dimensions that already have relevant information in the basic material and need to replace relevant information, such as the need to replace the illustrations in the basic material; or they can be design dimensions that lack relevant information in the basic material and need to be filled with relevant information, such as the need to supplement illustrations because there are no illustrations in the basic material.

[0121] S103: Inputting the characteristic information of the elements in the basic material and the at least one design dimension into a design draft generation model to obtain a target design draft that matches the design requirement information.

[0122] In the embodiment of the present disclosure, by determining at least one design dimension that matches the design requirement information, and inputting the characteristic information of the elements in the basic material and the determined at least one design dimension into the design draft generation model, a target design draft that matches the design requirement information is automatically generated without the need for manual design and production, which is beneficial to saving labor costs and design time, improving the efficiency of design draft generation, and reducing the requirements for the professional level of designers.

[0123] In addition, since the design dimensions are matched with the design requirement information, a target design draft that better meets the user's design requirements can be generated in a targeted manner.

[0124] As previously mentioned, a basic material may be a material containing at least one element. Therefore, feature extraction may be performed on each element in the basic material to obtain feature information corresponding to each element in the basic material. The feature information may include color information, font information, image feature information, text feature information, location information, and category information.

[0125] It should be understood that for the picture elements in the basic material, the characteristic information they have may include color information, picture characteristic information, position information and category information; for the text elements in the basic material, the characteristic information they have may include color information, text characteristic information, font information, position information and category information.

[0126] It can be understood that after obtaining the characteristic information of each element and determining at least one design dimension, the characteristic information of each element and the at least one design dimension can be input into the design draft generation model. Based on the characteristic information of each element and at least one design dimension, the design draft model can predict the characteristic information of each element that matches the design requirement information, and then generate the target design draft based on these characteristic information.

[0127] For example, taking poster design as an example, please refer to Figures 2 and 3 at the same time. Figure 2 is a schematic diagram of the basic material provided by some embodiments of the present disclosure, and Figure 3 is a schematic diagram of the target design draft provided by some embodiments of the present disclosure. As shown in Figure 2, the basic material includes the picture element of the "vehicle model" and some text elements, such as "service at home", "time-saving and quality-maintaining", and "beauty starts from the car"; if the design requirement information input by the user is "I want to generate a car wash shop promotional poster with a reasonable layout", then it can be determined that the design dimensions that match the design requirement information include the position design dimension; at this time, the feature information of the picture element of the "vehicle model" and the text elements such as "service at home", "time-saving and quality-maintaining", and "beauty starts from the car" in the basic material, as well as the position design dimension, are input into the design draft generation model to generate the target design draft (poster) shown in Figure 3. As can be seen from Figure 3, the target design draft has rearranged the promotional poster of the car wash shop.

[0128] It should be noted that Figures 2 and 3 are only used to reflect the relationship and difference between the target design draft and the basic materials and design requirement information. The style and position layout of each element in the target design draft are only schematic.

[0129] Optionally, elements may be extracted from the basic material, and characteristic information of the elements under a plurality of preset characteristic dimensions may be determined, thereby obtaining characteristic information of each element in the basic material.

[0130] Here, feature dimensions can include color features, font features (including font and size), image features, text features, location features, and category features. By extracting elements from the basic material and then extracting features from the elements under multiple feature dimensions, the granularity of the extracted features is made finer, thereby improving the accuracy of the target design draft generated in the subsequent steps.

[0131] Among them, feature extraction of elements under multiple feature dimensions refers to extracting features of each element separately under multiple feature dimensions, so as to obtain feature information corresponding to each element. For example, elements A, B, and C are extracted from the basic material, and the feature information of elements A, B, and C under multiple feature dimensions is determined.

[0132] In some embodiments, an image encoder may be used to extract features from the basic material, wherein the image encoder may be implemented using any existing encoder for image encoding, for example, a Transformer-based deep image encoder, which is not specifically limited here.

[0133] For example, please refer to FIG4 , which is a schematic diagram of feature information of elements in the basic material provided in some embodiments of the present disclosure. As shown in FIG4 , when there are multiple elements in the basic material, the feature information of the elements in the basic material can form a multi-dimensional matrix, where each column represents an element and each row corresponds to a feature dimension. For example, for element A, if element A is a text element, the dimension of image feature in the column does not exist (it can be left blank); for another example, if element B is a picture element, the dimension of text feature in the column does not exist either. Of course, the dimension of font feature does not exist either.

[0134] Based on the example in Figure 4, it should be noted that a design dimension can correspond to one or more feature dimensions. For example, the illustration design dimension can correspond to image features, and the attribute design dimension can correspond to position features, color features, and font features. It should be understood that category features are used to characterize the type of element and do not correspond to design dimensions.

[0135] In addition, it should be understood that since different basic materials contain different numbers of elements, the dimensions of the matrices formed by the feature information of the elements in different basic materials may also be different.

[0136] As described above, the design draft generation model can output a target design draft based on the input feature information and design requirement information. Therefore, in order to realize the above-mentioned function of the design draft generation model, it is usually necessary to perform corresponding model training. The training process of the design draft generation model is described in detail below. Specifically, please refer to Figure 5, which includes the following training steps S501 to S502:

[0137] S501: Acquire a first training sample set; the first training sample set includes a plurality of first material samples.

[0138] The first material sample may be a poster, such as a movie poster, a product poster, or a publication, such as a book cover, newspaper, magazine cover, comic, etc. It may also be a web page, an advertisement, etc., and is not limited here.

[0139] Here, in order to increase the generation accuracy of the design draft generation model, the first material sample is usually some material with excellent layout, concise content and beautiful color matching.

[0140] S502 , training a design draft generation model of a first training phase by performing random prediction processing on feature information of elements in the first material sample under multiple feature dimensions.

[0141] The element in the first material sample may be one or more, which is not specifically limited here. The feature information in the random prediction process may be one or more.

[0142] Here, a design draft generation model to be trained can be obtained and, based on the trained design draft generation model, feature information of each element in the first material sample in multiple feature dimensions can be randomly predicted. This random prediction method can improve the model's comprehensive prediction capabilities, and enhance its robustness and adaptability.

[0143] It should be noted that the process of extracting the feature information of the elements in the first material sample under multiple feature dimensions is similar to the process of extracting the feature information of the elements in the basic material under multiple preset feature dimensions in the aforementioned embodiment. For specific details, please refer to the aforementioned embodiment and will not be repeated here.

[0144] Optionally, the design draft generation model to be trained may be a deep learning model, such as a large language model.

[0145] Specifically, when training the design draft generation model of the first training phase by performing random prediction processing on the feature information of the elements in the first material sample under multiple feature dimensions, the following steps (a) to (c) may be included:

[0146] (a) Obtaining a first feature matrix corresponding to the first material sample by determining feature information of elements in the first material sample in multiple feature dimensions; wherein the feature information of any element in any feature dimension may be empty.

[0147] Here, determining the feature information of the elements in the first material sample in multiple feature dimensions refers to determining the feature information of each element in the first material sample in multiple feature dimensions.

[0148] For example, see Figure 6, which is a schematic diagram of a first feature matrix provided in some embodiments of the present disclosure. As shown in Figure 6, the first feature matrix includes feature information of elements A, B, and C under feature dimensions (color feature, font feature, image feature, text feature, position feature, and category feature).

[0149] The fact that the feature information of any element in any feature dimension is empty may mean that during the feature representation process, one or more elements may lack certain feature information, for example, due to factors such as missing features or incomplete feature extraction. In some embodiments, it may also mean that certain features are not available for the element, for example, a text element does not have image features, or an image element does not have text features or font features.

[0150] It should be noted that A1-A6, B1-B6, and C1-C6 are only representative symbols of each feature information, and are not used to represent the feature information itself.

[0151] (b) Using each feature information in the first feature matrix as a minimum masking unit, performing random masking processing on the first feature matrix to obtain a second feature matrix after masking.

[0152] It can be understood that the minimum mask unit refers to the feature information of an element in a feature dimension, and the random masking process on the first feature matrix refers to the masking process on any one or more feature information.

[0153] For example, please refer to Figure 6 again. Each feature information in A1-A6, B1-B6, and C1-C6 can be used as a minimum mask unit to randomly mask the first feature matrix. That is, one or more feature information can be randomly selected from A1-A6, B1-B6, and C1-C6 for masking. For example, the feature information B6 and C2 can be masked to obtain the second feature matrix after masking (as shown in Figure 7).

[0154] In some implementations, mask processing of feature information may be implemented using a preset mask processing algorithm, for example, a LaMa erasing (Large Mask Inpainting) algorithm based on fast Fourier convolution, which is not limited here.

[0155] (c) Using the second feature matrix as input information and the first feature matrix as supervision information, training is performed to obtain a design draft generation model in the first training stage.

[0156] Here, a design draft generation model to be trained can be obtained, and the second feature matrix can be input as input information into the design draft generation model to be trained to obtain first output information. Here, the first output information can refer to a predicted feature matrix corresponding to the second feature matrix, wherein the predicted feature matrix includes predicted feature information corresponding to the masked feature information. Then, the first feature matrix is ​​compared with the first output information, and a first loss value is calculated. The model parameters of the design draft generation model to be trained are adjusted according to the first loss value until the training result meets the first preset requirements, thereby obtaining the design draft generation model of the first training stage.

[0157] In the embodiment of the present disclosure, the first feature matrix is ​​randomly masked, and the second feature matrix obtained by masking is used as input information to train the design draft generation model to be trained. In this way, the design draft generation model to be trained has the ability to generate attribute features of each element.

[0158] It is understandable that since the design draft generation model in the first training phase is trained based on the second feature matrix obtained through random masking as input information, in order to improve the model's prediction accuracy, the embodiment of the present disclosure also performs a fine-grained second training phase on the design draft generation model obtained in the first training phase based on the design dimension. In this way, the model can be intensively trained under different design dimensions, improving the model's prediction accuracy in specific design dimensions.

[0159] The second training phase is described in detail below in conjunction with FIG8 . FIG8 is a flowchart of a model training method for the second training phase provided in some embodiments of the present disclosure. As shown in FIG8 , the method includes S801 to S802 :

[0160] S801: Acquire a second training sample set; the second training sample set includes a plurality of second material samples.

[0161] The second training sample set may be the same as or different from the first training sample set in the aforementioned embodiment. For related descriptions, please refer to the description of the first training sample set, which will not be repeated here.

[0162] S802 , with respect to a target design dimension, predicting feature information of elements in the second material sample in at least one feature dimension corresponding to the target design dimension, and training to obtain a design draft generation model of a second training phase.

[0163] The target design dimension may be any one of an image design dimension, a text design dimension, or an attribute design dimension.

[0164] Here, the first model can be trained on the image design dimension, then based on the first model, the second model can be trained on the text design dimension, and finally, based on the second model, the attribute design dimension can be trained to obtain the design draft generation model of the second training stage. The above training order can be adjusted arbitrarily and is not specifically limited here.

[0165] Of course, in other implementations, training may be performed on only one or two design dimensions according to actual needs, which is not limited here.

[0166] Optionally, when predicting the feature information of the elements in the second material sample in at least one feature dimension corresponding to the target design dimension and training to obtain the design draft generation model of the second training stage, the following steps (1) to (3) may be included:

[0167] (1) Determine feature information of elements in the second material sample in multiple feature dimensions to obtain a third feature matrix corresponding to the second material sample.

[0168] The content of step (1) is similar to that of step (a) in the aforementioned embodiment. For related description, please refer to step (a) and will not be repeated here.

[0169] (2) Using each feature information in the third feature matrix as a minimum mask unit, masking is performed on the feature information in the third feature matrix under the target design dimension to obtain a fourth feature matrix after masking.

[0170] Here, the minimum mask unit refers to the feature information of an element in a feature dimension.

[0171] The masking process is performed on the feature information in the third feature matrix under the target design dimension, which means that the masking process is performed on all feature information corresponding to the target design dimension.

[0172] For example, please refer to Figure 9, which is a schematic diagram of the third feature matrix provided in some embodiments of the present disclosure. As shown in Figure 9, each feature information in A1-A6, B1-B6, and C1-C6 is used as the minimum mask unit. Taking the target design dimension as the illustration design dimension as an example, the feature information corresponding to the illustration design dimension in the third feature matrix is ​​masked. Since the feature dimension involved in the illustration design dimension is a text feature, the multiple feature information (A4, B4, C4) in the row of the text feature can be masked to obtain the masked fourth feature matrix (as shown in Figure 10).

[0173] (3) Using the fourth feature matrix as input information and the third feature matrix as supervision information, training is performed to obtain a design draft generation model in the second training stage.

[0174] In this way, after obtaining the fourth feature matrix, the fourth feature matrix can be input as input information into the design draft generation model of the first training stage to obtain second output information, wherein the second output information can refer to the predicted feature matrix corresponding to the fourth feature matrix, wherein the predicted feature matrix corresponding to the fourth feature matrix includes predicted feature information corresponding to the feature information under the masked target design dimension, and then, the third feature matrix is ​​compared with the second output information, the second loss value is calculated, and the model parameters of the design draft generation model of the first training stage are adjusted according to the second loss value until the training results meet the second preset requirements. That is, in this embodiment, targeted and fine-grained reinforcement training is carried out for each design dimension, which can improve the prediction accuracy of the model, so that the target design draft generated subsequently based on the design draft generation model is more in line with user needs.

[0175] It should be noted that, in the embodiment of the present disclosure, in order to maximize the accuracy of the design draft generation model, a design draft generation model of the second training phase is obtained after training for each design dimension.

[0176] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0177] Based on the same inventive concept, the embodiment of the present disclosure also provides a design draft generation device corresponding to the design draft generation method. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned design draft generation method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0178] 11 is a schematic diagram of a design draft generation device provided in some embodiments of the present disclosure, wherein the design draft generation device 1100 includes:

[0179] The material acquisition module 1110 is used to obtain the design requirement information input by the user and the basic materials used to generate the design draft;

[0180] A dimension determination module 1120 is configured to determine, based on the design requirement information, at least one design dimension that matches the design requirement information;

[0181] The design draft generation module 1130 is used to input the feature information of the elements in the basic material and the at least one design dimension into the design draft generation model to obtain a target design draft that matches the design requirement information.

[0182] In a possible implementation, the material acquisition module 1110 is specifically configured to:

[0183] Obtain the basic material uploaded by the user; or,

[0184] Based on the design requirement information, searching the basic material library to obtain basic materials that match the design requirement information; or,

[0185] A material generation tool is used to generate basic materials that match the design requirement information.

[0186] In a possible implementation, the dimension determination module 1120 is specifically configured to:

[0187] performing intent recognition on the design requirement information using an intent recognition model to determine at least one design dimension predicted by the design requirement information; and / or,

[0188] According to the design draft type indicated by the design requirement information, at least one key design dimension matching the design draft type is determined, and the at least one key design dimension is used as the design dimension matching the design requirement information.

[0189] 12 is a schematic diagram of a design draft generation device provided in some other embodiments of the present disclosure. The device further includes a feature extraction module 1140. The feature extraction module 1140 is used to:

[0190] Elements are extracted from the basic material, and feature information of the elements under a plurality of preset feature dimensions is determined.

[0191] In one possible implementation, the at least one design dimension includes at least one of the following:

[0192] Illustration design dimension, text design dimension, attribute design dimension; the attribute design dimension includes multiple sub-dimensions, and each sub-dimension corresponds to a type of attribute information.

[0193] In a possible implementation, the apparatus further includes a model training module 1150, wherein the model training module 1150 is configured to:

[0194] Acquire a first training sample set; the first training sample set includes a plurality of first material samples;

[0195] By performing random prediction processing on feature information of elements in the first material sample under multiple feature dimensions, a design draft generation model of the first training stage is trained.

[0196] In one possible implementation, the model training module 1150 is specifically configured to:

[0197] Obtaining a first feature matrix corresponding to the first material sample by determining feature information of elements in the first material sample in multiple feature dimensions; wherein the feature information of any element in any feature dimension may be empty;

[0198] Using each feature information in the first feature matrix as a minimum mask unit, performing random masking on the first feature matrix to obtain a second feature matrix after masking;

[0199] The second feature matrix is ​​used as input information and the first feature matrix is ​​used as supervision information to train a design draft generation model in the first training stage.

[0200] In one possible implementation, the model training module 1150 is further configured to:

[0201] Acquire a second training sample set; the second training sample set includes a plurality of second material samples;

[0202] With respect to the target design dimension, feature information of the elements in the second material sample in at least one feature dimension corresponding to the target design dimension is predicted and processed, and a design draft generation model of the second training phase is obtained through training.

[0203] In one possible implementation, the model training module 1150 is specifically configured to:

[0204] determining feature information of elements in the second material sample in multiple feature dimensions to obtain a third feature matrix corresponding to the second material sample;

[0205] Using each feature information in the third feature matrix as a minimum mask unit, masking the feature information in the third feature matrix under the target design dimension to obtain a masked fourth feature matrix;

[0206] The fourth feature matrix is ​​used as input information and the third feature matrix is ​​used as supervision information to train a design draft generation model in the second training stage.

[0207] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference can be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0208] Based on the same technical concept, an embodiment of the present disclosure also provides an electronic device. Referring to FIG13 , there is shown a schematic diagram of the structure of an electronic device 1300 provided in an embodiment of the present disclosure, which includes a processor 1301, a memory 1302, and a bus 1303. The memory 1302 is used to store execution instructions and includes a memory 13021 and an external memory 13022. The memory 13021 is also referred to as internal memory and is used to temporarily store calculation data in the processor 1301, as well as data exchanged with an external memory 13022, such as a hard disk. The processor 1301 exchanges data with the external memory 13022 through the memory 13021.

[0209] In the embodiment of the present application, the memory 1302 is specifically used to store application code for executing the solution of the present application, and the execution is controlled by the processor 1301. That is, when the electronic device 1300 is running, the processor 1301 communicates with the memory 1302 via the bus 1303, so that the processor 1301 executes the application code stored in the memory 1302, thereby performing the method described in any of the aforementioned embodiments.

[0210] Among them, the memory 1302 can be, but is not limited to, random access memory (RAM), read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0211] The processor 1301 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0212] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 1300. In other embodiments of the present application, the electronic device 1300 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0213] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program executes the steps of generating the design draft in the above method embodiment. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0214] The present disclosure also provides a computer program product that carries program code. The program code includes instructions that can be used to execute the steps of generating a design draft in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0215] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0216] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0217] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0218] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0219] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0220] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be subject to the scope of protection of the claims.

Claims

1. A design draft generation method, comprising: Obtain design requirement information input by the user, as well as basic materials for generating design drafts; Based on the design requirement information, determining at least one design dimension matching the design requirement information; The characteristic information of the elements in the basic material and the at least one design dimension are input into a design draft generation model to obtain a target design draft matching the design requirement information.

2. The design draft generation method according to claim 1, wherein: Obtain the basic material for generating the design draft in at least one of the following ways: Get the basic materials uploaded by the user; Searching the basic material library to obtain basic materials that match the design requirement information; A material generation tool is used to generate basic materials that match the design requirement information.

3. The design draft generation method according to claim 1 or 2, wherein: The determining, based on the design requirement information, at least one design dimension matching the design requirement information comprises: Using an intent recognition model to perform intent recognition on the design requirement information, and determining at least one design dimension predicted by the design requirement information; and / or, According to the design draft type indicated by the design requirement information, at least one key design dimension matching the design draft type is determined, and the at least one key design dimension is used as the design dimension matching the design requirement information.

4. The design draft generation method according to any one of claims 1 to 3, wherein: Determine the characteristic information of the elements in the basic material according to the following steps: Elements are extracted from the basic material, and feature information of the elements under a plurality of preset feature dimensions is determined.

5. The design draft generation method according to any one of claims 1 to 4, wherein: The at least one design dimension includes at least one of the following: Illustration design dimension, text design dimension, attribute design dimension; wherein the attribute design dimension includes multiple sub-dimensions, each of the multiple sub-dimensions corresponds to a type of attribute information.

6. The design draft generation method according to any one of claims 1 to 5, wherein: The design draft generation model is trained according to the following steps: Acquire a first training sample set; the first training sample set includes a plurality of first material samples; The design draft generation model of the first training phase is trained by performing random prediction processing on the feature information of the elements in the first material sample under multiple feature dimensions.

7. The design draft generation method according to claim 6, wherein: The step of training a design draft generation model in a first training phase by randomly predicting feature information of elements in the first material sample under multiple feature dimensions includes: By determining feature information of elements in the first material sample in multiple feature dimensions, a first feature matrix corresponding to the first material sample is obtained; wherein the feature information of any element in any feature dimension of the multiple feature dimensions may be empty; Taking each feature information in the first feature matrix as a minimum mask unit, performing random masking processing on the first feature matrix to obtain a second feature matrix after masking processing; The second feature matrix is ​​used as input information and the first feature matrix is ​​used as supervision information to train a design draft generation model in the first training stage.

8. The design draft generation method according to claim 6 or 7, wherein: After the training obtains the design draft generation model of the first training stage, the method further includes: Acquire a second training sample set; the second training sample set includes a plurality of second material samples; With respect to the target design dimension, feature information of the elements in the second material sample in at least one feature dimension corresponding to the target design dimension is predicted and processed, and a design draft generation model of the second training phase is obtained through training.

9. The design draft generation method according to claim 8, wherein: The predicting and processing of the feature information of the elements in the second material sample in at least one feature dimension corresponding to the target design dimension, and training to obtain a design draft generation model in the second training phase, includes: Determine feature information of elements in the second material sample in multiple feature dimensions to obtain a third feature matrix corresponding to the second material sample; Taking each feature information in the third feature matrix as a minimum mask unit, masking the feature information in the third feature matrix under the target design dimension to obtain a fourth feature matrix after masking; The fourth feature matrix is ​​used as input information and the third feature matrix is ​​used as supervision information to train and obtain the design draft generation model of the second training stage.

10. A design draft generating device comprises: A material acquisition module is configured to acquire design requirement information input by a user and basic materials for generating the design draft; A dimension determination module, configured to determine at least one design dimension matching the design requirement information based on the design requirement information; as well as The design draft generation module is configured to input the feature information of the elements in the basic material and the at least one design dimension into the design draft generation model to obtain a target design draft that matches the design requirement information.

11. An electronic device, comprising: A processor, a memory and a bus, wherein the memory stores machine-readable requests executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable request is executed by the processor, the design draft generation method as described in any one of claims 1 to 9 is executed.

12. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the design draft generation method according to any one of claims 1 to 9 is executed.

Images