Man-machine interaction full-process intelligent plane design method

Abstract

The invention relates to a human-computer interaction full-process intelligent plane design method, which comprises the following steps: firstly, on the basis of alignment, repetition, superposition and proximity principles, quantifying the relationship between design elements, constructing a structured aesthetic perception graph, and converting a complex design mode into a computable data model; secondly, performing pattern mining on the AG through a clustering algorithm, integrating multi-source design data to construct an aggregation knowledge graph, fusing explicit layout rules and potential cross-graph association, and providing a reference framework for diversified design; a self-constraint random walk strategy is put forward, design content is dynamically expanded according to user input, an expansion boundary is adaptively controlled through similarity scores, and content reasonability is ensured; and finally, designing an aesthetic perception layout generator to realize accurate mapping from the element sequence to the high-quality layout. According to the method, the limitation of traditional template design is broken through, a non-professional user is supported to quickly generate a design scheme with aesthetic specifications and individuation through a small amount of input, the design threshold is remarkably reduced, and the efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of graphic design, and more specifically, to a fully intelligent graphic design method with human-computer interaction. Background Technology

[0002] With the advent of the digital age, graphic design is increasingly being used in brand promotion, product marketing, and content dissemination. From social media posters to e-commerce product brochures, and to the visual identity design of corporate brands, graphic design has become an important means for businesses and individuals to convey information and shape their image.

[0003] Currently, various web applications for human-computer interaction graphic design have emerged in the market, such as Luban and Canva. These platforms help users quickly complete design tasks by providing rich design templates, resource libraries, and intuitive interfaces. Taking Canva as an example, the platform offers a variety of design tools and templates, allowing users to choose suitable templates for editing according to their needs. Canva also supports multi-user collaboration, enabling team members to edit and comment on design works in real time, greatly improving design efficiency. In addition, Canva provides a variety of design elements, such as images, icons, and fonts, which users can freely combine to create unique designs.

[0004] Despite this, existing intelligent design systems still have some problems. First, while these platforms offer a wealth of design templates and resource libraries, they often lack personalization and innovation, making it difficult for users to create unique designs using these templates and resources. Second, although the design tools on these platforms are easy to use, complex design tasks still require users to have certain design skills and experience, posing a learning curve for beginners. Furthermore, the design processes on these platforms are often relatively fixed, making it difficult for users to flexibly adjust them to their needs, thus limiting the diversity and flexibility of designs. Summary of the Invention

[0005] To overcome at least one deficiency in the prior art, this application provides a full-process intelligent graphic design method for human-computer interaction.

[0006] Firstly, a full-process intelligent graphic design method for human-computer interaction is provided, including: Step 1: Obtain the training dataset. The samples in the training dataset are posters containing multiple elements. For each poster in the training dataset, quantify the relationships between elements, including alignment, repetition, overlap, and proximity, to obtain the aesthetic perception map corresponding to the poster. Step 2: Represent the aesthetic perception map corresponding to each poster as a fusion matrix, and train the convolutional neural network based on the fusion matrix to obtain the trained convolutional neural network; the convolutional neural network includes an encoder and a decoder. Step 3: Input the fusion matrix into the trained convolutional neural network, and the encoder outputs intermediate feature vectors; perform k-means clustering on all intermediate feature vectors to obtain multiple subclasses, each of which includes multiple intermediate feature vectors; integrate the aesthetic perception maps corresponding to all intermediate feature vectors in each subclass into an aggregated knowledge graph. Step 4: Represent the aesthetic perception map corresponding to each poster as an aesthetic perception embedding code, and train the Transformer encoder based on the aesthetic perception embedding code to obtain the trained Transformer encoder. Step 5: Perform a random walk on each aggregated knowledge graph, starting from the user input element, and expand the nodes and edges in the aggregated knowledge graph to obtain subgraphs of the aggregated knowledge graph; integrate all subgraphs of the aggregated knowledge graph to obtain the total content expansion; Step 6: Expand the total content and input it into the trained Transformer encoder to generate a poster layout, which includes the coordinates, height, and width of each element. Step 7: Determine if the user has adjusted the poster layout. If not, the current poster layout is the final poster layout. If yes, generate the adjusted poster layout, return to step 5, and use the adjusted poster layout as the new user input element.

[0007] In one embodiment, a random walk is performed on each aggregated knowledge graph, and the condition for determining the end of the expansion is: when When the extension ends, it is determined that the extension has ended; among them, For the calculation of the score in step t, The score is calculated in step t+1;

[0008] in, To aggregate knowledge graphs after the first... The walking sequence after step expansion, For the first The aesthetic perception diagram unfolds in sequence corresponding to each poster. Number of posters; The ratio of the most similar common strings between two sequences.

[0009] Secondly, a fully intelligent graphic design system with human-computer interaction is provided, including: The aesthetic perception map generation module is used to obtain the training dataset, in which the samples are posters containing multiple elements; for each poster in the training dataset, the relationship between elements is quantified, including alignment, repetition, superposition and proximity, to obtain the aesthetic perception map corresponding to the poster. The first training module is used to represent the aesthetic perception map corresponding to each poster as a fusion matrix, and to train the convolutional neural network based on the fusion matrix to obtain the trained convolutional neural network; the convolutional neural network includes an encoder and a decoder. The classification and aggregation knowledge graph production module is used to input the fusion matrix into the trained convolutional neural network, and the encoder outputs intermediate feature vectors; k-means clustering is performed on all intermediate feature vectors to obtain multiple subclasses, each of which includes multiple intermediate feature vectors; and the aesthetic perception graphs corresponding to all intermediate feature vectors in each subclass are integrated into an aggregation knowledge graph. The second training module is used to represent the aesthetic perception map corresponding to each poster as an aesthetic perception embedding code, and to train the Transformer encoder based on the aesthetic perception embedding code to obtain the trained Transformer encoder. The content expansion module is used to perform a random walk on each aggregated knowledge graph, starting from the user input element, to expand the nodes and edges in the aggregated knowledge graph to obtain a subgraph of the aggregated knowledge graph; and to integrate the subgraphs of all aggregated knowledge graphs to obtain the total content expansion. The poster layout generation module is used to expand the total content input to the trained Transformer encoder and generate a poster layout, which includes the coordinates, height and width of each element. The judgment module is used to determine whether the user has adjusted the poster layout. If not, the current poster layout is the final poster layout. If so, the adjusted poster layout is generated and returned to the content expansion module, which uses the adjusted poster layout as a new user input element.

[0010] In one embodiment, a random walk is performed on each aggregated knowledge graph, and the condition for determining the end of the expansion is: when When the extension ends, it is determined that the extension has ended; among them, For the calculation of the score in step t, The score is calculated in step t+1;

[0011] in, To aggregate knowledge graphs after the first... The walking sequence after step expansion, For the first The aesthetic perception diagram unfolds in sequence corresponding to each poster. Number of posters; The ratio of the most similar common strings between two sequences.

[0012] Thirdly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, enables the aforementioned intelligent planar design method for the entire process of human-computer interaction.

[0013] Fourthly, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the aforementioned intelligent graphic design method for the entire process of human-computer interaction.

[0014] Compared to existing technologies, this application offers the following advantages: The full-process intelligent graphic design method for human-computer interaction, firstly, quantifies the relationships between design elements and constructs a structured aesthetic perception graph (AG) based on the principles of alignment, repetition, superposition, and proximity in art and design theory, transforming complex design patterns into a computable data model. Secondly, it performs pattern mining on the AG using clustering algorithms, integrates multi-source design data to construct an aggregated knowledge graph (AKG), and merges explicit layout rules with potential cross-graph associations to provide a reference framework for diverse designs. Furthermore, it proposes a self-constrained random walk strategy to dynamically expand design content based on user input and adaptively controls the expansion boundary through similarity scoring to ensure content rationality. Finally, it designs an aesthetic perception layout generator based on the Transformer architecture, achieving accurate mapping from element sequences to high-quality layouts through aesthetic perception embedding encoding and design perception loss function optimization. This embodiment overcomes the limitations of traditional template-based design, enabling non-professional users to quickly generate aesthetically pleasing and personalized design schemes with minimal input, significantly reducing the design threshold and improving efficiency. Attached Figure Description

[0015] This application can be better understood by referring to the description given below in conjunction with the accompanying drawings, which, together with the detailed description below, are incorporated in and form part of this specification. In the drawings: Figure 1 A flowchart illustrating the entire process of intelligent graphic design for human-computer interaction is shown. Detailed Implementation

[0016] Exemplary embodiments of the present application will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of the actual embodiments are described in the specification. However, it should be understood that many embodiment-specific decisions can be made in the development of any such actual embodiment to achieve the developer’s specific objectives, and these decisions may vary as the embodiments differ.

[0017] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the device structure closely related to the solution according to this application is shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0018] It should be understood that this application is not limited to the described embodiments by virtue of the following description with reference to the accompanying drawings. In this document, embodiments may be combined with each other, features may be substituted or borrowed between different embodiments, and one or more features may be omitted in one embodiment, where feasible.

[0019] This application provides a full-process intelligent graphic design method with human-computer interaction. Figure 1 A flowchart illustrating the entire process of intelligent graphic design for human-computer interaction is shown. (See attached image) Figure 1 The method mainly includes the following steps: Step 1: Obtain the training dataset. The samples in the training dataset are posters with multiple elements. For each poster in the training dataset, quantify the relationships between the elements, including alignment, repetition, overlay, and proximity, to obtain the aesthetic perception map corresponding to the poster.

[0020] Effective representation of layout design information is crucial for pattern mining. Inspired by art theory, most layout design patterns can be explained through four relationships between element pairs: alignment, repetition, overlap, and proximity. Therefore, we present a computable representation and quantification of these relationships based on design attributes, so that each poster can be represented as an aesthetic perception map (AG) with novel design relationships. The specific quantification method is as follows: Alignment: If elements and bounding box and No intersection ,and Simultaneously satisfying:

[0021] The relationship between them is then defined as alignment.

[0022] Repetition: If an element And satisfy If so, then the relationship between them is defined as repetition.

[0023] Overlay (Unity): If elements and bounding box and The intersection area is greater than Then the relationship between them is defined as superposition.

[0024] Proximity: If an element is adjacent to another element, the element is adjacent to another element. and The center distance of the bounding box is greater than or If they are adjacent, then the relationship between them is defined as proximity.

[0025] in, Represents element The bounding box, This indicates the coordinates of the top-left corner of the bounding box. and These represent the width and height of the bounding box, respectively. and These represent the width and height of the poster, respectively. All are preset thresholds.

[0026] Step 2: Represent the aesthetic perception map corresponding to each poster as a fusion matrix, and train the convolutional neural network based on the fusion matrix to obtain the trained convolutional neural network; the convolutional neural network includes an encoder and a decoder.

[0027] For any poster containing N design elements, let the first element be the first element. A poster For example, an aesthetic perception map can be represented as a matrix. Regarding the first There are 1 design element, and the element matrix is ​​1. , for the Design Relationships The relation matrix is Then the first A poster The corresponding aesthetic perception map is represented by a fusion matrix. :

[0028] in, Indicates splicing.

[0029] Here, the convolutional neural network has an encoder-decoder structure, and the fusion matrix... Input to encoder In the process, obtain intermediate feature representations. intermediate feature representation Input into the decoder The reconstructed fusion matrix is ​​obtained. .

[0030] Then, by constructing the loss function Training a convolutional neural network This represents the reconstruction loss, resulting in the trained convolutional neural network.

[0031] Step 3: Input the fusion matrix into the trained convolutional neural network, and the encoder outputs intermediate feature vectors; perform k-means clustering on all intermediate feature vectors to obtain multiple subclasses, each subclass containing multiple intermediate feature vectors; integrate the aesthetic perception maps corresponding to all intermediate feature vectors in each subclass into an aggregated knowledge graph.

[0032] Here, in response to user inputs of varying complexity and greater diversity, aesthetic perception graphs belonging to the same subclass are integrated into a larger graph based on associations of the same design elements. An aggregate knowledge graph (AKG) is constructed for each class, which contains both explicit relationships existing in a poster and potential relationships mixed between different posters.

[0033] Step 4: Represent the aesthetic perception map corresponding to each poster as an aesthetic perception embedding code, and train the Transformer encoder based on the aesthetic perception embedding code to obtain the trained Transformer encoder.

[0034] To enable the model to better understand aesthetically plural (AG) maps and generate layouts, each AG is first unfolded into a sequence, allowing the Transformer encoder to perform computation and inference. Specifically, for the graph... , ,in Represents a set of nodes. Represents the set of edges. For a given set containing... Graph with stripes It can be represented as a random sequence of edges, where Representing the Edge, and For the edge The subject and object of the link, diagram The corresponding expanded sequence T is:

[0035] Next, by Add the separator | to construct the input sequence of the transformer. as follows:

[0036] For the input sequence The word embeddings of semantic features of design elements are extracted using a pre-trained BERT model, which yields the aesthetic perception embedding encoding.

[0037] The aesthetic perception embedding code is input into the Transformer encoder (position predictor) to obtain the poster layout, which includes the coordinates, height and width of each element.

[0038] Here, to evaluate the difference between the generated poster layout and the real poster layout, a loss function is needed to guide model training. The loss function used during training is... for:

[0039] in, To rebuild the losses, To combat the losses, This is a hyperparameter used to adjust the weights of reconstruction loss and adversarial loss in the final loss.

[0040] By continuously optimizing this loss function, the generated layout becomes closer to the actual layout. This is achieved by calculating the loss function. The generator (Transformer encoder) is adjusted based on feedback to improve the quality of the generated layout.

[0041] Step 5: Perform a random walk on each aggregated knowledge graph, starting from the user input element, and expand the nodes and edges in the aggregated knowledge graph to obtain a subgraph of the aggregated knowledge graph; integrate all the subgraphs of the aggregated knowledge graph to obtain the total content expansion.

[0042] Specifically, a random walk is performed on each aggregated knowledge graph, and the condition for the extension to terminate is determined as follows: when When the extension ends, it is determined that the extension has ended; among them, For the calculation of the score in step t, The score is calculated in step t+1;

[0043] in, To aggregate knowledge graphs after the first... The walking sequence after step expansion, For the first The aesthetic perception diagram unfolds in sequence corresponding to each poster. Number of posters; The ratio of the most similar common strings between two sequences.

[0044] Here, a self-constrained strategy is proposed to adaptively determine when to complete the expansion process. The higher, The more consistent it is with common sense in the real world of design, the better.

[0045] Step 6: Expand the total content and input it into the trained Transformer encoder to generate a poster layout, which includes the coordinates, height, and width of each element.

[0046] Step 7: Determine if the user has adjusted the poster layout. If not, the current poster layout is the final poster layout. If yes, generate the adjusted poster layout, return to step 5, and use the adjusted poster layout as the new user input element.

[0047] This embodiment first quantifies the relationships between design elements and constructs a structured aesthetic perception graph (AG) based on the principles of alignment, repetition, superposition, and proximity in art and design theory, transforming complex design patterns into a computable data model. Second, it performs pattern mining on the AG using clustering algorithms, integrates multi-source design data to construct an aggregated knowledge graph (AKG), and merges explicit layout rules with potential cross-graph associations to provide a reference framework for diverse designs. Building upon this, a self-constrained random walk strategy is proposed to dynamically expand design content based on user input, and adaptively controls the expansion boundary through similarity scoring to ensure content rationality. Finally, an aesthetic perception layout generator is designed based on the Transformer architecture, achieving accurate mapping from element sequences to high-quality layouts through aesthetic perception embedding encoding and design perception loss function optimization. This embodiment overcomes the limitations of traditional template-based design, enabling non-professional users to quickly generate aesthetically pleasing and personalized design schemes with minimal input, significantly lowering the design threshold and improving efficiency.

[0048] Based on the same inventive concept as the end-to-end intelligent graphic design method with human-computer interaction, this embodiment also provides a corresponding end-to-end intelligent graphic design system with human-computer interaction, including: The aesthetic perception map generation module is used to obtain the training dataset, in which the samples are posters containing multiple elements; for each poster in the training dataset, the relationship between elements is quantified, including alignment, repetition, superposition and proximity, to obtain the aesthetic perception map corresponding to the poster. The first training module is used to represent the aesthetic perception map corresponding to each poster as a fusion matrix, and to train the convolutional neural network based on the fusion matrix to obtain the trained convolutional neural network; the convolutional neural network includes an encoder and a decoder. The classification and aggregation knowledge graph production module is used to input the fusion matrix into the trained convolutional neural network, and the encoder outputs intermediate feature vectors; k-means clustering is performed on all intermediate feature vectors to obtain multiple subclasses, each of which includes multiple intermediate feature vectors; and the aesthetic perception graphs corresponding to all intermediate feature vectors in each subclass are integrated into an aggregation knowledge graph. The second training module is used to represent the aesthetic perception map corresponding to each poster as an aesthetic perception embedding code, and to train the Transformer encoder based on the aesthetic perception embedding code to obtain the trained Transformer encoder. The content expansion module is used to perform a random walk on each aggregated knowledge graph, starting from the user input element, to expand the nodes and edges in the aggregated knowledge graph to obtain a subgraph of the aggregated knowledge graph; and to integrate the subgraphs of all aggregated knowledge graphs to obtain the total content expansion. The poster layout generation module is used to expand the total content input to the trained Transformer encoder and generate a poster layout, which includes the coordinates, height and width of each element. The judgment module is used to determine whether the user has adjusted the poster layout. If not, the current poster layout is the final poster layout. If so, the adjusted poster layout is generated and returned to the content expansion module, which uses the adjusted poster layout as a new user input element.

[0049] The full-process intelligent graphic design system for human-computer interaction in this embodiment has the same inventive concept as the full-process intelligent graphic design method for human-computer interaction described above. Therefore, the specific implementation of this system can be found in the embodiment section of the full-process intelligent graphic design method for human-computer interaction described above, and its technical effects correspond to the technical effects of the above method, so it will not be repeated here.

[0050] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the above-described intelligent graphic design method for the entire process of human-computer interaction.

[0051] This application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the aforementioned intelligent graphic design method for the entire process of human-computer interaction.

[0052] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A fully interactive intelligent graphic design method, characterized in that, include: Step 1: Obtain the training dataset, wherein the samples in the training dataset are posters containing multiple elements; For each poster in the training dataset, the relationships between elements are quantified, including alignment, repetition, overlay, and proximity, to obtain the aesthetic perception map corresponding to the poster; Step 2: Represent the aesthetic perception map corresponding to each poster as a fusion matrix, and train the convolutional neural network based on the fusion matrix to obtain the trained convolutional neural network; the convolutional neural network includes an encoder and a decoder; Step 3: Input the fusion matrix into the trained convolutional neural network, and the encoder outputs the intermediate feature vector; K-means clustering is performed on all intermediate feature vectors to obtain multiple subclasses, each of which includes multiple intermediate feature vectors; the aesthetic perception graphs corresponding to all intermediate feature vectors in each subclass are integrated into an aggregated knowledge graph. Step 4: Represent the aesthetic perception map corresponding to each poster as an aesthetic perception embedding code, and train the Transformer encoder based on the aesthetic perception embedding code to obtain the trained Transformer encoder. Step 5: Perform a random walk on each of the aggregated knowledge graphs, starting from the user input element, and expand the nodes and edges in the aggregated knowledge graph to obtain a subgraph of the aggregated knowledge graph; Integrate all subgraphs of the aggregated knowledge graph to obtain the total expanded content; Step 6: Expand the total content and input it into the trained Transformer encoder to generate a poster layout, which includes the coordinates, height and width of each element. Step 7: Determine whether the user has adjusted the poster layout. If not, the current poster layout is the final poster layout. If yes, generate the adjusted poster layout, return to step 5, and use the adjusted poster layout as a new user input element.

2. The method as described in claim 1, characterized in that, in, A random walk is performed on each of the aggregated knowledge graphs, and the condition for the extension to end is determined as follows: when When the extension ends, it is determined that the extension has ended; among them, For the calculation of the score in step t, The score is calculated in step t+1; in, To aggregate knowledge graphs after the first... The walking sequence after step expansion, For the first The aesthetic perception diagram unfolds in sequence corresponding to each poster. Number of posters; The ratio of the most similar common strings between two sequences.

3. A fully intelligent graphic design system with human-computer interaction, characterized in that, include: An aesthetic perception map generation module is used to obtain a training dataset, wherein the samples in the training dataset are posters containing multiple elements. For each poster in the training dataset, the relationships between elements are quantified, including alignment, repetition, overlay, and proximity, to obtain the aesthetic perception map corresponding to the poster; The first training module is used to represent the aesthetic perception map corresponding to each poster as a fusion matrix, and to train the convolutional neural network based on the fusion matrix to obtain the trained convolutional neural network; the convolutional neural network includes an encoder and a decoder. The classification and aggregation knowledge graph production module is used to input the fusion matrix into the trained convolutional neural network, and the encoder outputs the intermediate feature vector; K-means clustering is performed on all intermediate feature vectors to obtain multiple subclasses, each of which includes multiple intermediate feature vectors; the aesthetic perception graphs corresponding to all intermediate feature vectors in each subclass are integrated into an aggregated knowledge graph. The second training module is used to represent the aesthetic perception map corresponding to each poster as an aesthetic perception embedding code, and to train the Transformer encoder based on the aesthetic perception embedding code to obtain the trained Transformer encoder. The content expansion module is used to perform a random walk on each of the aggregated knowledge graphs, starting from the user input element, to expand the nodes and edges in the aggregated knowledge graphs to obtain subgraphs of the aggregated knowledge graphs; Integrate all subgraphs of the aggregated knowledge graph to obtain the total expanded content; The poster layout generation module is used to expand the total content input to the trained Transformer encoder to generate a poster layout, wherein the poster layout includes the coordinates, height and width of each element; The judgment module is used to determine whether the user has adjusted the poster layout. If not, the current poster layout is the final poster layout. If so, the adjusted poster layout is generated and returned to the content expansion module, which uses the adjusted poster layout as a new user input element.

4. The system as described in claim 3, characterized in that, in, A random walk is performed on each of the aggregated knowledge graphs, and the condition for the extension to end is determined as follows: when When the extension ends, it is determined that the extension has ended; among them, For the calculation of the score in step t, The score is calculated in step t+1; in, To aggregate knowledge graphs after the first... The walking sequence after step expansion, For the first The aesthetic perception diagram unfolds in sequence corresponding to each poster. Number of posters; The ratio of the most similar common strings between two sequences.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the full-process intelligent planar design method for human-computer interaction as described in any one of claims 1-2.

6. A computer program product, characterized in that, It includes a computer program / instruction, which, when executed by a processor, implements the full-process intelligent planar design method for human-computer interaction as described in any one of claims 1-2.

Images