Smart card layout generation method and device, storage medium and computer equipment

Abstract

The invention relates to the technical field of smart cards, and particularly provides a smart card layout generation method and device, a storage medium and computer equipment. According to the method, the intelligent card field knowledge graph can be pre-constructed to describe the incidence relation between the theme keyword and at least one type of keyword in the style keyword, the visual element keyword, the description element keyword and the color keyword. In the process of generating the intelligent card layout, firstly, a layout description text input by a user is obtained, and an associated element text associated with the layout description text is inquired in an intelligent card domain knowledge graph; and then performing text expansion on the layout description text and the associated element text by utilizing the target large language model, and performing text generation on the basis of a target prompt text obtained by expansion to obtain an intelligent card layout image. Through dynamic retrieval and frame synthesis, the method can reduce the use threshold of the user, has the advantage of high practicability, can improve the intelligent degree and the automation degree, and improves the layout generation efficiency.

Description

Technical Field

[0001] This application relates to the field of smart card technology, and in particular to a method, apparatus, storage medium and computer equipment for generating smart card layouts. Background Technology

[0002] With the widespread application of AIGC technology in graphic design, text-driven image generation tools such as MidJourney and DALL-E have been applied to smart card layout design. When generating a smart card layout, users can input descriptive text as prompts into the image generation tool. The tool then uses this descriptive text to perform reasoning, converting text into an image and ultimately outputting the smart card layout. In this process, the aesthetic appeal and suitability of the smart card layout are closely related to the descriptive text, requiring users to possess certain artistic expertise and layout design experience to create effective prompts. Furthermore, users often need to manually adjust the descriptive text multiple times to obtain a smart card layout image that meets their requirements.

[0003] It is evident that existing technologies not only limit the use by non-professionals, resulting in a high barrier to entry for users, but also heavily rely on manual editing and adjustment of prompts, leading to a low level of intelligence and reduced efficiency in layout generation. Summary of the Invention

[0004] The purpose of this application is to at least address one of the aforementioned technical defects, particularly the technical defects in the existing technology that restrict the use by non-professionals, resulting in a high user threshold, and that heavily rely on manual editing and adjustment of prompts, leading to a low level of intelligence and reduced efficiency in page generation.

[0005] In a first aspect, embodiments of this application provide a method for generating a smart card layout, including:

[0006] Get the layout description text entered by the user;

[0007] Using the layout description text as the query keyword, a pre-constructed smart card domain knowledge graph is queried to obtain related element text; wherein, the smart card domain knowledge graph is used to describe the relationship between multiple entities, the multiple entities include topic keywords, and the multiple entities also include at least one of style keywords, visual element keywords, descriptive element keywords, and color keywords;

[0008] The initial prompt text is obtained by concatenating the layout description text and the associated element text.

[0009] The initial prompt text is input into the target large language model, so that the target large language model expands the initial prompt text and outputs the target prompt text;

[0010] The target prompt text is input into the text-to-image model to obtain the smart card layout image.

[0011] In some embodiments, the text-based image model is a diffusion model, and the training process of the text-based image model includes:

[0012] Obtain the attention intensity obtained by the diffusion model during the current training round;

[0013] Based on the attention intensity of the current training round and the preset information retention area location, calculate the average attention intensity of each information retention area in the current training round;

[0014] The spatial constraint loss value for the current training round is determined based on the average attention intensity of each information retention region in the current training round.

[0015] Based on the spatial constraint loss value of the current training round, calculate the total loss value of the current training round;

[0016] Determine whether the preset training termination condition is met based on the total loss value;

[0017] If the conditions are met, the text-based graph model is obtained; otherwise, proceed to the next training round.

[0018] In some embodiments, inputting the target prompt text into the text image model and obtaining the smart card layout image includes:

[0019] The target prompt text is input into the text-to-image model to obtain the initial layout image output by the text-to-image model;

[0020] According to the preset information retention area position, the image located in the information retention area in the initial layout image is erased to obtain the erased image;

[0021] The smart card layout image is generated based on the erased image.

[0022] In some embodiments, the process of constructing the smart card domain knowledge graph includes:

[0023] Obtain multiple historical page layout images, and obtain the theme attribute association relationship for each of the historical page layout images; the theme attribute association relationship includes theme keywords and attribute keywords, and the attribute keywords include at least one of style keywords, visual element keywords, descriptive element keywords, and color keywords;

[0024] Based on the relationship between the various topic attributes, determine the correlation strength between each keyword;

[0025] Based on the relationship between the various topic attributes and the strength of the relationship between the various keywords, a Neo4j graph database is constructed using keywords as nodes to obtain the smart card domain knowledge graph.

[0026] In some embodiments, the smart card domain knowledge graph indexes each topic keyword and each visual element keyword.

[0027] In some embodiments, the construction process of the target large language model includes:

[0028] Obtain a pre-trained large language model;

[0029] The LORA algorithm is used to fine-tune the pre-trained large language model to obtain the target large language model.

[0030] In some embodiments, concatenating the layout description text and the associated element text to obtain the initial prompt text includes:

[0031] The layout description text, the associated element text, and the preset behavior description text are concatenated to form the initial prompt text; wherein, the behavior description text is used to instruct the large model to design the smart card layout and output the smart card layout image.

[0032] Secondly, embodiments of this application provide a smart card layout generation device, comprising:

[0033] The text acquisition module is used to acquire the layout description text input by the user;

[0034] The query module is used to query the pre-constructed smart card domain knowledge graph using the layout description text as the query keyword, and obtain the associated element text; wherein, the smart card domain knowledge graph is used to describe the relationship between multiple entities, the multiple entities include topic keywords, and the multiple entities also include at least one of style keywords, visual element keywords, descriptive element keywords, and color keywords;

[0035] The splicing module is used to splice the layout description text and the associated element text to obtain the initial prompt text;

[0036] The text expansion module is used to input the initial prompt text into the target large language model, so that the target large language model expands the initial prompt text and outputs the target prompt text.

[0037] The layout image generation module is used to input the target prompt text into the text-generated image model and obtain the smart card layout image.

[0038] Thirdly, embodiments of this application provide a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the smart card layout generation method described in any of the above embodiments.

[0039] Fourthly, embodiments of this application provide a computer device, which includes: one or more processors, and a memory;

[0040] The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the smart card layout generation method described in any of the above embodiments.

[0041] In the smart card layout generation method, apparatus, storage medium, and computer equipment provided in some embodiments of this application, a smart card domain knowledge graph can be pre-constructed to describe the association relationships between at least one type of keyword, including topic keywords, style keywords, visual element keywords, descriptive element keywords, and color keywords, thereby modeling the association relationships between the layout theme and layout design constraints. During the smart card layout generation process, this application first obtains the layout description text input by the user and queries the smart card domain knowledge graph to find associated element texts related to the layout description text. The layout description text and associated element texts are then concatenated to form an initial prompt text, thereby expanding the elements related to the theme involved in the layout description text. After obtaining the initial prompt text, this application can use a target large language model to expand the initial prompt text and generate a text-to-image based on the expanded target prompt text to obtain the smart card layout image. Through a dynamic retrieval and synthesis framework, this application can intelligently process the expansion of prompt words from specific topics to broad abstract topics, thereby automatically expanding short layout description texts into rich and professional detailed prompt words. In this way, users can obtain a high-quality smart card layout image that meets their needs by inputting a short layout description text, which can lower the user threshold, has the advantage of being highly practical, and can improve the level of intelligence and automation, and improve the efficiency of layout generation. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating the smart card layout generation method in some embodiments;

[0044] Figure 2 This is a schematic diagram of the structure of the smart card layout generation device in some embodiments;

[0045] Figure 3 This is a diagram of the internal structure of a computer device in some embodiments. Detailed Implementation

[0046] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0047] The technical terms used in this application are explained below.

[0048] NLP: Natural Language Processing is a technology that enables computers to understand, interpret, and manipulate human language.

[0049] CV: Computer Vision is a technology that enables computers to gain a high level of understanding from digital images or videos.

[0050] AIGC: Artificial Intelligence Generated Content, refers to content such as text, images, audio, and video that are automatically generated using artificial intelligence technology.

[0051] Diffusion Model: An advanced generative AI model that generates high-quality images from random noise through a stepwise denoising process.

[0052] Layout Constraint: Layout constraints refer to mandatory rules regarding the position, size, font, etc. of specific elements (such as text, logo) in an image.

[0053] BK-KG: Knowledge Graph in the Bank Card Domain. It refers to a structured knowledge system built for bank card design scenarios to support the accurate generation of prompt words.

[0054] The method for generating smart card layouts provided in this application is described below.

[0055] In some embodiments, this application provides a method for generating a smart card layout. The following embodiments illustrate this method by applying it to a computer device. It is understood that the computer device involved in this application refers to a device with data input and data processing functions, which may be, but is not limited to, tablet computers, laptops, desktop computers, server clusters, etc. This application does not impose specific limitations on this.

[0056] like Figure 1 As shown, the smart card layout generation method provided in this application may include the following steps:

[0057] S102: Obtain the layout description text input by the user.

[0058] The layout description text can be used to describe the design theme of the smart card layout to be generated. A smart card layout refers to the layout design of a smart card, which may include elements such as the card image, card colors, card text, and chip / magnetic stripe location. A smart card is a card with an embedded microchip, and may include, but is not limited to, bank cards, transportation cards, access control cards, etc. For ease of explanation, this application uses a bank card as an example for description.

[0059] In this step, the computer device can receive layout description text input by the user to understand the user's requirements for the smart card layout design. For example, the layout description text could be "Mid-Autumn Festival in the Year of the Snake," which indicates that the user wants a layout image that includes elements of the Year of the Snake and the Mid-Autumn Festival.

[0060] S104: Using the layout description text as the query keyword, query the pre-constructed smart card domain knowledge graph and obtain the associated element text; wherein, the smart card domain knowledge graph is used to describe the relationship between multiple entities, and the multiple entities include topic keywords, and the multiple entities also include at least one of style keywords, visual element keywords, descriptive element keywords, and color keywords.

[0061] The theme keywords refer to the design theme around which the layout is designed, including but not limited to festival keywords, zodiac keywords, city keywords, and cultural IP keywords. Festival keywords can include Spring Festival, Labor Day, Mid-Autumn Festival, and National Day, zodiac keywords can include each of the twelve zodiac animals, city keywords can include Beijing, Shanghai, Guangzhou, Shenzhen, Xi'an, etc., and cultural IPs can include Dunhuang, anime names, etc.

[0062] Style keywords refer to the artistic style involved in the layout design, such as Chinese style, ink painting, cyberpunk, cartoon style, oil painting style, minimalism, and meticulous brushwork. Visual element keywords refer to the pattern and texture elements present in the smart card layout. Pattern elements may include jade rabbits, mooncakes, osmanthus flowers, auspicious clouds, dragon patterns, and dragon heads; texture elements may include meander patterns and crackle patterns. Descriptive element keywords are adjectives associated with a specific theme, style, or element, such as reunion and joy, a perfect full moon, a bright moon in the sky, winding, dynamic, majestic, ancient city, ancient capital, the long river of history, a dazzling pearl, romantic, and weathered. Color element keywords refer to the colors involved in the layout design, such as vermilion, dark green, and gilded.

[0063] It is understandable that a topic can have multiple related elements (i.e., multiple attributes) to describe attributes such as artistic style, visual elements, adjectives, and colors under that topic. Similarly, an artistic style can have multiple related elements to describe attributes such as visual elements, artistic style, adjectives, and colors under that artistic style. For example, the related elements corresponding to the style keyword "traditional Chinese style" may include auspicious clouds, meander patterns, bright red, bright yellow, and meticulous brushwork lines.

[0064] In this step, the computer equipment, with the entire smart card design process constraints as its core, pre-constructs a smart card domain knowledge graph. This graph describes the relationships between at least one type of keyword: theme keywords, style keywords, visual element keywords, descriptive element keywords, and color keywords. During the generation of the smart card layout design image, the computer equipment uses the user-input layout description text as query keywords to search the smart card domain knowledge graph and obtain elements associated with the layout description text, thus retrieving the associated element text.

[0065] For example, when the page description text is "Mid-Autumn Festival", the retrieved related element texts could be: Jade Rabbit, Mooncake, Osmanthus, Golden Yellow, Vermilion, Traditional Chinese Style. Similarly, when the page description text is "Year of the Snake", the retrieved related element texts could be: Snake-shaped Auspicious Cloud Pattern, Simple Snake Outline, Fine Brushwork, Lines.

[0066] In some examples, when the length of the layout description text exceeds a preset length threshold, this application can perform keyword extraction on the layout description text to extract descriptive keywords, which can then be used as query keywords to search the smart card domain knowledge graph. For example, if the layout description text is: "Please help generate a smart card layout with the theme of the Mid-Autumn Festival in the Year of the Snake," then "Mid-Autumn Festival in the Year of the Snake" can be extracted as a descriptive keyword, and this keyword can be used to search the smart card domain knowledge graph to obtain the corresponding related element text.

[0067] S106: Concatenate the layout description text and related element text to obtain the initial prompt text.

[0068] In this step, the user-input layout description text and the related element text retrieved from the smart card domain knowledge graph are concatenated to obtain the initial prompt text. It can be understood that the initial prompt text includes both the layout description text and the related element text.

[0069] In some examples, S106 may include: concatenating the layout description text, associated element text, and preset behavior description text into an initial prompt text; wherein the behavior description text is used to instruct the large model to design the smart card layout and output the smart card layout image.

[0070] For example, when generating a bank card layout, the action description text could be: "Please generate a bank card layout design prompt." Similarly, when generating a transportation card layout, the action description text could be: "Please generate a layout design prompt for the transportation card in City A."

[0071] In this example, the layout description text, related element text, and behavior description text can be concatenated to obtain the initial prompt text. Thus, the initial prompt text can simultaneously include the user's requirements for the smart card layout design, relevant elements under the layout design theme, and behavioral instructions for the large model. This allows the output of the large language model to better match the layout image generation needs, further improving the level of intelligence and generation efficiency.

[0072] For example, when the layout description text is "Mid-Autumn Festival in the Year of the Snake", the associated element text is "Associated elements: Jade Rabbit, Osmanthus, Golden Yellow, Traditional Chinese Style", and the behavior description text is "Please generate bank card design prompts", the initial prompt text obtained after text concatenation could be "Theme: Mid-Autumn Festival in the Year of the Snake. Associated elements: Jade Rabbit, Osmanthus, Golden Yellow, Traditional Chinese Style. Please generate bank card design prompts".

[0073] S108: Input the initial prompt text into the target large language model so that the target large language model can expand the initial prompt text and output the target prompt text.

[0074] In this step, given the initial prompt text, it can be used as input data for the target large language model. Upon receiving the initial prompt text, the target large language model can perform inference based on it, expand the text, and output the expanded prompt text (i.e., the target prompt text). In some examples, the target prompt text may be structured prompt words.

[0075] For example, when the initial prompt text is "Theme: Mid-Autumn Festival in the Year of the Snake. Related elements: Jade Rabbit, Osmanthus, Golden Yellow, Traditional Chinese Style. Please generate prompt text for bank card design", the target prompt text output by the target large language model can be "Traditional Chinese style bank card, background is a full moon in the night sky, the foreground right side is a golden snake coiled around, the lower left corner is decorated with osmanthus and lanterns, the overall color scheme is golden yellow and dark green, the card number area is located at the bottom center with clear space, and the font is bold Songti".

[0076] Thus, this application can achieve the triple constraints of "topic-function-style" through the knowledge graph of the smart card domain and the target large language model, and can generate prompt words that can be directly used to drive the text-generated graph model based on the short layout description text input by the user. This process does not require manual intervention.

[0077] S110: Input the target prompt text into the text image model and obtain the smart card layout image.

[0078] The text-to-image model refers to a large model that takes text as input and outputs an image. It can perform inference based on the received text and output an image corresponding to the input text. In this step, the target prompt text, after text expansion, can be input into the text-to-image model so that it can perform inference and output a smart card layout image that matches the user's needs.

[0079] Through a dynamic retrieval and synthesis framework, this application can intelligently process the expansion of prompt words from specific topics to broad abstract topics, thereby automatically expanding short layout description text into rich and professional detailed prompt words. In this way, users can obtain a high-quality smart card layout image that meets their needs simply by inputting a short layout description text, thus lowering the user threshold, offering strong practicality, and improving the level of intelligence and automation, thereby increasing layout generation efficiency.

[0080] In some embodiments, the text-based image model is a diffusion model, and the training process of the text-based image model includes:

[0081] Obtain the attention intensity obtained by the diffusion model during the current training round;

[0082] Based on the attention intensity of the current training round and the preset information retention area location, calculate the average attention intensity of each information retention area in the current training round;

[0083] The spatial constraint loss value for the current training round is determined based on the average attention intensity of each information retention area in the current training round.

[0084] Calculate the total loss value for the current training round based on the spatial constraint loss value.

[0085] Determine whether the preset training termination condition is met based on the total loss value;

[0086] If the conditions are met, the text-based graph model is obtained; otherwise, proceed to the next training round.

[0087] Specifically, the layout design of smart cards typically needs to meet functional compliance requirements. Taking bank cards as an example, the image on a bank card needs to display functional areas such as the issuing bank, card number, and signature strip in specific locations, and its layout design must comply with regulations such as the "Bank Card Specification." However, existing smart card layout image generation solutions do not incorporate the constraints of the smart card field, resulting in smart card layout images that do not meet the compliance requirements for smart cards (e.g., the card number area is covered by an image, or the image overlaps with key information), rendering the layout image unusable in practice.

[0088] To address this issue, this embodiment introduces a spatial layout constraint mechanism during image generation. By employing the attention control mechanism within the deep diffusion model's denoising process, the text-based image model automatically avoids preset information retention areas when generating smart card layout images. This fundamentally ensures that the smart card layout images meet the rigid layout requirements of smart cards, thereby improving the practicality and compliance of the smart card layout images. This further enhances the level of intelligence and automation, ultimately increasing layout generation efficiency.

[0089] Specifically, this application uses a diffusion model as the text-to-image model. During the denoising process of the diffusion model, spatial layout constraints are injected by modifying the attention map and / or loss function. The diffusion model can learn region suppression and guidance mechanisms through model training. When generating the smart card layout image, the trained diffusion model (i.e., the text-to-image model) can simultaneously perform content generation and layout compliance, ensuring that the generated smart card layout image automatically avoids the preset information retention area.

[0090] In this embodiment, a layout mask can be predefined. The layout mask includes design templates for various types of smart cards and clearly identifies the information storage area (i.e., the pattern generation restricted area) and the pattern preference area for each type of smart card. The information storage area refers to the area in the smart card used to implement specific functions, such as an information recording area or a magnetic stripe / chip setting area. The information recording area includes, but is not limited to, the card number area, the name area, the signature strip area, and the bank logo placement area.

[0091] During the training of the diffusion model, this application can obtain the attention intensity obtained by the diffusion model in the current training round, and calculate the average attention intensity corresponding to each information retention region based on the attention intensity of the current training round. Then, based on the average attention intensity of all information retention regions, the spatial constraint loss value of the current training round is calculated. For example, the average attention intensity of all information retention regions can be weighted and summed or accumulated to obtain the spatial constraint loss value.

[0092] In some examples, the spatial constraint loss value can be expressed as expression L. constraint =λ×Σ R max(0,S R -τ) is calculated to give R = 1, 2, ..., n. Where L constraint Let S be the spatial constraint loss value, λ be the penalty weight, and S be the spatial constraint loss value. R Let λ be the average attention intensity of the Rth information retention region, n be the total number of information retention regions, and τ be the attention threshold. It is understood that the specific values ​​of λ and τ can be determined based on the actual situation, and this application does not impose specific restrictions on them. For example, λ can be 1.0, and τ can be 0.1 to indicate that the attention intensity of the restricted area should not be too high.

[0093] This application uses the spatial constraint loss value as a penalty to determine the total loss value for the current training epoch. In some examples, the total loss value can be the sum of the original denoising loss value and the spatial constraint loss value. The original denoising loss value can be calculated using any existing denoising loss function and is used to characterize the degree of difference between the denoising result of the current training epoch and the training samples. This application can adjust the model parameters of the diffusion model based on the total loss value until a preset training termination condition is met, resulting in the final usable Wensheng graph model.

[0094] When the image generated by the diffusion model produces a high-activation attention map within the information retention area (i.e., the model's view generates pattern content in that area), the spatial constraint loss value increases. This penalizes the model for generating high-attention content within the image generation forbidden zone, guiding the diffusion model to reduce the activation in that area during the next denoising step and instead generate content within the preferred pattern generation area. In this way, the diffusion model can automatically learn to guide pattern content generation towards the preferred pattern generation area, thus enabling the automatic generation of smart card layout images that meet compliance requirements through the text-based image model.

[0095] In some examples, during each denoising step of the diffusion model, the computer device can calculate the correlation between the cue words input to the diffusion model and the image regions. Spatial constraints are enhanced by modifying the cross-attention mechanism of the diffusion model.

[0096] In some embodiments, the target prompt text is input into the text-to-image model to obtain a smart card layout image, including:

[0097] Input the target prompt text into the text-to-image model to obtain the initial layout image output by the text-to-image model;

[0098] Based on the preset information retention area location, the image located in the information retention area of ​​the initial layout image is erased to obtain the erased image;

[0099] Generate a smart card layout image based on the erased image.

[0100] In this embodiment, to ensure that the smart card layout image meets the rigid layout requirements of the smart card, and to further improve the level of intelligence, automation and layout generation efficiency, this embodiment can use a post-generation repair method to obtain the smart card layout image.

[0101] Specifically, this application can use the target prompt text as a prompt word in the text-to-image model, inputting it into the text-to-image model so that the model can perform inference based on the target prompt text and output an initial layout image. Once the initial layout image is obtained, this application can employ image erasure technology to erase the pattern located in the information retention area while retaining the pattern in the non-information retention area, thereby obtaining an erased image, and thus obtaining the smart card layout image.

[0102] In some examples, this application can erase the image as the smart card layout image. Alternatively, this application can use the erased image as a basis to add functional patterns such as the bank's logo or signature strip pattern to the information retention area to obtain the smart card layout image.

[0103] In some embodiments, the process of constructing a smart card domain knowledge graph includes:

[0104] Obtain multiple historical page images, and obtain the theme attribute association relationship for each historical page image; the theme attribute association relationship includes theme keywords and attribute keywords, and attribute keywords include at least one of the following: style keywords, visual element keywords, descriptive element keywords, and color keywords;

[0105] Based on the relationships between various thematic attributes, determine the strength of association between each keyword;

[0106] Based on the relationships between various topic attributes and the strength of associations between various keywords, a Neo4j graph database is constructed using keywords as nodes to obtain a knowledge graph for the smart card domain.

[0107] In this embodiment, a Neo4j graph database can be constructed based on the thematic attribute relationships contained in multiple historical page images to support visual display and efficient querying.

[0108] Specifically, the process of constructing a knowledge graph in the smart card field can include a data acquisition sub-step, a knowledge fusion and graph construction sub-step, and a batch import sub-step. The data acquisition sub-step involves acquiring multiple historical layout images, all of which represent compliant smart card layout designs. The thematic attribute relationships of each historical layout image can be extracted using methods such as multimodal large-scale models and large language model platforms.

[0109] In some examples, during the data acquisition sub-step, this application may first acquire 1000 historical layout images and use a multimodal large model, such as qwen2.5-VL-72B, to generate image-to-text descriptions for each historical layout image. Then, this application can generate design theme keywords for each historical layout image based on its image description text using a general large language model. Furthermore, it can generate visual element keywords corresponding to each design theme keyword using the general large language model and a Chinese traditional culture database. Further, to improve the text quality of the image description text, this application can manually clean and rearrange the image description text before generating the design theme keywords using the large language model.

[0110] In the knowledge fusion and graph construction sub-step, this application can systematically organize the structured knowledge obtained from the data acquisition sub-step, as well as the pre-set entity types and relationship types, to obtain a queryable, reasonable, and scalable graphical knowledge network (i.e., a smart card domain knowledge graph), which serves as the domain brain for subsequent prompt word generation.

[0111] The preset entity types can include theme keywords, style keywords, visual element keywords, descriptive element keywords, and color keywords. Relationship types can include relationships from theme keywords to visual elements, from theme keywords to style keywords, from theme keywords to descriptive element keywords, from theme keywords to color keywords, from style keywords to color keywords, and from visual element keywords to style keywords, etc.

[0112] When creating a Neo4j graph database, it's necessary to standardize the structured knowledge obtained from the data collection sub-steps into a consistent format. For example, a structured CSV (Comma-Separated Values) file can be used to record the thematic attribute relationships obtained during the collection process. A CSV file can include three columns: the subject term "head", the relation "relation", and the attribute term "tail". Taking the thematic attribute relationship "Mid-Autumn Festival—Golden Yellow" as an example, in the same row of the CSV file, it can be recorded as: head="Mid-Autumn Festival", relation="has_color", tail="Golden Yellow".

[0113] Furthermore, in Neo4j, nodes represent entities (i.e., various keywords), such as "Mid-Autumn Festival" and "Jade Rabbit". Computer devices can create nodes for each entity, and each node can include a name field, a type field, a description field, and a confidence_score field to facilitate subsequent queries and sorting, thereby improving the retrieval speed of the smart card domain knowledge graph and thus increasing the efficiency of smart card layout image generation.

[0114] For example, the node corresponding to the entity "Mid-Autumn Festival" can be created using the command `CREATE (:theme{name: 'Mid-Autumn Festival', type: 'Festival', description: 'The 15th day of the eighth lunar month, the Reunion Festival', confidence_score:0.95})`. Similarly, the node corresponding to "Jade Rabbit" can be created using the command `CREATE (:visual element{name: 'Jade Rabbit', type: 'pattern', description: 'The rabbit in the moon palace, symbolizing good fortune', confidence_score: 0.9})`. For example, the node corresponding to "Traditional Chinese Style" can be created using the command CREATE (:Art Style{name: 'Traditional Chinese Style', type: 'Style', description: 'Gongbi, Shuimo, Xiangyun Pattern', confidence_score: 0.98}); the node corresponding to "Flowers in Full Bloom and Full Moon" can be created using the command CREATE (:Description Element{name: 'Flowers in Full Bloom and Full Moon', type: 'Idiom', description: 'Symbolizing Reunion and Happiness', confidence_score: 0.92}).

[0115] After nodes are established, computer devices can connect them using relationships to form a network. The connection of nodes is based on the topic attribute relationships obtained from the data acquisition sub-step. By constructing relationships, a radial knowledge network centered on topic nodes can be formed.

[0116] For example, when the collected theme attribute relationships include Mid-Autumn Festival—Jade Rabbit and Mid-Autumn Festival—Golden Yellow, the MATCH (t:theme{name: 'Mid-Autumn Festival'}), (e:visual element{name: 'Jade Rabbit'}) and CREATE (t)-[:has_element {strength: 0.9}]->(e) instructions can be used to connect the "Mid-Autumn Festival" node and the "Jade Rabbit" node, and the MATCH (t:theme{name: 'Mid-Autumn Festival'}), (c:color{name: 'Golden Yellow'}) and CREATE (t)-[:has_color{strength: 0.85}]->(c) instructions can be used to connect the "Mid-Autumn Festival" node and the "Golden Yellow" node.

[0117] In the Neo4j graph database, each relationship has a correlation strength, which represents the reliability of the association between the two nodes connected by that relationship. This strength can be determined based on the collinearity frequency of keywords or a preset scoring mechanism. This facilitates subsequent sorting and querying.

[0118] In some embodiments, after establishing the Neo4j graph database, the knowledge graph can be viewed through the graphical interface of the Neo4j Browser to verify whether the connections are correct, thus achieving visual verification.

[0119] To accelerate query speed and prevent data conflicts, database-level optimization mechanisms are required. In some embodiments, the smart card domain knowledge graph indexes each topic keyword and each visual element keyword. For example, this application may index the name field of all topic nodes and the name field of all visual element nodes to achieve millisecond-level response and accelerate search speed.

[0120] In some embodiments, the smart card domain knowledge graph may also require that the identifier of each functional area be unique to prevent accidental duplication.

[0121] In some embodiments, the construction process of the target large language model includes:

[0122] Obtain a pre-trained large language model;

[0123] The LORA algorithm is used to fine-tune the pre-trained large language model and obtain the target large language model.

[0124] In this embodiment, the attention layer of the pre-trained large language model can be fine-tuned using the LORA (Low-Rank Adaptation) algorithm, so that the fine-tuned target large language model can understand the correspondence between "topic-functional related requirements" and automatically generate multi-dimensional prompt words containing topic elements, functional constraints, and technical parameters.

[0125] It is understood that this application can be implemented using large language models of any structure, and this application does not impose any specific restrictions on this. For ease of explanation, this application uses the Qwen-7B model as an example. In the process of constructing the target large language model, it can be achieved through training sample construction sub-steps, fine-tuning adaptation sub-steps, and fine-tuning training sub-steps. Among them, in the training sample construction sub-step, topic keywords can be used as query keywords to search in BK-KG and obtain the associated element text corresponding to the topic keywords. Then, a general large language model can be used to expand the topic keywords and associated element text to obtain training text. For example, the topic keyword can be "Year of the Snake", the associated element text can be "colorful auspicious clouds, dazzling fireworks, gilded", and the training text output by the general large language model can be "A golden Chinese zodiac mythical beast sits on a giant rock, surrounded by auspicious clouds and flames, with the city skyline at night and dazzling fireworks in the background. The card number area is left blank, and the font is BankGothic." This application can prepare multiple sets of training samples in this way, each set of training samples including topic keywords, associated element text, and training text.

[0126] In the fine-tuning adaptation sub-step, this application allows for setting LORA parameters and fine-tuning parameters. For example, the rank of the low-rank matrix can be set to 8, the scaling factor to 16, the dropout factor to 0.05 to prevent overfitting, and the fine-tuning target can be set to the Q and V matrices of the attention layer. Furthermore, specific settings can be made for training parameters such as the learning rate and the batch size.

[0127] In the fine-tuning training sub-step, a pre-trained Qwen-7B model can be loaded, and the LORA adapter can be injected using the PEFT library. Using the topic keywords and related element texts from the training samples as input, the model is guided to generate output text. The loss value is calculated based on the output text and the training text, and the LORA parameters are updated via backpropagation based on the loss value (freezing the original large language model) while preserving the LORA weights.

[0128] By fine-tuning the intelligent prompt word generation algorithm of a large language model using BK-KG and LORA, this application can automatically generate target prompt text specific to the field of smart card layout design, thereby obtaining smart card layout images that meet user needs.

[0129] The smart card layout generation apparatus provided in the embodiments of this application is described below. The smart card layout generation apparatus described below can be referred to in correspondence with the smart card layout generation method described above.

[0130] In some embodiments, such as Figure 2 As shown, this application provides a smart card layout generation device 200, including:

[0131] The text acquisition module 202 is used to acquire the layout description text input by the user;

[0132] The query module 204 is used to query the pre-constructed smart card domain knowledge graph using the layout description text as the query keyword, and obtain the associated element text; wherein, the smart card domain knowledge graph is used to describe the relationship between multiple entities, the multiple entities include topic keywords, and the multiple entities also include at least one of style keywords, visual element keywords, descriptive element keywords, and color keywords;

[0133] The splicing module 206 is used to splice the layout description text and the associated element text to obtain the initial prompt text;

[0134] The text expansion module 208 is used to input the initial prompt text into the target large language model, so that the target large language model expands the initial prompt text and outputs the target prompt text;

[0135] The layout image generation module 210 is used to input the target prompt text into the text-generated image model and obtain the smart card layout image.

[0136] In some embodiments, the text image model is a diffusion model. The smart card layout generation device 200 of this application includes a text image model training module, which includes:

[0137] The attention intensity acquisition unit is used to acquire the attention intensity obtained by the diffusion model during the current training round.

[0138] The average intensity calculation unit is used to calculate the average attention intensity of each information retention area in the current training round based on the attention intensity of the current training round and the preset information retention area location.

[0139] The spatial constraint loss calculation unit is used to determine the spatial constraint loss value of the current training round based on the average attention intensity of each information retention area in the current training round.

[0140] The total loss calculation unit is used to calculate the total loss value for the current training round based on the spatial constraint loss value of the current training round.

[0141] The termination judgment unit is used to determine whether the preset training termination condition is met based on the total loss value;

[0142] The model acquisition unit is used to obtain the text-based image model if the conditions are met; otherwise, it proceeds to the next training round.

[0143] In some embodiments, the layout image generation module 210 of this application includes:

[0144] The initial image acquisition unit is used to input the target prompt text into the text-to-image model and obtain the initial layout image output by the text-to-image model;

[0145] The erasure unit is used to erase the image located in the information retention area of ​​the initial layout image according to the preset information retention area position, so as to obtain the erased image;

[0146] The layout image generation unit is used to generate a smart card layout image based on the erased image.

[0147] In some embodiments, the smart card layout generation device 200 of this application includes a knowledge graph construction module, the knowledge graph construction module comprising:

[0148] The association relationship acquisition unit is used to acquire multiple historical page images and acquire the theme attribute association relationship of each historical page image; the theme attribute association relationship includes theme keywords and attribute keywords, and the attribute keywords include at least one of the following: style keywords, visual element keywords, descriptive element keywords, and color keywords;

[0149] The association strength determination unit is used to determine the association strength between each keyword based on the association relationships of each topic attribute;

[0150] The knowledge graph construction unit is used to build a Neo4j graph database based on the relationship between various topic attributes and the strength of the relationship between various keywords, using keywords as nodes, to obtain a knowledge graph for the smart card domain.

[0151] In some embodiments, the smart card domain knowledge graph indexes each topic keyword and each visual element keyword.

[0152] In some embodiments, the smart card layout generation apparatus 200 of this application includes a target large language model acquisition module, the target large language model acquisition module comprising:

[0153] The pre-trained model acquisition unit is used to acquire a pre-trained large language model;

[0154] The fine-tuning unit is used to fine-tune the pre-trained large language model using the LORA algorithm to obtain the target large language model.

[0155] In some embodiments, the splicing module 206 of this application includes:

[0156] The splicing unit is used to splice the layout description text, associated element text, and preset behavior description text into the initial prompt text; among them, the behavior description text is used to instruct the large model to design the smart card layout and output the smart card layout image.

[0157] In one embodiment, this application also provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the smart card layout generation method as described in any embodiment.

[0158] In one embodiment, this application also provides a computer device storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the smart card layout generation method as described in any embodiment.

[0159] Indicatively, Figure 3 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of this application. In one example, the computer device can be a server. (Refer to...) Figure 3 The computer device 900 includes a processing component 902, which further includes one or more processors, and memory resources represented by memory 901 for storing instructions executable by the processing component 902, such as application programs. The application programs stored in memory 901 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 902 is configured to execute instructions to perform the steps of the smart card layout generation method described in any of the above embodiments.

[0160] The computer device 900 may also include a power supply component 903 configured to perform power management of the computer device 900, a wired or wireless network interface 904 configured to connect the computer device 900 to a network, and an input / output (I / O) interface 905. The computer device 900 may operate on an operating system stored in memory 901, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.

[0161] Those skilled in the art will understand that the internal structure of the computer device shown in this application is merely a block diagram of a portion of the structure related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0162] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this document, "a," "an," "the," "the," and "its" may also include plural forms unless the context clearly indicates otherwise. "Multiple" refers to at least two, such as 2, 3, 5, or 8, etc. "And / or" includes any and all combinations of the related listed items.

[0163] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0164] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating a smart card layout, characterized in that, include: Get the layout description text entered by the user; Using the layout description text as the query keyword, a pre-constructed smart card domain knowledge graph is queried to obtain related element text; wherein, the smart card domain knowledge graph is used to describe the relationship between multiple entities, the multiple entities include topic keywords, and the multiple entities also include at least one of style keywords, visual element keywords, descriptive element keywords, and color keywords; The initial prompt text is obtained by concatenating the layout description text and the associated element text. The initial prompt text is input into the target large language model, so that the target large language model expands the initial prompt text and outputs the target prompt text; The target prompt text is input into the text-to-image model to obtain the smart card layout image.

2. The smart card layout generation method according to claim 1, characterized in that, The text-based image model is a diffusion model, and the training process of the text-based image model includes: Obtain the attention intensity obtained by the diffusion model during the current training round; Based on the attention intensity of the current training round and the preset information retention area location, calculate the average attention intensity of each information retention area in the current training round; The spatial constraint loss value for the current training round is determined based on the average attention intensity of each information retention region in the current training round. Based on the spatial constraint loss value of the current training round, calculate the total loss value of the current training round; Determine whether the preset training termination condition is met based on the total loss value; If the conditions are met, the text-based graph model is obtained; otherwise, proceed to the next training round.

3. The smart card layout generation method according to claim 1, characterized in that, The step of inputting the target prompt text into the text-to-image model and obtaining the smart card layout image includes: The target prompt text is input into the text-to-image model to obtain the initial layout image output by the text-to-image model; According to the preset information retention area position, the image located in the information retention area in the initial layout image is erased to obtain the erased image; The smart card layout image is generated based on the erased image.

4. The smart card layout generation method according to claim 1, characterized in that, The process of constructing the smart card domain knowledge graph includes: Multiple historical page layout images are obtained, and the theme attribute association relationship of each historical page layout image is obtained respectively; the theme attribute association relationship includes theme keywords and attribute keywords, and the attribute keywords include at least one of style keywords, visual element keywords, descriptive element keywords, and color keywords; Based on the relationship between the various topic attributes, determine the correlation strength between each keyword; Based on the relationship between the various topic attributes and the strength of the relationship between the various keywords, a Neo4j graph database is constructed using keywords as nodes to obtain the smart card domain knowledge graph.

5. The smart card layout generation method according to claim 4, characterized in that, The smart card domain knowledge graph indexes each topic keyword and each visual element keyword.

6. The smart card layout generation method according to any one of claims 1 to 5, characterized in that, The construction process of the target large language model includes: Obtain a pre-trained large language model; The LORA algorithm is used to fine-tune the pre-trained large language model to obtain the target large language model.

7. The smart card layout generation method according to any one of claims 1 to 5, characterized in that, The step of concatenating the layout description text and the associated element text to obtain the initial prompt text includes: The layout description text, the associated element text, and the preset behavior description text are concatenated to form the initial prompt text; wherein, the behavior description text is used to instruct the large model to design the smart card layout and output the smart card layout image.

8. A smart card layout generation device, characterized in that, include: The text acquisition module is used to acquire the layout description text input by the user; The query module is used to query the pre-constructed smart card domain knowledge graph using the layout description text as the query keyword, and obtain the associated element text; wherein, the smart card domain knowledge graph is used to describe the relationship between multiple entities, the multiple entities include topic keywords, and the multiple entities also include at least one of style keywords, visual element keywords, descriptive element keywords, and color keywords; The splicing module is used to splice the layout description text and the associated element text to obtain the initial prompt text; The text expansion module is used to input the initial prompt text into the target large language model, so that the target large language model expands the initial prompt text and outputs the target prompt text. The layout image generation module is used to input the target prompt text into the text-generated image model and obtain the smart card layout image.

9. A storage medium, characterized in that, The storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the smart card layout generation method as described in any one of claims 1 to 7.

10. A computer device, characterized in that, include: One or more processors, and memory; The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the smart card layout generation method as described in any one of claims 1 to 7.

Images