A smart card page generation method and device, a storage medium and a computer device

Abstract

The application relates to the technical field of smart cards, and specifically provides a smart card layout generation method and device, a storage medium and computer equipment. The application can pre-construct a smart card field knowledge graph to describe the association relationship between at least one type of keywords, such as theme keywords, style keywords, visual element keywords, description element keywords and color keywords. In the process of generating a smart card layout, first, a layout description text input by a user is acquired, and associated element text associated with the layout description text is queried in the smart card field knowledge graph. Then, a target large language model is used to perform text expansion on the layout description text and the associated element text, and text-to-image is performed based on the target prompt text obtained through expansion, so as to obtain a smart card layout image. Through dynamic retrieval and a synthesis framework, the application can reduce the use threshold of a user, has the advantage of strong practicability, and can improve the intelligent degree and the automation degree, and improve the layout generation efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart cards, and particularly relates to a smart card layout generation method and device, a storage medium and computer equipment. BACKGROUND

[0002] With the wide application of AIGC technology in the field of graphic design, text-driven image generation tools such as MidJourney and DALL-E have been applied to the smart card layout design scene. When generating a smart card layout, a user can input a description text of the smart card layout and input it as a prompt word into an image generation tool, prompting the image generation tool to reason according to the description text to achieve text-to-image conversion, and finally output the smart card layout. In this process, the pattern beauty and requirement matching degree of the smart card layout are closely related to the description text, which requires the user to have certain art professional knowledge and layout design experience to write an effective prompt word. In addition, the user also needs to manually adjust the description text multiple times to obtain a smart card layout image that meets the requirements.

[0003] Therefore, the prior art not only limits the use of non-professionals and has the defect of high user threshold, but also highly depends on manual writing and adjustment of prompt words, has low intelligent level, and reduces the layout generation efficiency. SUMMARY

[0004] The present application aims to at least solve one of the above technical defects, particularly the technical defect that the prior art limits the use of non-professionals and has the defect of high user threshold, highly depends on manual writing and adjustment of prompt words, has low intelligent level, and reduces the layout generation efficiency.

[0005] In a first aspect, an embodiment of the present application provides a smart card layout generation method, comprising:

[0006] obtaining a layout description text input by a user;

[0007] querying a pre-constructed smart card domain knowledge graph with the layout description text as a query keyword, and obtaining an associated element text; wherein the smart card domain knowledge graph is used to describe the association relationship between a plurality of entities, the plurality of entities include theme keywords, and the plurality of entities further include at least one type of style keywords, visual element keywords, description element keywords, and color keywords;

[0008] splicing the layout description text and the associated element text to obtain an initial prompt text;

[0009] inputting the initial prompt text into a target large language model to make the target large language model perform text expansion on the initial prompt text and output a target prompt text;

[0010] inputting the target prompt text into a text-to-image model, and obtaining a smart card layout image.

[0011] In some embodiments, the text-to-image model is a diffusion model, and a training process of the text-to-image model includes:

[0012] obtaining an attention intensity obtained by the diffusion model in a current training round;

[0013] According to the attention intensity of the current training round and the preset information retention area position, the average attention intensity of each information retention area in the current training round is calculated respectively;

[0014] According to the average attention intensity of each information retention area in the current training round, a spatial constraint loss value of the current training round is determined;

[0015] Based on the spatial constraint loss value of the current training round, a total loss value of the current training round is calculated;

[0016] According to the total loss value, it is judged whether a preset training end condition is met;

[0017] If yes, the text-to-image model is obtained, otherwise, the next training round is entered.

[0018] In some embodiments, the inputting the target prompt text into the text-to-image model and obtaining the smart card layout image includes:

[0019] inputting the target prompt text into the text-to-image model to obtain an initial layout image output by the text-to-image model;

[0020] According to a preset information retention area position, the image in the information retention area of the initial layout image is erased to obtain an erased image;

[0021] According to the erased image, the smart card layout image is generated.

[0022] In some embodiments, the construction process of the smart card field knowledge graph includes:

[0023] Obtain a plurality of historical layout images, and obtain a theme attribute association relationship of each historical layout image respectively; the theme attribute association relationship includes a theme keyword and an attribute keyword, and the attribute keyword includes at least one of a style keyword, a visual element keyword, a description element keyword, and a color keyword;

[0024] According to the association relationship between each of the theme attributes, the association strength between each of the keywords is determined respectively;

[0025] Based on the association relationship between each of the theme attributes and the association strength between each of the keywords, a Neo4j graph database is constructed with keywords as nodes, and the intelligent card field knowledge graph is obtained.

[0026] In some embodiments, the intelligent card field knowledge graph establishes an index for each theme keyword and each visual element keyword.

[0027] In some embodiments, the construction process of the target large language model includes:

[0028] Obtaining a pre-trained large language model;

[0029] Using the LORA algorithm to fine-tune the pre-trained large language model, and obtaining the target large language model.

[0030] In some embodiments, the splicing of the layout description text and the associated element text to obtain the initial prompt text includes:

[0031] Splicing the layout description text, the associated element text, and a preset behavior description text into the initial prompt text; wherein the behavior description text is used to instruct the large model to perform intelligent card layout design and output an intelligent card layout image.

[0032] In a second aspect, the embodiments of the present application provide an intelligent card layout generation device, which includes:

[0033] A text acquisition module is configured to acquire a layout description text input by a user;

[0034] A query module is configured to query a pre-constructed intelligent card field knowledge graph by taking the layout description text as a query keyword, and obtain an associated element text; wherein the intelligent card field knowledge graph is used to describe the association relationship between a plurality of entities, and the plurality of entities include theme keywords, and the plurality of entities further include at least one type of style keywords, visual element keywords, description element keywords, and color keywords;

[0035] A splicing module is configured to splice the layout description text and the associated element text to obtain an initial prompt text;

[0036] A text expansion module is configured to input the initial prompt text into a target large language model, so that the target large language model performs text expansion on the initial prompt text and outputs a target prompt text;

[0037] The page image generation module is configured to input the target prompt text into a text-to-image model, and obtain the smart card page image.

[0038] In a third aspect, an embodiment of the present application provides a storage medium, which stores computer readable instructions. When the computer readable instructions are executed by one or more processors, the one or more processors perform the steps of the smart card page generation method according to any of the above embodiments.

[0039] In a fourth aspect, an embodiment of the present application provides a computer device, which includes one or more processors and a memory.

[0040] The memory stores computer readable instructions. When the computer readable instructions are executed by the one or more processors, the steps of the smart card page generation method according to any of the above embodiments are performed.

[0041] In the smart card page generation method, device, storage medium and computer device provided by some embodiments of the present application, a smart card field knowledge graph can be constructed in advance to describe the association relationship between at least one type of keywords, such as theme keywords, style keywords, visual element keywords, description element keywords and color keywords, so as to model the association relationship between the page theme and the page design constraint. In the process of generating the smart card page, the present application can first obtain the page description text input by the user, and query the associated element text associated with the page description text in the smart card field knowledge graph. The page description text and the associated element text are spliced into an initial prompt text to expand the elements related to the theme described in the page description text. After obtaining the initial prompt text, the present application can use a target large language model to expand the initial prompt text, and generate a text-to-image based on the target prompt text obtained by expansion, to obtain a smart card page image. Through the dynamic retrieval and synthesis framework, the present application can intelligently process the expansion of prompt words from specific themes to widely abstract themes, and then automatically expand the short page description text into rich and professional detail prompt words. In this way, the user can input a short page description text to obtain a high-quality smart card page image that meets his needs, thereby reducing the user's threshold, having the advantage of strong practicability, and improving the intelligent degree and automation degree, and improving the page generation efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0043] Figure 1 For some embodiments, the flowchart of the smart card layout generation method;

[0044] Figure 2 For some embodiments, the structural diagram of the smart card layout generation device;

[0045] Figure 3 For some embodiments, the internal structure diagram of the computer device. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0047] The technical terms involved in the present application will be explained below.

[0048] NLP: Natural Language Processing (Natural Language Processing), is a technology that enables computers to understand, interpret and manipulate human language.

[0049] CV: Computer Vision (Computer Vision), is a technology that enables computers to obtain high-level understanding from digital images or videos.

[0050] AIGC: Artificial Intelligence Generated Content, refers to the use of artificial intelligence technology to automatically generate text, images, audio, video and other content.

[0051] Diffusion Model: Diffusion Model, an advanced generative AI model, generates high-quality images from random noise through a gradual denoising process.

[0052] Layout Constraint: Layout Constraint, refers to the mandatory specification of the position, size, font, etc. of specific elements (such as text, logo) in the image.

[0053] BK-KG: Bank Card Domain Knowledge Graph, refers to a structured knowledge system built for bank card design scenarios, used to support accurate generation of prompt words.

[0054] The smart card layout generation method provided by the present application will be described below.

[0055] In some embodiments, the present application provides a smart card layout generation method. The following embodiments are described with the method applied to a computer device. It should be understood that the computer device referred to in the present application refers to a device with data input and processing functions, which can be but is not limited to a tablet computer, a notebook computer, a desktop computer, a server cluster, etc., and the present application does not make specific limitations thereto.

[0056] As shown in Figure 1 The smart card layout generation method provided by the present application can include the following steps:

[0057] S102: Obtain the layout description text input by the user.

[0058] The layout description text can be used to describe the design theme of the smart card layout to be generated. The smart card layout refers to the layout design of a smart card, which can include card patterns, card colors, card texts, chip / magnetic stripe positions, etc. The smart card refers to a card with a microchip embedded therein, which can be but is not limited to a bank card, a transportation card, an access control card, etc. For ease of description, the present application describes a bank card as an example.

[0059] In this step, the computer device can receive the layout description text input by the user in order to understand the user's requirements for the smart card layout design. For example, the layout description text can be "snake year Mid-Autumn Festival", which indicates that the user wants to get a layout image containing snake year elements and Mid-Autumn Festival elements.

[0060] S104: Query the pre-constructed smart card domain knowledge graph with the layout description text as the query keyword, and obtain the associated element text; wherein the smart card domain knowledge graph is used to describe the association between multiple entities, and the multiple entities include theme keywords, and the multiple entities also include at least one of style keywords, visual element keywords, description element keywords, and color keywords.

[0061] The theme keyword refers to the design theme around which the layout design is centered, including but not limited to holiday keywords, zodiac keywords, city keywords, and cultural IP keywords. The holiday keywords can include the Spring Festival, Labor Day, the Mid-Autumn Festival, and the National Day, etc., the zodiac keywords can include each of the twelve Chinese zodiac animals, the city keywords can include Beijing, Shanghai, Guangzhou, Shenzhen, Xi'an, etc., and the cultural IP can include Dunhuang and animation names, etc.

[0062] The style keyword refers to the artistic style involved in the layout design, which may include, for example, Chinese style, ink painting, cyberpunk, cartoon style, oil painting style, minimalism, and meticulous drawing. The visual element keyword refers to the pattern elements and texture elements possessed by the smart card layout, the pattern elements may include the moon rabbit, moon cake, osmanthus, auspicious cloud, dragon pattern, dragon head, and the like, and the texture elements may include the fret, ice crack, and the like. The description element keyword refers to the adjectives associated with a specific theme, a specific picture style, and a specific element, such as reunion, happy, full moon, winding, lively, majestic, ancient city, millennium capital, long history, bright pearl, and romantic. The color element keyword refers to the color involved in the layout design, which may include, for example, red, green, and gold.

[0063] It can be understood that a theme may have multiple associated elements (i.e., multiple attributes) to describe the artistic style, visual elements, artistic style, adjectives, colors, and the like attributes under this theme. Similarly, an artistic style may have multiple associated elements to describe the visual elements, artistic style, adjectives, colors, and the like attributes under this artistic style. For example, the associated elements corresponding to the traditional Chinese style keyword may include auspicious clouds, frets, red, yellow, and meticulous lines.

[0064] In this step, the computer device may pre-construct a smart card domain knowledge graph with the smart card design whole-process constraints as the core, to describe the association between the theme keyword and at least one type of keyword among the style keyword, visual element keyword, description element keyword, and color keyword through the smart card domain knowledge graph. In the process of generating a smart card layout design image, the computer device takes the layout description text input by the user as a query keyword, retrieves the smart card domain knowledge graph, and obtains the associated elements related to the layout description text, so as to obtain the associated element text.

[0065] For example, when the layout description text is “Mid-Autumn Festival”, the associated element text obtained by retrieval may be: moon rabbit, moon cake, osmanthus, golden yellow, red, and traditional Chinese style. For another example, when the layout description text is “snake year”, the associated element text obtained by retrieval may be: snake-shaped auspicious cloud pattern, simple snake outline, meticulous drawing, and line.

[0066] In some examples, when the text length of the layout description text is greater than a preset length threshold, the present application may perform keyword extraction on the layout description text to extract a description keyword therefrom, and take the description keyword as a query keyword to retrieve the smart card domain knowledge graph. For example, if the layout description text is: please help generate a smart card layout with the theme of snake year Mid-Autumn Festival, the snake year Mid-Autumn Festival may be extracted as a description keyword, and the smart card domain knowledge graph may be retrieved accordingly to obtain the corresponding associated element text.

[0067] S106: splice the layout description text and the associated element text to obtain an initial prompt text.

[0068] In this step, the layout description text input by the user and the associated element text obtained by searching the smart card field knowledge graph can be spliced to obtain an initial prompt text. It can be understood that the initial prompt text includes the text content of the layout description text and the text content of the associated element text.

[0069] In some examples, S106 can include splicing the layout description text, the associated element text, and a preset behavior description text into an initial prompt text; wherein the behavior description text is used to instruct the large model to perform smart card layout design and output a smart card layout image.

[0070] For example, when a bank card layout needs to be generated, the behavior description text can be: please generate a bank card layout design prompt word. For another example, when a traffic card layout needs to be generated, the behavior description text can be: please generate a layout design prompt word for an A city traffic card.

[0071] In this example, the layout description text, the associated element text, and the behavior description text can be spliced to obtain an initial prompt text. In this way, the initial prompt text can include the user's demand for smart card layout design, the relevant elements under the layout design theme, and the behavior instruction to the large model, so that the output of the large language model can better meet the layout image generation demand, further improving the intelligent degree and generation efficiency.

[0072] For example, when the layout description text is "Snake Year Mid-Autumn Festival", the associated element text is "associated elements: Jade Rabbit, Osmanthus, Golden Yellow, Traditional Chinese Style", and the behavior description text is "please generate a bank card design prompt word", after text splicing, the obtained initial prompt text can be "theme: Snake Year Mid-Autumn Festival. Associated elements: Jade Rabbit, Osmanthus, Golden Yellow, Traditional Chinese Style. Please generate a bank card design prompt word."

[0073] S108: input the initial prompt text into the target large language model to make the target large language model perform text expansion on the initial prompt text and output a target prompt text.

[0074] In this step, the initial prompt text can be input into the target large language model as input data of the large language model. After receiving the initial prompt text, the target large language model can perform reasoning according to the initial prompt text, perform text expansion based on the initial prompt text, and output the expanded prompt text (i.e., the target prompt text). In some examples, the target prompt text can be a structured prompt word.

[0075] For example, when the initial prompt text is "Theme: Mid-Autumn Festival in the Year of the Snake. Associated elements: Jade Rabbit, Osmanthus, Golden Yellow, Traditional Chinese Style. Please generate bank card design prompt words", the target prompt text output by the target large language model can be "Traditional Chinese style bank card, background is a full moon night sky, foreground right side is a golden colored coiled dragon, left lower corner is decorated with osmanthus and lanterns, overall color scheme is golden yellow and dark green, card number area is located at the bottom center, leaving clear space, font is Songti bold."

[0076] Thus, the present application can realize "theme-function-style" triple constraints through the intelligent card field knowledge graph and the target large language model, and can generate prompt words that can be directly used to drive the text-to-image model according to the short layout description text input by the user. This process does not require human intervention.

[0077] S110: input the target prompt text into the text-to-image model and obtain the intelligent card layout image.

[0078] The text-to-image model refers to a large model with text input and image output. It can perform reasoning according to the received text and output an image corresponding to the input text. In this step, the target prompt text expanded by text can be input into the text-to-image model to perform reasoning using the text-to-image model and output an intelligent card layout image that meets the user's needs.

[0079] Through the dynamic retrieval and synthesis framework, the present application can intelligently process prompt word expansion from specific themes to widely abstract themes, and then automatically expand the short layout description text into rich and professional detailed prompt words. In this way, the user can input a short layout description text to obtain a high-quality intelligent card layout image that meets their needs, thereby reducing the user's usage threshold, having the advantage of strong practicality, and improving the intelligent level and automation level, and improving the layout generation efficiency.

[0080] In some embodiments, the text-to-image model is a diffusion model, and the training process of the text-to-image model includes:

[0081] obtaining the attention intensity of the diffusion model obtained by reasoning in the current training round;

[0082] According to the attention intensity of the current training round and the preset information retention area position, the average attention intensity of each information retention area in the current training round is calculated;

[0083] According to the average attention intensity of each information retention area in the current training round, the spatial constraint loss value of the current training round is determined;

[0084] Based on the spatial constraint loss value of the current training round, the total loss value of the current training round is calculated;

[0085] determine whether the preset training end condition is met according to the total loss value;

[0086] If yes, a text-to-image model is obtained, otherwise, the next training round is entered.

[0087] Specifically, the layout design of the smart card usually needs to meet the functional compliance. Taking a bank card as an example, the layout image of the bank card needs to set the function areas such as the opening bank, card number, signature bar, etc. in a specific position, and the layout design needs to comply with the regulations such as “Bank Card Card Specification”. However, the existing smart card layout image generation scheme does not incorporate the constraints in the smart card field, so that the generated smart card layout image does not comply with the compliance layout of the smart card (for example, the card number area is covered by the pattern, the pattern overlaps with the key information, etc.), which makes the layout image unable to be put into actual use.

[0088] To solve this problem, the embodiment introduces a spatial layout constraint mechanism in the image generation process. By deeply diffusing the attention control mechanism of the denoising process of the model, it is ensured that the text-to-image model automatically avoids the preset information retention area when generating the smart card layout image, which fundamentally guarantees that the smart card layout image complies with the rigid layout requirements of the smart card, thereby improving the practicality and compliance of the smart card layout image. In this way, the degree of intelligence and automation can be further improved, thereby improving the layout generation efficiency.

[0089] Specifically, the present application uses a diffusion model as a text-to-image model. In the denoising process of the diffusion model, the spatial layout constraint is injected by modifying the attention map (Attention Map) and / or the loss function, and the diffusion model can learn the area suppression and guidance mechanism through model training. When generating the smart card layout image, the trained diffusion model (i.e. the text-to-image model) can simultaneously generate content and comply with the layout, so that the generated smart card layout image automatically avoids the preset information retention area.

[0090] In the embodiment, a layout mask can be predefined. The layout mask includes design templates of various types of smart cards, and clearly identifies the information retention area (i.e. the pattern generation prohibited area) and the pattern preferred area of each type of smart card. Among them, the information retention area refers to the area of the smart card used to realize a specific function, for example, it can be an information recording area or a magnetic stripe / chip setting area, and the information recording area includes but is not limited to a card number area, a name area, a signature bar area, and a bank logo placement area, etc.

[0091] In the process of model training on the diffusion model, the application can obtain the attention intensity inferred by the diffusion model in the current training round, and calculate the average attention intensity corresponding to each information retention region according to the attention intensity of the current training round. Then, according to the average attention intensity of all information retention regions, the spatial constraint loss value of the current training round is calculated. For example, the average attention intensity of all information retention regions can be weighted and summed or accumulated to obtain the spatial constraint loss value.

[0092] In some examples, the spatial constraint loss value can be calculated by the expression L constraint = λ × Σ R max(0, S R - τ) R = 1, 2, … n. In the expression, L constraint is the spatial constraint loss value, λ is the penalty weight, S R is the average attention intensity of the Rth information retention region, n is the total number of information retention regions, and τ is the attention threshold. It can be understood that the specific values of λ and τ can be determined according to actual conditions, and the application does not make specific limitations. For example, λ can be 1.0, and τ can be 0.1 to indicate that the attention degree of the forbidden area should not be too high.

[0093] The application can use the spatial constraint loss value as a penalty to determine the total loss value of the current training round. In some examples, the total loss value can be the sum of the original denoising loss value and the spatial constraint loss value. The original denoising loss value can be calculated by any existing denoising loss function, and is used to represent the difference between the denoising result of the current training round and the training sample. The application can adjust the model parameters of the diffusion model based on the total loss value until the preset training end condition is met and the final diffusion model is obtained.

[0094] When the image generated by the diffusion model produces a high-activation attention map in the information retention region (i.e., the model view generates pattern content in the region), the spatial constraint loss value will increase, thereby penalizing the model for generating high attention in the image generation forbidden area, guiding the diffusion model to reduce the activation of the region in the next denoising step, and turning to generate content in the pattern preferred area. In this way, the diffusion model can automatically learn to guide the generation of pattern content to the pattern generation preferred area, so that the intelligent card layout image meeting the compliance requirements can be automatically generated by the text-to-image model.

[0095] In some examples, in each step of the denoising process of the diffusion model, the computer device can calculate the correlation between the prompt words input to the diffusion model and the image region. By modifying the cross-attention mechanism of the diffusion model, the spatial constraint is enhanced.

[0096] In some embodiments, the target prompt text is input into the text-to-image model, and an intelligent card layout image is obtained, including:

[0097] The target prompt text is input into the text-to-image model to obtain an initial layout image output by the text-to-image model.

[0098] According to the preset information retention area position, the image in the initial layout image located in the information retention area is erased to obtain an erased image.

[0099] According to the erased image, an intelligent card layout image is generated.

[0100] In this embodiment, in order to ensure that the intelligent card layout image meets the rigid layout requirements of the intelligent card, and to further improve the intelligent degree, the automation degree and the layout generation efficiency, the intelligent card layout image can be obtained by using the post-generation repair method.

[0101] Specifically, the target prompt text can be input into the text-to-image model as a prompt word of the text-to-image model, so that the text-to-image model can infer based on the target prompt text and output an initial layout image. In the case of obtaining the initial layout image, the image erasing technology can be used to erase the pattern located in the information retention area and retain the pattern in the non-information retention area, thereby obtaining an erased image, and obtaining an intelligent card layout image accordingly.

[0102] In some examples, the application can erase the image as an intelligent card layout image. Alternatively, the application can add functional patterns such as opening bank identification and signature strip patterns in the information retention area based on the erased image, and obtain an intelligent card layout image.

[0103] In some embodiments, the construction process of the intelligent card field knowledge graph includes:

[0104] A plurality of historical layout images are obtained, and the theme attribute association relationship of each historical layout image is obtained respectively. The theme attribute association relationship includes theme keywords and attribute keywords, and the attribute keywords include at least one of style keywords, visual element keywords, description element keywords and color keywords.

[0105] According to each theme attribute association relationship, the association strength between each keyword is determined.

[0106] Based on each theme attribute association relationship and the association strength between each keyword, a Neo4j graph database is constructed with keywords as nodes to obtain an intelligent card field knowledge graph.

[0107] In this embodiment, the Neo4j graph database can be constructed according to the theme attribute association relationship contained in the plurality of historical layout images to support image display and efficient query.

[0108] Specifically, the construction process of the smart card field knowledge graph can include a data collection sub-step, a knowledge fusion and graph construction sub-step, and a batch import sub-step. The data collection sub-step specifically includes obtaining a plurality of historical layout images, which are all compliant layout designs of smart cards, and extracting the theme attribute association of each historical layout image through, for example, a multi-modal large model, a large language model platform, and the like.

[0109] In some examples, during the execution of the data collection sub-step, the present application can first collect 1000 historical layout images, and use a multi-modal large model such as qwen2.5-VL-72B to generate text from images to obtain the image description text of each historical layout image. Then, the present application can use a general large language model to generate the design theme keywords of each historical layout image according to the image description text of each historical layout image. The visual element keywords corresponding to each design theme keyword can be generated with the help of a general large language model and a Chinese traditional culture database. Further, to improve the text quality of the image description text, the present application can also clean and rearrange the image description text by artificial means before generating the design theme keywords with the large language model.

[0110] In the knowledge fusion and graph construction sub-step, the present application can systematically organize the structured knowledge obtained in the data collection sub-step and the pre-set entity types and relationship types, and obtain a queryable, inferable, and extensible graphical knowledge network (i.e., a smart card field knowledge graph), which serves as the domain brain for subsequent prompt word generation.

[0111] The pre-set entity types can include theme keywords, style keywords, visual element keywords, description element keywords, and color keywords. The relationship types can include relationships from theme keywords to visual elements, relationships from theme keywords to style keywords, relationships from theme keywords to description element keywords, relationships from theme keywords to color keywords, relationships from style keywords to color keywords, relationships from visual element keywords to style keywords, and the like.

[0112] In the process of creating the Neo4j graph database, the structured knowledge obtained by the data collection sub-step needs to be unified into the same format. For example, the subject-attribute association relationship obtained by the collection step can be recorded by a structured CSV (Comma-Separated Values) file. The CSV file can include three columns, namely the subject word head, the relationship relation, and the attribute word tail. Taking the subject-attribute association relationship of the Mid-Autumn Festival-golden as an example, it can be recorded as head = "Mid-Autumn Festival", relation = "has_color", and tail = "golden" in the same row of the CSV file.

[0113] In addition, in Neo4j, a node (Node) represents an entity (i.e., each keyword), such as "Mid-Autumn Festival" and "Jade Rabbit". The computer device can create a node for each entity, and each node can include a name field name, a type field type, a description field description, and a confidence score field confidence_score, so as to facilitate subsequent query and sorting, thereby improving the retrieval rate of the smart card domain knowledge graph and further improving the generation efficiency of the smart card layout image.

[0114] For example, the node corresponding to the entity "Mid-Autumn Festival" can be created by the instruction CREATE (:subject {name: 'Mid-Autumn Festival', type: 'festival', description: 'the 15th day of the eighth lunar month, reunion festival', confidence_score: 0.95}). For another example, the node corresponding to "Jade Rabbit" can be created by the instruction CREATE (:visual element {name: 'Jade Rabbit', type: 'pattern', description: 'rabbit in the moon palace, symbol of good luck', confidence_score: 0.9}). For another example, the node corresponding to "traditional Chinese style" can be created by the instruction CREATE (:art style {name: 'traditional Chinese style', type:'style', description:'meticulous brushwork, ink, auspicious cloud pattern', confidence_score: 0.98}); and the node corresponding to "flower and moon" can be created by the instruction CREATE (:description element {name: 'flower and moon', type: 'proverb', description:'symbol of happy reunion', confidence_score: 0.92}).

[0115] After the nodes are established, the computer device can connect the nodes by using relationships to form a network. The connection of the nodes is based on the subject attribute association relationship obtained in the data collection sub-step. By constructing the relationships, a radial knowledge network centered on the subject node can be formed.

[0116] For example, when the obtained subject attribute association relationship includes Mid-Autumn Festival-Yutu, Mid-Autumn Festival-gold, the MATCH (t:subject {name: 'Mid-Autumn Festival'}), (e:visual element {name: 'Yutu'}) and CREATE (t)-[:has_element {strength: 0.9}]->(e) instructions can be used to connect the 'Mid-Autumn Festival' node and the 'Yutu' node, and the MATCH (t:subject {name: 'Mid-Autumn Festival'}), (c:color {name: 'gold'}) and CREATE (t)-[:has_color {strength: 0.85}]->(c) instructions can be used to connect the 'Mid-Autumn Festival' node and the 'gold' node.

[0117] In the Neo4j graph database, each relationship has an association strength, which is used to represent the association credibility of the two nodes connected by the relationship, which can be determined based on the co-occurrence frequency of keywords or a preset scoring mechanism. In this way, subsequent sorting and querying can be facilitated.

[0118] In some embodiments, after the Neo4j graph database is established, the knowledge graph can be checked whether it is correctly connected through the graphical interface of the Neo4j Browser, and visual verification can be achieved.

[0119] In order to speed up the query and prevent data conflicts, a database-level optimization mechanism needs to be set. In some embodiments, the smart card field knowledge graph establishes an index for each subject keyword and each visual element keyword. For example, the present application can establish an index for the name field of all subject nodes and an index for the name field of all visual element nodes to achieve millisecond-level response and speed up the search.

[0120] In some embodiments, the smart card field knowledge graph can also require that the identification of each functional area must be unique to prevent repeated creation caused by misoperation.

[0121] In some embodiments, the construction process of the target large language model includes:

[0122] Obtaining a pre-trained large language model;

[0123] Using the LORA algorithm to fine-tune the pre-trained large language model, and obtaining a target large language model.

[0124] In this embodiment, the LORA (Low-Rank Adaptation) algorithm can be used to fine-tune the attention layer of the pre-trained large language model, so that the target large language model obtained by fine-tuning can understand the corresponding relationship of "theme-function association demand" and automatically generate multi-dimensional prompt words containing theme elements, function constraints and technical parameters.

[0125] It can be understood that the present application can use any structure of large language model to implement, and the present application does not make specific limitation. For the convenience of description, the present application takes Qwen-7B model as an example. In the construction process of the target large language model, it can be realized by training sample construction substep, fine-tuning adaptation substep and fine-tuning training substep. In the training sample construction substep, the theme keyword can be used as the query keyword, searched in BK-KG, and the associated element text corresponding to the theme keyword is obtained. Then, the theme keyword and the associated element text can be expanded by the general large language model to obtain the training text. For example, the theme keyword can be "snake year", the associated element text can be "five-color auspicious cloud, dazzling fireworks, gilded", and the training text output by the general large language model can be "The golden Chinese zodiac beast is perched on a huge stone, surrounded by auspicious clouds and flames, and the background is the city skyline and dazzling fireworks at night. The card number area is blank, and the font is Bank Gothic." The present application can prepare multiple sets of training samples in this way, and each set of training sample includes theme keyword, associated element text and training text.

[0126] In the fine-tuning adaptation substep, the present application can set LORA parameters and fine-tuning parameters. For example, the rank of the low-rank matrix can be set to 8, the scaling coefficient can be set to 16, the dropout coefficient can be set to 0.05 to prevent overfitting, and the fine-tuning target can be set to the Q and V matrices of the attention layer. In addition, the learning rate, batch data size and other training parameters can also be set.

[0127] In the fine-tuning training substep, the pre-trained Qwen-7B model can be loaded, and the LORA adapter can be injected using the PEFT library. The theme keyword and the associated element text in the training sample are used as input to guide the model to generate output text, and the loss value is calculated according to the output text and the training text. The LORA parameters are updated (the original large language model is frozen) according to the loss value and the LORA weights are saved.

[0128] Through the intelligent prompt word generation algorithm of BK-KG and LORA fine-tuning large language model, the present application can automatically generate target prompt text exclusive in the field of smart card layout design, so as to obtain the smart card layout image meeting the user's demand.

[0129] The intelligent card layout generation device provided by the embodiments of the present application is described below. The intelligent card layout generation device described below can be correspondingly referred to the intelligent card layout generation method described above.

[0130] In some embodiments, as shown in Figure 2 The present application provides an intelligent card layout generation device 200, which comprises:

[0131] A text acquisition module 202 is configured to acquire a layout description text input by a user.

[0132] A query module 204 is configured to query a pre-constructed intelligent card field knowledge graph by taking the layout description text as a query keyword, and obtain an associated element text. The intelligent card field knowledge graph is configured to describe the association relationship between a plurality of entities, and the plurality of entities comprise at least one of a theme keyword, a style keyword, a visual element keyword, a description element keyword, and a color keyword.

[0133] A splicing module 206 is configured to splice the layout description text and the associated element text to obtain an initial prompt text.

[0134] A text expansion module 208 is configured to input the initial prompt text into a target large language model, so that the target large language model performs text expansion on the initial prompt text and outputs a target prompt text.

[0135] A layout image generation module 210 is configured to input the target prompt text into a text-to-image model, and obtain an intelligent card layout image.

[0136] In some embodiments, the text-to-image model is a diffusion model, and the intelligent card layout generation device 200 of the present application comprises a text-to-image model training module, which comprises:

[0137] An attention intensity acquisition unit is configured to acquire an attention intensity obtained by reasoning of the diffusion model in a current training round.

[0138] An average intensity calculation unit is configured to calculate an average attention intensity of each information retention area in the current training round according to the attention intensity of the current training round and a preset information retention area position.

[0139] A spatial constraint loss value calculation unit is configured to determine a spatial constraint loss value of the current training round according to the average attention intensity of each information retention area in the current training round.

[0140] A total loss value calculation unit is configured to calculate a total loss value of the current training round based on the spatial constraint loss value of the current training round.

[0141] The end judgment unit is configured to judge whether a preset training end condition is met according to the total loss value.

[0142] The model obtaining unit is configured to obtain the text-to-image model if the condition is met, or enter a next training round.

[0143] In some embodiments, the layout image generation module 210 of the present application comprises:

[0144] The initial image obtaining unit is configured to input the target prompt text into the text-to-image model to obtain an initial layout image output by the text-to-image model.

[0145] The erasing unit is configured to erase an image in the initial layout image located in the information retention area according to a preset information retention area position to obtain an erased image.

[0146] The layout image generation unit is configured to generate a smart card layout image according to the erased image.

[0147] In some embodiments, the smart card layout generation device 200 of the present application comprises a knowledge graph construction module, which comprises:

[0148] The association relationship obtaining unit is configured to obtain a plurality of historical layout images and obtain a theme attribute association relationship of each historical layout image respectively; the theme attribute association relationship comprises a theme keyword and an attribute keyword, and the attribute keyword comprises at least one of a style keyword, a visual element keyword, a description element keyword, and a color keyword.

[0149] The association strength determination unit is configured to determine an association strength between each keyword according to each theme attribute association relationship.

[0150] The knowledge graph construction unit is configured to construct a Neo4j graph database by taking a keyword as a node based on each theme attribute association relationship and the association strength between each keyword to obtain a smart card field knowledge graph.

[0151] In some embodiments, the smart card field knowledge graph establishes an index for each theme keyword and each visual element keyword.

[0152] In some embodiments, the smart card layout generation device 200 of the present application comprises a target large language model obtaining module, which comprises:

[0153] The pre-training model obtaining unit is configured to obtain a pre-trained large language model.

[0154] The fine-tuning unit is configured to fine-tune the pre-trained large language model by using a LORA algorithm to obtain a target large language model.

[0155] In some embodiments, the splicing module 206 of the present application comprises:

[0156] a splicing unit, configured to splice the layout description text, the associated element text and the preset behavior description text into the initial prompt text; wherein the behavior description text is used to instruct the large model to perform intelligent card layout design and output an intelligent card layout image.

[0157] In one embodiment, the present application further provides a storage medium, wherein the storage medium stores computer readable instructions, and the computer readable instructions are executed by one or more processors to make the one or more processors perform the steps of the intelligent card layout generation method in any embodiment.

[0158] In one embodiment, the present application further provides a computer device, wherein the computer device stores computer readable instructions, and the computer readable instructions are executed by one or more processors to make the one or more processors perform the steps of the intelligent card layout generation method in any embodiment.

[0159] Schematically, Figure 3 An internal structure schematic diagram of a computer device provided by an embodiment of the present application is shown in the figure, and in one example, the computer device can be a server. Referring to Figure 3 , the computer device 900 comprises a processing component 902, which further comprises one or more processors, and a memory resource represented by a memory 901, used to store instructions executable by the processing component 902, such as an application program. The application program stored in the memory 901 can comprise one or more than one module each corresponding to a set of instructions. In addition, the processing component 902 is configured to execute the instructions to perform the steps of the intelligent card layout generation method described in any embodiment.

[0160] The computer device 900 can further comprise a power supply component 903 configured to perform power management of the computer device 900, a wired or wireless network interface 904 configured to connect the computer device 900 to a network, and an input output (I / O) interface 905. The computer device 900 can operate based on an operating system stored in the memory 901, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM or the like.

[0161] Those skilled in the art can understand that the internal structure of the computer device shown in the present application is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. A specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0162] Finally, it should be noted that the terms such as first and second, etc. are used herein only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Also, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device including the element. In this document, "a", "an", "said", "the" and "it" can also include plural forms, unless the context clearly indicates otherwise. A plurality means at least two, such as 2, 3, 5 or 8, etc. "And / or" includes any and all combinations of the related listed items.

[0163] The various embodiments in the specification are described in a progressive manner, each embodiment focusing on the differences from other embodiments, and the various embodiments can be combined as needed, and the same and similar parts refer to each other.

[0164] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating a smart card layout, characterized in that, The method comprises the following steps: obtaining a layout description text input by a user; querying a pre-constructed smart card field knowledge graph with the layout description text as a query keyword, and obtaining associated element text; wherein the smart card field knowledge graph is used to describe the association relationship between a plurality of entities, and the plurality of entities include theme keywords, and the plurality of entities further include at least one type of style keywords, visual element keywords, description element keywords, and color keywords; splicing the layout description text and the associated element text to obtain an initial prompt text; inputting the initial prompt text into a target large language model to make the target large language model perform text expansion on the initial prompt text and output a target prompt text; inputting the target prompt text into a text-to-image model to obtain a smart card layout image; wherein the text-to-image model is a diffusion model, and the training process of the text-to-image model comprises: obtaining the attention intensity obtained by the diffusion model in the current training round; calculating the average attention intensity of each information retention area in the current training round according to the attention intensity of the current training round and the preset information retention area position; determining the spatial constraint loss value of the current training round according to the average attention intensity of each information retention area in the current training round; calculating the total loss value of the current training round based on the spatial constraint loss value of the current training round; determining whether the preset training end condition is met according to the total loss value; if yes, the text-to-image model is obtained, otherwise, the next training round is entered.

2. The method of claim 1, wherein, The method of inputting the target prompt text into the text-to-image model to obtain the smart card layout image comprises: inputting the target prompt text into the text-to-image model to obtain an initial layout image output by the text-to-image model; erasing the image in the information retention area in the initial layout image according to the preset information retention area position to obtain an erased image; generating the smart card layout image according to the erased image.

3. The method of claim 1, wherein, The construction process of the smart card field knowledge graph comprises: obtaining a plurality of historical layout images, and obtaining a theme attribute association relationship of each historical layout image; the theme attribute association relationship includes theme keywords and attribute keywords, and the attribute keywords include at least one type of style keywords, visual element keywords, description element keywords, and color keywords; determining the association strength between each keyword according to each theme attribute association relationship; constructing a Neo4j graph database with keywords as nodes based on each theme attribute association relationship and the association strength between each keyword to obtain the smart card field knowledge graph.

4. The method of claim 3, wherein, The smart card field knowledge graph establishes an index for each theme keyword and each visual element keyword.

5. The method of claim 1 to 4, wherein, The construction process of the target large language model comprises: obtaining a pre-trained large language model; performing model fine-tuning on the pre-trained large language model by using a LORA algorithm to obtain the target large language model.

6. The method of claim 1 to 4, wherein, The step of concatenating the layout description text and the associated element text to obtain the initial prompt text includes: The layout description text, the associated element text, and the preset behavior description text are concatenated to form the initial prompt text; wherein, the behavior description text is used to instruct the large model to design the smart card layout and output the smart card layout image.

7. An intelligent card layout generating apparatus characterized by comprising: include: The text acquisition module is used to acquire the layout description text input by the user; The query module is used to query the pre-constructed smart card domain knowledge graph using the layout description text as the query keyword, and obtain the associated element text; wherein, the smart card domain knowledge graph is used to describe the relationship between multiple entities, the multiple entities include topic keywords, and the multiple entities also include at least one of style keywords, visual element keywords, descriptive element keywords, and color keywords; The splicing module is used to splice the layout description text and the associated element text to obtain the initial prompt text; The text expansion module is used to input the initial prompt text into the target large language model, so that the target large language model expands the initial prompt text and outputs the target prompt text. A layout image generation module is used to input the target prompt text into a text-based image model and obtain a smart card layout image. The text-based image model is a diffusion model, and its training process includes: obtaining the attention intensity inferred by the diffusion model in the current training round; calculating the average attention intensity of each information retention area in the current training round based on the attention intensity of the current training round and the preset information retention area positions; determining the spatial constraint loss value of the current training round based on the average attention intensity of each information retention area in the current training round; calculating the total loss value of the current training round based on the spatial constraint loss value; determining whether a preset training termination condition is met based on the total loss value; if met, the text-based image model is obtained; otherwise, the next training round is entered.

8. A storage medium, characterized by The storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the smart card layout generation method as described in any one of claims 1 to 6.

9. A computer device, comprising: include: One or more processors, and memory; The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the smart card layout generation method as described in any one of claims 1 to 6.

Images