Graph typesetting method and device

Abstract

The embodiment of the invention provides a graphic typesetting method. The graphic typesetting method comprises the following steps: acquiring a background image, a target typesetting element and typesetting prompt information; extracting visual features of the background image by adopting a visual model; and inputting the visual features, the target typesetting element and the typesetting prompt information into a target large language model, and performing graphic typesetting by utilizing the target large language model according to the visual features, the target typesetting element and the typesetting prompt information. According to the technical scheme of the embodiment of the invention, the typesetting efficiency can be improved, the large-module production requirement can be quickly met, and the design with complex or specific requirements can be realized.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular, to a graphic typesetting method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] Traditional image editing and typesetting mainly rely on manual design using image editing software (such as Adobe Photoshop), where designers manually adjust the layout and style of elements to carry out specific designs. Although this method is flexible, it is inefficient and difficult to quickly adapt to large-scale production needs. It is difficult to quickly adjust or adapt when faced with complex visual content and diverse user needs.

[0003] In order to solve the efficiency problem of manual typesetting, some template-based automatic typesetting tools have emerged. These tools usually provide preset layout templates, and users can simply adjust the position and size of text, images and other elements in the layout templates to quickly generate design outputs. However, the fixed nature of layout templates limits creativity and personalized expression. Users can only make limited adjustments within the preset framework, making it difficult to achieve complex or specific designs.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0005] The embodiments of the present application provide a graphic typesetting method, apparatus, computer equipment, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems raised above.

[0006] One aspect of an embodiment of the present application provides a graphic typesetting method, the method comprising: Get the background image, target layout element and layout hint information; Extracting visual features of the background image using a visual model; The visual features, the target typeset elements and the typeset prompt information are input into a target large language model, and the target large language model is used to perform graphic typeset according to the visual features, the target typeset elements and the typeset prompt information.

[0007] Optionally, extracting visual features of the background image using a visual model includes: A target visual information transformer is used to extract visual features of the background image.

[0008] Optionally, the method further comprises: The visual information transformer is fine-tuned using a first training data set to obtain the target visual information transformer, wherein the first training data set includes N training background images, and the training background images include images of multiple colors, textures and / or styles.

[0009] Optionally, the target layout elements include at least some of the following elements: titles, project logos, projects and decorative elements.

[0010] Optionally, the method further comprises: A second training data set is used to fine-tune the large language model to obtain the target large language model, wherein the second training data set includes a plurality of paired combinations of training visual features, training typographical elements, and training typographical prompt information, the training visual features include visual features extracted from different background images, the training typographical elements include a variety of different titles, project logos, projects and decorative elements, and the training typographical prompt information includes at least part of the information on the position, size, color, font, font size, hierarchy, spacing, layout mode, image style, and image processing of the elements.

[0011] Optionally, the performing graphic typeset according to the visual features, the target typeset elements and the typeset prompt information by using the target large language model includes: Obtaining graphic layout information of the target large language model; The target layout element is placed in the background image according to the graphic layout information to obtain a layout image.

[0012] Optionally, placing the target layout element in the background image according to the graphic layout information includes: Determine whether the target layout element has a layout conflict; In the case of a layout conflict, adjusting the elements of the layout conflict using a preset adjustment method to obtain an adjustment result, wherein the preset adjustment method at least includes moving, scaling and rotating; The target layout element is placed in the background image according to the adjustment result.

[0013] Another aspect of an embodiment of the present application provides a graphic typesetting device, the device comprising: The acquisition module is used to obtain the background image, target layout element and layout prompt information; An extraction module, used for extracting visual features of the background image using a visual model; The typesetting module is used to input the visual features, the target typesetting elements and the typesetting prompt information into a target large language model, and use the target large language model to perform graphic typeset according to the visual features, the target typesetting elements and the typesetting prompt information.

[0014] Another aspect of an embodiment of the present application provides a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein: the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described above.

[0015] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0016] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the method described above when executed by a processor.

[0017] The above technical solution adopted in the embodiment of the present application may have the following advantages: By acquiring the background image, target typesetting elements and typesetting prompt information, using the visual model to extract the visual features of the background image, the visual features, target typesetting elements and typesetting prompt information are input into the target large language model, and the target large language model is used to perform graphic typesetting according to the visual features, target typesetting elements and typesetting prompt information. By combining the visual model with the large language model for graphic typesetting, visual understanding and typesetting requirements can be effectively connected, the efficiency of typesetting can be effectively improved, and it can quickly adapt to large-scale production needs and effectively respond to complex visual content and diverse user needs; and, through the background image, typesetting elements and typesetting prompt information input by the user, the design of complex or specific needs can be effectively realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0019] Figure 1 The operating environment diagram of the graphic typesetting method according to the first embodiment of the present application is schematically shown; Figure 2 A flowchart of a graphic typesetting method according to Embodiment 1 of the present application is schematically shown; Figure 3 The newly added process of the graphic typesetting method according to the first embodiment of the present application is schematically shown; Figure 4 The schematic diagram shows a principle diagram of a graphic typesetting method according to the first embodiment of the present application; Figure 5 Schematically shows Figure 3 Flow chart of sub-steps of step S302; Figure 6 A block diagram schematically shows a graphic typesetting device according to the second embodiment of the present application; and Figure 7 The hardware architecture diagram of the computer device according to the third embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0021] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0022] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0023] First, the following terms are explained: Large Language Model: A model with billions of parameters that has been pre-trained to obtain extensive knowledge from a large unlabeled text corpus. It can understand and generate natural language and has achieved remarkable success in various natural language processing tasks. In addition, through methods such as multimodal instruction fine-tuning, the large language model can also incorporate information from other modalities such as vision to handle more complex multimodal tasks.

[0024] Layout Generation: Layout generation refers to the way of arranging the positions and sizes of various multimodal design elements in a graphic design to achieve visual pleasure and follow constraints. It plays a key role in all kinds of graphic designs such as commercial posters, mobile app UI, web pages, video thumbnails, etc.

[0025] Vision Transformer (VIT): VIT introduces the Transformer architecture from the field of natural language processing to computer vision for processing image data. It treats images as a series of "visual words" or "tokens" instead of continuous arrays of pixels, thereby achieving image feature extraction and classification.

[0026] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: In related technologies, image editing and typesetting mainly rely on manual design using image editing software, which is inefficient and difficult to quickly adapt to large-scale production needs; while template-based automated typesetting tools limit creativity and personalized expression due to the fixed nature of layout templates, making it difficult to achieve designs with complex or specific needs.

[0027] To this end, the embodiment of the present application provides a graphic typesetting technical solution. In this technical solution, the efficiency of typesetting can be improved, the needs of large-scale production can be quickly adapted, and the design of complex or specific needs can be realized. See below for details.

[0028] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0029] like Figure 1 As shown, the environment diagram includes a service platform 2, a network 4, and a client 6, wherein: The service platform 2 may be composed of a single or multiple computing devices. The multiple computing devices may include virtualized computing instances. Virtualized computing instances may include virtual machines, such as simulations of computer systems, operating systems, servers, etc. The computing device may load a virtual machine based on a virtual image and / or other data defining specific software (e.g., operating system, dedicated application, server) for simulation. As the demand for different types of processing services changes, different virtual machines may be loaded and / or terminated on one or more computing devices. A hypervisor may be implemented to manage the use of different virtual machines on the same computing device.

[0030] The service platform 2 may be configured to communicate with the client 6 or the like via a network 4. The network 4 includes various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or the like. The network 4 may include physical links, such as coaxial cable links, twisted pair cable links, optical fiber links, combinations thereof, and the like, or wireless links, such as cellular links, satellite links, Wi-Fi links, and the like.

[0031] The service platform 2 can provide storage, reading, writing, querying, deleting and other services, such as providing graphic typesetting services for clients.

[0032] The client 6 may be an electronic device running an operating system such as Windows, Android™ or iOS, such as a smart phone, a tablet device, a laptop computer, a virtual reality device, a gaming device, a set-top box, a vehicle terminal, or a smart TV. Based on the above operating system, various application programs may be run, such as an application program that assists in performing graphic typesetting operations.

[0033] The client 6 can provide / configure a user access page for manipulating the service platform 2 or uploading objects, etc. For example, background images, graphic layout elements and layout prompt information required for typesetting can be uploaded to the service platform 2.

[0034] It should be noted that the above devices are exemplary, and the number and type of devices are adjustable in different scenarios or according to different needs.

[0035] The following uses the service platform as the execution subject and introduces the technical solution of the present application through multiple embodiments. It should be noted that these embodiments can be implemented in a variety of different forms and should not be interpreted as being limited to the embodiments described here. In addition, if the client has sufficient computing power, the client can also serve as the execution subject.

[0036] Embodiment 1 Figure 2 The flowchart of the graphic typesetting method according to the first embodiment of the present application is schematically shown.

[0037] like Figure 2 As shown, the graphic typesetting method may include steps S200 to S204, wherein: Step S200, obtaining a background image, a target layout element and layout prompt information.

[0038] Step S202: extracting visual features of the background image using a visual model.

[0039] Step S204: input the visual features, target layout elements and layout prompt information into the target large language model, and use the target large language model to perform graphic layout according to the visual features, target layout elements and layout prompt information.

[0040] The graphic typesetting method provided in the present embodiment obtains a background image, target typesetting elements and typesetting prompt information, uses a visual model to extract visual features of the background image, inputs the visual features, target typesetting elements and typesetting prompt information into a target large language model, and uses the target large language model to perform graphic typeset according to the visual features, target typesetting elements and typesetting prompt information. By combining the visual model with the large language model for graphic typesetting, visual understanding and typesetting requirements can be effectively connected, and the efficiency of typesetting can be effectively improved. It can quickly adapt to large-scale production needs and effectively respond to complex visual content and diverse user needs. In addition, through the background image, typesetting elements and typesetting prompt information input by the user, the design of complex or specific needs can be effectively realized.

[0041] The following combination Figure 2 , each step in steps S200~S204 and other optional steps are described in detail.

[0042] Step S200 , get the background image, target layout element and layout hint information.

[0043] The background image is the background image corresponding to the target typesetting element. The background image and the target typesetting element can be uploaded to the service platform by the user logging into the client. After being uploaded to the service platform, they can be stored in a specified location, and the service platform then obtains the background image and the target typesetting element from the specified location. The typesetting prompt information can be input by the user logging into the client. After the user completes the input, the client sends the typesetting prompt information to the service platform, so that the service platform can obtain the typesetting prompt information.

[0044] The target layout element may include elements such as text, image, logo, etc. In an exemplary embodiment, the target layout element may include at least some elements of a title, a project logo, a project, and a decorative element.

[0045] Among them, the title can include the main title and subtitle, or the first-level title, second-level title, tertiary title, etc. The titles of different levels can be distinguished by different styles of fonts, font sizes, emphasis (such as bold, italics or underline), etc.; the project logo can be specifically obtained by graphics, patterns, text or their combination; the project can be a paragraph or item of text content, which can be represented by a table, value, chart or key point overview; decorative elements can include lines, borders, patterns, vector elements, patterns, color blocks, etc.

[0046] The typesetting prompt information is the user's requirement information for typesetting, which may include typesetting layout, alignment, line spacing, font, character spacing, color and other requirement information. For example, it may be: the title is placed at the top, the subtitle is placed at the bottom, the project is left-aligned, the line spacing is 1.5 times, the character spacing is 2 pixels, and the text color is dark blue.

[0047] Optionally, the client or service platform can set default layout prompt information. When the user does not limit a certain layout option, the default layout prompt information is used, so that the user can focus on the layout prompt information that needs to be specially designed without entering all the layout elements. For example, the line spacing can be set to 1 times by default. If the layout prompt information entered by the user does not include the line spacing, the default line spacing is used. Finally, the layout prompt information entered by the user and the default layout prompt information are used as the layout prompt information.

[0048] Step S202 , a visual model is used to extract the visual features of the background image.

[0049] The visual model may be implemented using a convolutional neural network (CNN), a recurrent neural network (RNN), or a generative adversarial network (GAN), etc. Optionally, step S202 may include: extracting visual features of the background image using a target visual information transformer.

[0050] The visual information transformer is a visual model based on the Transformer architecture. By utilizing its attention mechanism and adaptability, it can efficiently process complex and diverse visual information in images. The process of extracting the background image by the visual information transformer can be as follows: slicing the background image to decompose it into multiple small patches. Each patch can be regarded as the basic component of the background image and contains part of the information of the background image. Then, the visual information transformer converts these patches into a series of visual feature vectors. In this conversion process, the visual information transformer performs progressive attention calculations through a multi-layer attention mechanism to identify and extract the key visual features of the background image.

[0051] By adopting the target visual information transformer to extract the visual features of the background image, the visual features can provide rich visual context information for the subsequent target large language model, helping the target large language model to better understand the content of the background image, thereby more effectively performing graphic typesetting, which helps to improve the accuracy of typesetting.

[0052] In an optional embodiment, the graphic typesetting method may further include: fine-tuning the visual information converter using a first training data set to obtain a target visual information converter, wherein the first training data set includes N training background images, and the training background images include images of multiple colors, textures and / or styles.

[0053] Wherein, N can be a positive integer. In order to adapt the target visual information converter to various background images, N can be, for example, tens of thousands or hundreds of thousands. A single training image can be an image of one or more colors, one or more textures, one style or multiple styles, or an image composed of one or more colors, textures or styles.

[0054] The texture of the background image may include rough or smooth texture, geometric texture, organic texture, fibrous texture, granular texture and the like, and the image style may include realistic style, impressionist style, abstract style, comic style, watercolor style, retro style and the like.

[0055] A target visual information converter is obtained by fine-tuning the visual information converter using a first training set, wherein the first training data set includes N training background images, and the training background images include images of various colors, textures and / or styles. The target visual information converter obtained after fine-tuning can be better adapted to various background images and tasks of extracting visual features of background images, thereby improving the accuracy of visual feature extraction of background images.

[0056] Step S204 , the visual features, target typesetting elements and typesetting prompt information are input into the target large language model, and the target large language model is used to perform graphic typeset according to the visual features, target typesetting elements and typesetting prompt information.

[0057] The typesetting prompt information serves as the prompt input of the target large language. By parsing the typesetting prompt information, the target large language model can accurately understand the user's needs, such as a simple and elegant typesetting style, specific color matching, or highlighting specific elements. The target typesetting element is another input of the target large language model, which can be reasonably arranged based on the user needs obtained from the typesetting prompt information and the overall layout.

[0058] Specifically, the target large language model can first analyze the typesetting prompt information and convert it into specific typesetting instructions; then, combined with the visual features obtained by the visual information converter, determine the position, size and proportion of the target typesetting elements, so that the typesetting of the target typesetting elements and the image content corresponding to the visual features echo each other and coordinate and unify. For example, if the typesetting prompt information contains a requirement for a simple style, the main elements in the visual features are relatively simple, and the color distribution is relatively elegant, then the target large language model will generate a simple and clear typesetting plan based on this information; for another example, if the typesetting prompt information contains a requirement to highlight the key points, then the target large language model will determine the key elements in the background image based on the visual features and highlight them in the typesetting.

[0059] In order to make the target large language model more adaptable to the needs of typesetting, a large amount of training data can be used to fine-tune the large language model. In an optional embodiment, the graphic typesetting method can further include: fine-tuning the large language model using a second training data set to obtain a target large language model, wherein the second training data set includes a pairing combination of multiple training visual features, training typesetting elements, and training typesetting prompt information, the training visual features include visual features extracted from different background images, the training typesetting elements include a variety of different titles, project logos, projects and decorative elements, and the training typesetting prompt information includes at least part of the information in the position, size, color, font, font size, level, spacing, layout mode, image style, and image processing of the elements.

[0060] Among them, the layout method includes, for example, floating layout, positioning layout or adaptive layout; the image style includes, for example, filter effect, transparency or border shadow; the image processing includes, for example, cropping, scaling or rotating.

[0061] By using the second training data set to fine-tune the large language model to obtain a target large language model, the final target large language model can be better adapted to the needs of the typesetting task, thereby improving the accuracy and typesetting effect of the final typesetting.

[0062] In step S204, the target large language model is used to perform graphic typesetting, and the target large language model may be expanded into a multimodal model so that the model can directly output the final result of the graphic typesetting. Alternatively, the target large language model may output graphic typesetting information, and then the visualization result is obtained by rendering according to the graphic typesetting information.

[0063] In an optional embodiment, if Figure 3 As shown, in step S204, the target large language model is used to perform graphic layout according to visual features, target layout elements and layout prompt information, which may include steps S300 to S302: Step S300, obtaining graphic layout information of the target large language model.

[0064] Step S302: placing the target layout element in the background image according to the graphic layout information to obtain a layout image.

[0065] The graphic layout information output by the target large language model may include detailed information such as the final layout position, size, rotation angle, etc. of the elements, as well as various attributes of the text, such as font, font size, color, spacing, etc.

[0066] After obtaining the graphic layout information of the target large language model, the service platform can use the graphic rendering technology to place the target layout elements in the background image according to the graphic layout information to obtain the final layout image.

[0067] Please refer to Figure 4 , which is an example diagram of the principle of the graphic typesetting method. As shown in the figure, the background image is extracted from the visual features through VIT (Visual Information Transformer), and the extracted visual features are input into the large language model; the seal element (i.e., the target typesetting element) is input into the large language model as another input, and the user demand (i.e., the typesetting prompt information) is input into the large language model as the prompt of the large language; then, the large language model outputs the graphic typesetting information in JSON format; finally, the final typesetting image is rendered according to the graphic typesetting information.

[0068] In this optional embodiment, by acquiring the graphic layout information of the target large language model, the target layout elements are placed in the background image according to the graphic layout information to obtain a layout image. The final layout image can be effectively obtained based on the output results of the large language model, thereby effectively utilizing the large language model to obtain the desired layout effect.

[0069] In an optional embodiment, in step S302, the target layout element is placed in the background image according to the graphic layout information, such as Figure 5 As shown, it may further include steps S400 to S404: Step S400, determining whether there is a layout conflict in the target layout element.

[0070] Step S402, in the case of a layout conflict, adjusting the elements of the layout conflict using a preset adjustment method to obtain an adjustment result, wherein the preset adjustment method at least includes moving, scaling and rotating.

[0071] Step S404: placing the target layout element in the background image according to the adjustment result.

[0072] Specifically, the service platform checks the graphic layout information to determine whether there are elements of typesetting conflict in the graphic layout information, for example, by checking whether there are overlapping parts between every two elements to determine whether there is a conflict; in the case where there are no elements of typesetting conflict, the target typesetting element can be directly placed in the background image according to the graphic layout information to obtain a typesetting image; in the case where there are elements of typesetting conflict, the typesetting conflict elements are adjusted using a preset adjustment method, such as moving, scaling or rotating at least one of the elements of typesetting conflict so that the conflict disappears or is within an acceptable range, to obtain an adjustment result, and then the target typesetting element is placed in the background image according to the adjustment result.

[0073] In this optional embodiment, by determining whether a target typesetting element has a typesetting conflict, if a typesetting conflict exists, the element with the typesetting conflict is adjusted using a preset adjustment method, and the target typesetting element is placed in the background image according to the adjustment result. Optimization can be performed based on the output result of the large language model to generate a typesetting image without typesetting conflicts, thereby improving the accuracy and typesetting effect of the typesetting.

[0074] Embodiment 2 Figure 6 The block diagram of the graphic typesetting device 500 according to the second embodiment of the present application is schematically shown. The device 500 can be divided into one or more program modules, one or more program modules are stored in a storage medium, and are executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Figure 6 As shown, the apparatus 500 may include an acquisition module 510, an extraction module 520 and a typesetting module 530, wherein: The acquisition module 510 is used to acquire the background image, the target layout element and the layout prompt information; An extraction module 520, configured to extract visual features of the background image using a visual model; The typesetting module 530 is used to input the visual features, the target typesetting elements and the typesetting prompt information into a target large language model, and use the target large language model to perform graphic typeset according to the visual features, the target typesetting elements and the typesetting prompt information.

[0075] In an optional embodiment, the extraction module 520 is further configured to: A target visual information transformer is used to extract visual features of the background image.

[0076] In an optional embodiment, the graphic typesetting device 500 is further used for: The visual information transformer is fine-tuned using a first training data set to obtain the target visual information transformer, wherein the first training data set includes N training background images, and the training background images include images of multiple colors, textures and / or styles.

[0077] In an optional embodiment, the target layout element includes at least some of the elements of a title, a project logo, a project, and a decorative element.

[0078] In an optional embodiment, the graphic typesetting device 500 is further used for: A second training data set is used to fine-tune the large language model to obtain the target large language model, wherein the second training data set includes a plurality of paired combinations of training visual features, training typographical elements, and training typographical prompt information, the training visual features include visual features extracted from different background images, the training typographical elements include a variety of different titles, project logos, projects and decorative elements, and the training typographical prompt information includes at least part of the information on the position, size, color, font, font size, hierarchy, spacing, layout mode, image style, and image processing of the elements.

[0079] In an optional embodiment, the typesetting module 530 is further configured to: Obtaining graphic layout information of the target large language model; The target layout element is placed in the background image according to the graphic layout information to obtain a layout image.

[0080] In an optional embodiment, the typesetting module 530 is further configured to: Determine whether the target layout element has a layout conflict; In the case of a layout conflict, adjusting the elements of the layout conflict using a preset adjustment method to obtain an adjustment result, wherein the preset adjustment method at least includes moving, scaling and rotating; The target layout element is placed in the background image according to the adjustment result.

[0081] Embodiment 3 Figure 7 The schematic diagram of the hardware architecture of a computer device 10000 suitable for implementing a graphic typesetting method according to the third embodiment of the present application is shown. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Figure 7 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other through a system bus. Among them: The memory 10010 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 can be an internal storage module of the computer device 10000, such as a hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external storage device of the computer device 10000, such as a plug-in hard disk equipped on the computer device 10000, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10010 can also include both the internal storage module of the computer device 10000 and its external storage device. In this embodiment, the memory 10010 is generally used to store the operating system and various application software installed in the computer device 10000, such as program codes of the graphic typesetting method, etc. In addition, the memory 10010 can also be used to temporarily store various data that have been output or are to be output.

[0082] In some embodiments, the processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0083] The network interface 10030 may include a wireless network interface or a wired network interface, and the network interface 10030 is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0084] It should be pointed out that Figure 7 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementation of all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0085] In this embodiment, the graphic typesetting method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of the present application.

[0086] Embodiment 4 An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the graphic typesetting method in the embodiment are implemented.

[0087] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of a computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the computer-readable storage medium can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on the computer device, such as the program code of the graphic typesetting method in the embodiment, etc. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or are to be output.

[0088] Embodiment 5 An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0089] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0090] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A graphic typesetting method, characterized in that: The method comprises: Get the background image, target layout element and layout hint information; Extracting visual features of the background image using a visual model; The visual features, the target typeset elements and the typeset prompt information are input into a target large language model, and the target large language model is used to perform graphic typeset according to the visual features, the target typeset elements and the typeset prompt information.

2. The method according to claim 1, characterized in that The extracting the visual features of the background image by using a visual model comprises: A target visual information transformer is used to extract visual features of the background image.

3. The method according to claim 2, characterized in that The method further comprises: The visual information transformer is fine-tuned using a first training data set to obtain the target visual information transformer, wherein the first training data set includes N training background images, and the training background images include images of multiple colors, textures and / or styles.

4. The method according to any one of claims 1 to 3, characterized in that: The target layout elements include at least some of the elements among titles, project logos, projects and decorative elements.

5. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: A second training data set is used to fine-tune the large language model to obtain the target large language model, wherein the second training data set includes a plurality of paired combinations of training visual features, training typographical elements, and training typographical prompt information, the training visual features include visual features extracted from different background images, the training typographical elements include a variety of different titles, project logos, projects and decorative elements, and the training typographical prompt information includes at least part of the information on the position, size, color, font, font size, hierarchy, spacing, layout mode, image style, and image processing of the elements.

6. The method according to any one of claims 1 to 3, characterized in that: The step of performing graphic typeset according to the visual features, the target typeset elements and the typeset prompt information by using the target large language model includes: Obtaining graphic layout information of the target large language model; The target layout element is placed in the background image according to the graphic layout information to obtain a layout image.

7. The method according to claim 6, characterized in that Placing the target layout element in the background image according to the graphic layout information includes: Determine whether the target layout element has a layout conflict; In the case of a layout conflict, adjusting the elements of the layout conflict using a preset adjustment method to obtain an adjustment result, wherein the preset adjustment method at least includes moving, scaling and rotating; The target layout element is placed in the background image according to the adjustment result.

8. A graphic typesetting device, characterized in that: The device comprises: The acquisition module is used to obtain the background image, target layout element and layout prompt information; An extraction module, used for extracting visual features of the background image using a visual model; The typesetting module is used to input the visual features, the target typesetting elements and the typesetting prompt information into a target large language model, and use the target large language model to perform graphic typeset according to the visual features, the target typesetting elements and the typesetting prompt information.

9. A computer device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images