A method, system, and device for data chart illustration based on AIGC

Abstract

This application relates to a data chart illustration method, system, and device based on AIGC (AIGC Data Visualization Technology). The method includes: constructing a unified chart generation workflow based on a diffusion model, integrating multiple modules and models. When generating chart illustrations, a standard chart template is constructed according to the chart illustration specifications, and the template is then broken down to obtain a baseline structure. The template is processed by a script to generate background images for text and charts. The input control text is combined with the background image and input into the workflow. Various models process the background image based on the control text, and a control network controls the generation of the background image to obtain image content with semantic matching. Finally, the image content and text background image are mixed and output as the final chart. The technical solution of this application can significantly improve the automation and consistency of illustration production, reduce design costs, improve production efficiency, and significantly improve the usability and visual consistency of the final illustrations.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of AIGC data visualization, in particular to an AIGC-based data chart mapping method, system and device. BACKGROUND

[0002] In related technologies, chart mapping is widely used in many fields such as news reporting, advertising marketing, PPT making, and paper writing. Chart mapping generally has three methods: ①, using the chart generation function in the graphic design software such as PS and Ai alone by a graphic designer to design data charts. The chart mapping method usually produces static charts; ②, using existing professional data visualization software or online tool library to design, such as tableau, flourish, D3 (full name Data-Driven Documents, a data visualization tool library based on JavaScript) and the like; ③, directly writing code by a front-end engineer to design complex data visualization graphics. Among them, chart mapping methods ② and ③ usually produce interactive charts to make static output processing using interactive charts. Figure One

[0003] However, in the actual application of news data mapping and various data mapping, the above chart mapping methods all need to rely on manual designers to create data table mapping, which has the following obvious defects: first, the manual design efficiency is low, which cannot meet the demand of real-time generation of mapping for a large amount of news or data content; second, the levels of different designers are uneven, which leads to the difficulty in unifying the quality and style of mapping and the problem of inconsistent visual effects; third, due to the lack of systematic design standards and process constraints, the mapping results are often difficult to meet the established visual specifications, especially in large-scale production environment, it is difficult to guarantee the design consistency and professionalism; finally, the design levels of designers are different, and excellent designers are few, and the charts designed by most designers cannot reach a high degree of beauty.

[0004] Therefore, the existing chart mapping methods mainly have the problems of low chart mapping efficiency, poor specification of generated mapping, and poor beauty. SUMMARY

[0005] The present application provides an AIGC-based data chart mapping method, system and device, which mainly based on structured or semi-structured input content (such as news data, chart data, etc.), according to the preset design specification and visual template, realizes the ability of efficient, standardized and batch generation of mapping, controls the generated charts, improves the quality of chart mapping, and effectively solves the problems of low chart mapping efficiency, poor specification of generated mapping, and poor beauty in the prior art.

[0006] ​In a first aspect, the present application provides a method, comprising:

[0007] constructing a chart standard template based on a preset chart specification;

[0008] splitting the chart standard template according to dimensions and metrics, constructing a reference structure data of the chart, the reference structure data comprising a reference structure constructed by the dimensions and the metrics;

[0009] processing the chart standard template and the reference structure data through a preset script, generating a text base map and a chart base map, and taking the text base map, the chart base map, and obtained control text information as input data;

[0010] constructing a chart generation workflow based on a diffusion model, and adding a text encoder, a control network, and a fine-tuning model to the chart generation workflow, the control network being used to control content generation;

[0011] in the chart generation workflow, performing encoding control processing on the input data, and processing the input text base map and chart base map in a target dimension through a selected target fine-tuning model and the control network, to generate image content with structure control and semantic matching degree, and output a final chart.

[0012] Optionally, the chart generation workflow based on the diffusion model is constructed, and the text encoder, the control network, and the fine-tuning model are added to the chart generation workflow, comprising:

[0013] constructing the chart generation workflow, and creating a first loader according to a preset reference template, and loading the diffusion model to the chart generation workflow through the first loader;

[0014] in the chart generation workflow, loading the fine-tuning model through a second loader and controlling the generation proportion of the fine-tuning model;

[0015] in the chart generation workflow, introducing the text encoder, and introducing the control network to control the generated image content.

[0016] Optionally, in the chart generation workflow, the input data is subjected to encoding control processing, and the input text base map and chart base map are processed in a target dimension through a selected target fine-tuning model and the control network, to generate image content with structure control and semantic matching degree, and output a final chart, comprising:

[0017] in the chart generation workflow, selecting a target fine-tuning model of a target dimension based on the control text information, and generating a prompt word, processing the prompt word through the text encoder to obtain vector information, the target dimension comprising two dimensions and three dimensions;

[0018] Import the text background image and the chart background image as a loading image pair into the chart generation workflow, and identify the graphic outline from the chart background image.

[0019] Using vector information as the key guiding semantics for the diffusion model input, and leveraging the target fine-tuning model and control network, structurally controlled image content with semantic matching degree is generated based on graphical contours.

[0020] The blending module combines image content with text backgrounds to output the final chart.

[0021] Optionally, in the chart generation workflow, a target fine-tuning model is selected based on the control text information to select the target dimension, and prompt words are generated. The prompt words are then processed by a text encoder to obtain vector information, including:

[0022] The control text information is parsed to determine the diffusion model for matching scenarios and the target fine-tuning model for target dimensions. The control text information is then processed through a language model to generate prompt words.

[0023] The prompt words are input into a text encoder for conversion, resulting in vector information used for semantic understanding.

[0024] Optionally, identify graphic outlines from the chart's base image, including:

[0025] Output the chart base map to the fine-line preprocessor in the chart generation workflow to extract the graphic outlines from the base map. The graphic outlines are the outlines in the chart that have a clear boundary distinction.

[0026] Optionally, using vector information as key guiding semantics as input to the diffusion model, and leveraging a target fine-tuning model and control network, structurally controlled image content with semantic matching degree is generated based on graphical contours, including:

[0027] Vector information is input into the diffusion model to obtain the latent space parameters output by the model, which serve as the semantic direction to guide the generation of image content;

[0028] The target fine-tuning model generates chart style control information, and the diffusion model and the target fine-tuning model work together to affect the scale of the generated base map to obtain scale information.

[0029] Input the graphic contour as a constraint condition into the control network to generate control constraint information;

[0030] In the sampler of the diffusion model, image content is generated based on latent space parameters, chart style control information, scale information, and control constraint information.

[0031] Optionally, a pre-defined script processes the chart standard template and baseline structure data to generate text and chart background images, including:

[0032] Input the standard chart template and baseline structure data into the preset automation script;

[0033] In the automated script, baseline structure data is loaded through a preset module, and a preset drawing module is used to draw the base map, resulting in text base maps and chart base maps.

[0034] Optionally, in the automated script, baseline structure data is loaded through a preset module, and a preset drawing module is used to draw the base map, resulting in text and chart base maps, including:

[0035] The base map framework is constructed by parse and process the standard chart template and baseline structure data through automated scripts. The base map framework includes key components.

[0036] Based on the base map framework, the base map is drawn using preset drawing tools. Using the specified parameters in the chart standard template as a benchmark, the preset drawing functions are used to set parameters at the corresponding positions of key components to obtain the text base map.

[0037] Using drawing tools, update attribute labels based on key components and draw the chart base map.

[0038] Secondly, this application provides a data chart and graphing system based on AIGC, including:

[0039] The chart design module is used to build standard chart templates based on preset chart specifications;

[0040] The chart splitting module is used to split the standard chart template according to dimensions and measures to build the baseline structure data of the chart. The baseline structure data contains the baseline structure built by dimensions and measures.

[0041] The base map generation module is used to process chart standard templates and baseline structure data through preset scripts to generate text base maps and chart base maps, and uses text base maps, chart base maps and acquired control text information as input data;

[0042] The workflow building module is used to build a graph generation workflow based on a diffusion model, and adds a text encoder, a control network, and a fine-tuning model to the graph generation workflow. The control network is used to control content generation.

[0043] The chart generation module is used to encode and control the input data in the chart generation workflow. Through the selected target fine-tuning model and control network, it processes the input text background and chart background in the target dimension to generate image content with controlled structure and semantic matching degree, and outputs the final chart.

[0044] Thirdly, this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0045] Memory, used to store computer programs;

[0046] When a processor executes a program stored in memory, it implements the steps of the AIGC-based data chart plotting method as described in any embodiment of the first aspect.

[0047] In summary, this application's embodiments construct a unified chart generation workflow based on a diffusion model, integrating multiple modules and models. During chart generation, a standard chart template is first constructed according to the chart specifications. Based on the chart's characteristics, the standard template is split according to dimensions and metrics to obtain the chart's baseline structure. Then, the design template and baseline structure are processed by a script to generate background images for text and the chart. Finally, the user's control text is input into the workflow along with the background images. The integrated models in the workflow process the two background images according to the input text and control the generation of the background images through a control network, resulting in image content with semantic matching. Finally, the image content and text background images are blended to output the final chart. Therefore, the technical solution of this application can significantly improve the automation and consistency of chart production, reduce design costs, increase production efficiency, and significantly improve the usability and visual uniformity of the final chart. It has good application and promotion prospects and solves the problems of low chart matching efficiency, poor chart specifications, and poor aesthetics in existing technologies. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 A flowchart illustrating a data chart matching method based on AIGC provided in this application embodiment;

[0051] Figure 2 This is a flowchart illustrating the steps of an AIGC-based data chart plotting method provided in an optional embodiment of this application.

[0052] Figure 3This is an example image of a bar chart generation template provided in this application;

[0053] Figure 4 This is an example image of the text background provided in this application;

[0054] Figure 5 This is an example diagram of the underlying image provided in this application;

[0055] Figure 6 This is a flowchart of the diagram generation process provided as an example in this application;

[0056] Figure 7 This is an example of a 2D chart generation result provided in this application;

[0057] Figure 8 This is an example of a 3D chart generation result provided in this application;

[0058] Figure 9 A structural block diagram of a data chart illustration system based on AIGC provided in this application embodiment;

[0059] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] To facilitate understanding of the embodiments of this application, further explanations and descriptions will be provided below in conjunction with the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of this application.

[0062] Figure 1 A flowchart illustrating a data chart matching method based on AIGC provided in this application embodiment, the method specifically including the following steps:

[0063] Step 110: Based on the preset chart and graph specifications, construct a standard chart template.

[0064] In this embodiment, the chart specification refers to a series of pre-designed specifications for different charts. It is mainly based on the attributes and composition of different charts, and the specifications involved in matching chart types from the perspective of data dimension attributes. For example, a bar chart usually includes a title, chart body, horizontal and vertical axes, labels, and data source descriptions.

[0065] This embodiment pre-designs a standard template for charts based on the chart specifications. The standard template mainly involves two parts: the chart's composition structure and the corresponding parameters. For example, for the title, it usually includes the font size, font format, and the position of the title in the chart.

[0066] Therefore, in order to reduce reliance on designers' understanding and judgment of data, this embodiment pre-generates corresponding templates, so that users no longer need to understand the data before drawing. They can directly use the template or make simple modifications to the template to obtain the corresponding standard chart template.

[0067] It should be noted that the data chart matching method based on AIGC proposed in this application can be applied to the generation of different charts, such as bar charts, line charts, pie charts, heat maps, etc. This embodiment does not limit the specific chart.

[0068] Step 120: Split the standard chart template according to dimensions and metrics to construct the baseline structure data for the chart.

[0069] The baseline structure data contains a baseline structure built from dimensions and measures.

[0070] In this embodiment, the standard chart template is split according to dimensions and measures, allowing all chart data to be processed according to the template's format. After processing, the basic structure of the chart can be successfully constructed. For example, taking a bar chart, its basic structure includes horizontal and vertical axes; therefore, dimensions can be used as rows and measures as columns. After processing, the resulting basic structure uses dimensions as the chart's horizontal axis and measures as the chart's vertical axis. This completes the standardization of the chart structure.

[0071] Step 130: Process the chart standard template and baseline structure data through a preset script to generate text background and chart background, and use the text background, chart background and the acquired control text information as input data.

[0072] In this embodiment, the control text information is mainly used as one of the constraints, enabling the AI ​​to generate accurate charts and illustrations. The control text information can be a text description input by the user, or, in the news industry, news content, etc. This embodiment does not impose any restrictions on this.

[0073] This embodiment implements a chart generation method based on AIGC. Considering the strong randomness of AI in AIGC chart generation, while data charts require accurate proportions, this embodiment uses a preset script (such as an automated script written in Python) to process the standard chart template and baseline structure data to create the base image.

[0074] When creating the basemaps, reliable data chart basemaps are produced using data visualization modules such as echarts and matplotlib. Specifically, firstly, relevant functions are used to transform the baseline data structure; for example, for a bar chart, functions can be used to convert it into row and column data. Then, plotting tools (such as matplotlib templates) are used to draw the corresponding text basemaps and chart basemaps. Finally, the two basemaps and control text information are used as reference inputs in the subsequent AI image generation process.

[0075] Therefore, this embodiment pre-creates a reliable data chart base map, which can effectively constrain the output range of AI and thus accurately control the AI's generation results when using the control network later.

[0076] Step 140: Build a graph generation workflow based on the diffusion model and add the text encoder, control network, and fine-tuning model to the graph generation workflow.

[0077] The control network is used to control content generation.

[0078] The diffusion model used in this embodiment can be Stable Diffusion, a variant of the Latent Diffusion Model (LDM). Its workflow involves three parts: a training phase, the diffusion process, and latent space operations. It is primarily used for text-to-image or image-to-image generation, automatically generating high-quality images based on user-input text descriptions. This embodiment addresses the problem of relying on manual image creation in industries such as news reporting by developing a standard AI-automated image creation process, utilizing AI to efficiently and quickly generate a large number of controllable charts.

[0079] In this implementation, this example uses Stable Diffusion as a foundation to build a chart generation workflow for creating automated charts (hereinafter referred to as the comfyUI workflow). First, the Stable Diffusion model is loaded according to the standard Stable Diffusion template and set up in the comfyUI workflow. Then, multiple modules, loaders, and models are set up in the comfyUI workflow. The main modules / models / loaders include: a text encoder (such as the CLIP text encoder), a fine-tuning model (such as the LoRA fine-tuning model), multiple image loading modules (for loading base maps), and a control network (such as ControlNet).

[0080] In practical implementation, considering that charts include two-dimensional (2D) charts and three-dimensional (3D) charts, and the different application scenarios of charts (such as applications in news, advertising and marketing, PPT creation, academic writing, etc.), comfyUI workflow can set different LoRAs. In actual applications, the LoRA of the image to be processed can be selected according to the application scenario or dimension (2D or 3D chart).

[0081] Step 150: In the chart generation workflow, the input data is encoded and controlled, and the input text background and chart background are processed in the target dimension through the selected target fine-tuning model and control network to generate image content with controlled structure and semantic matching degree, and the final chart is output.

[0082] This embodiment can generate 2D or 3D charts (usually 3D stereoscopic charts). By parsing the input control text, it can identify whether the desired chart and illustrations are 2D or 3D, and determine the target dimension. Based on the target dimension, the LoRA for chart generation is selected to adapt to different application scenarios and chart production needs.

[0083] Specifically, in the comfyUI workflow, a fine-tuning model and a control network are selected based on the input control text. Then, the control text is parsed using the CLIP text encoder and input into Stable Diffusion. LoRA controls the scale of the generated chart based on the two input background maps. Text background maps are generated by setting parameters for each part of the output, and the effect of the generated background maps is controlled by ControlNet to obtain structurally controlled image content with semantic matching. Finally, the generated text background maps and image content are blended to output the final chart.

[0084] As can be seen, this embodiment first designs a general chart template. Then, it generates text and chart background images using the template. Finally, AI, through control of the chart background images, can generate content with a fixed structure, and by overlaying the text background images using an image overlay algorithm, a chart meeting the requirements can be generated. This embodiment significantly improves the automation and consistency of image production, reduces design costs, increases production efficiency, and significantly improves the usability, visual consistency, and aesthetics of the final images, demonstrating good application and promotion prospects.

[0085] Reference Figure 2 The diagram illustrates a step-by-step flowchart of an AIGC-based data chart plotting method according to an optional embodiment of this application. The method may specifically include the following steps:

[0086] Step 210: Based on the preset chart and graph specifications, construct a standard chart template.

[0087] In related technologies, existing techniques rely on the designer's understanding and judgment of the data, requiring the designer to actively set chart types that match the data. This can lead to inaccurate data representation, as some data has relatively fixed and common chart types to match, while not all chart types can be matched. If a match is not found, the data representation will be incorrect or cause misunderstanding.

[0088] To address this, this embodiment utilizes normalization processing to automatically understand and match chart types. Specifically, this embodiment pre-creates templates based on the data dimension attributes and matching chart types, eliminating the need to understand the data before drawing; instead, modifications are made to the template, thus eliminating reliance on the designer's understanding and judgment of the data.

[0089] Reference Figure 3 As shown, a bar chart is used as an example:

[0090] The chart design specifications for the bar chart in this embodiment include: overall canvas size: 750 × 1200 px, standard vertical bar chart template, adapted to mobile infographic content and automated chart platform output. Figure 3 In a chart, the components from top to bottom are: title area, chart body area, horizontal axis label area, and annotation area (including but not limited to the data source explanation area).

[0091] The corresponding attribute parameters include:

[0092] Title area font size: 44 px, bold font, centered alignment, near the top of the canvas, line height equal to font height, for easy and consistent layout.

[0093] The bar chart area dimensions are: width 505 px (from the leftmost bar to the rightmost bar), height 600 px (to accommodate the height of the largest bar), and the bars are arranged horizontally evenly within the 505 px bar area with equal spacing between them and the tops of the bars aligned to highlight the relative height differences of the data.

[0094] Spacing instructions: Left margin: 155 px from the start of the column area to the left edge of the canvas; Right margin: 90 px from the end of the column area to the right edge of the canvas; Top of the column area to the bottom of the title area: 75 px. Auxiliary alignment lines (reference lines) are distributed within the column area to reference the column height scale. These lines are evenly spaced for easy visual comparison of annotation text positions. If there are illustrations or arrow text, they are uniformly aligned to the left, aligned with the left edge of the column area, with a spacing of 34 px extending to the left from the start of the column area.

[0095] The horizontal axis label area uses the Semibold font with a font size of 20 pt. Each label is centered and aligned to the bottom of the corresponding column. The label text is 10 px from the bottom of the column, and the line height is 80 px. It is suitable for scenarios such as line breaks in Chinese and English labels or phonetic characters.

[0096] The bottom data source description area has a blank top margin, 60 px from the bottom of the chart area. The top and bottom margins within the block are each 75 px. The data source title is 20 px in font size and bold. The data source content is 18 px in font size and regular font. The line height of the text is the same as the font size.

[0097] Using the above chart and graph specifications, a corresponding standard chart template can be generated. The parameters involved in the standard chart template can be consistent with the chart and graph specifications, or they can be adjusted according to actual needs.

[0098] It should be noted that the above chart template design is only one type of bar chart. This is just one example of chart templates available. Those skilled in the art can freely design chart templates according to actual needs, including chart format, illustration specifications, and chart attribute parameters. No restrictions are placed on this. For example, slightly different chart templates can be designed specifically for different publishing channels, such as Weibo, WeChat, mobile, and PC. To achieve better results, the specific font, font size, and line spacing may be changed.

[0099] Step 220: Split the standard chart template according to dimensions and metrics to construct the baseline structure data for the chart.

[0100] The baseline structure data contains a baseline structure built from dimensions and measures.

[0101] Step 230: Process the chart standard template and baseline structure data through a preset script to generate text background and chart background, and use the text background, chart background and the acquired control text information as input data.

[0102] Optionally, the above-mentioned process of processing chart standard templates and baseline structure data through preset scripts to generate text and chart base maps may include the following sub-steps:

[0103] Sub-step 2301: Input the chart standard template and baseline structure data into the preset automated script.

[0104] Sub-step 2302: In the automated script, the baseline structure data is loaded through a preset module, and the base map is drawn using a preset drawing module to obtain the text base map and the chart base map.

[0105] A unified description is provided for sub-steps 2301 and 2302:

[0106] In practical implementation, since the chart template has already been generated according to standardization, and the baseline structure of the chart has been defined through dimension and metric decomposition, corresponding automated scripts can be written based on preset chart illustration specifications or standard chart templates, such as automated scripts written in Python. These automated scripts are then used to create base maps. These scripts can integrate various related transformation functions and can be integrated with or associated with charting tools. By processing the chart standard template and baseline structure through the script, text and chart base maps can be automatically drawn and generated using drawing tools, serving as one of the reference inputs for AI image generation.

[0107] In an optional embodiment, this embodiment loads baseline structure data through a preset module in the automation script, and uses a preset drawing module to draw the base map to obtain text base map and chart base map. Specifically, it may include: using the automation script to parse and process the chart standard template and baseline structure data to construct a base map framework, which includes key components; based on the base map framework, using a preset drawing tool to draw the base map, and using the specified parameters in the chart standard template as a benchmark, using preset drawing functions to set parameters at the corresponding positions of the key components to obtain the text base map; using the drawing tool to update the attribute calibration based on the key components to draw the chart base map.

[0108] Specifically, the background image frame, as a crucial component of text / chart background images, mainly comprises the text background image frame and the chart background image frame. The key components of the text background image frame include the main title, chart axes, axis labels, data source, and source information, corresponding to the structure in the standard chart template. The key components of the chart background image frame include the chart shape and rows / columns. The parameters related to the background image frame correspond to the parameters in the standard chart template.

[0109] As an example, this embodiment uses the pandas `read_excel` function for plotting and matplotlib for plotting. Taking a bar chart as an example, the pandas module loads the template data (i.e., the standard chart template and baseline structure) after data template processing. The pandas `read_excel` function can directly convert the row and column data, and then the matplotlib module is used to plot the corresponding text background and chart background. For the text background, please refer to... Figure 4 As shown, the base chart can be found here. Figure 5 As shown.

[0110] The following provides illustrative examples of the parameters used when creating text and chart backgrounds:

[0111] The text-based background image consists of a main title, chart axes, axis labels, data source, and source information. First, it's plotted using matplotlib, with the figure size specified by the template. Then, the axis positions are determined using the `figure.add_axes` function, and these positions must be set according to the parameters provided in the template. The axes and axis labels are black, and the chart shape color is white. This ensures the white chart shape blends in with the background color after the mask is generated. The text information, such as the main title, data source, and source information, is drawn in the corresponding blank positions using the `text` function from the `opencv` module, with the position also determined by the template. The chart data is loaded from pandas using `axes.bar`, with rows representing the x-axis and labels, and columns representing the height or scale. Finally, `savefig` is used to output the image. The background image is also plotted using matplotlib, with the graph color set to black, and the `show` attribute of the axes, labels, and title chart components set to `false`. Finally, `savefig` is used to obtain the background image.

[0112] Step 240: Build a graph generation workflow based on the diffusion model and add the text encoder, control network, and fine-tuning model to the graph generation workflow.

[0113] The control network is used to control content generation.

[0114] In a specific implementation, this application embodiment constructs a chart generation workflow based on a diffusion model and adds a text encoder, a control network, and a fine-tuning model to the chart generation workflow. Specifically, it may include: constructing a chart generation workflow and creating a first loader according to a preset baseline template, and loading the diffusion model into the chart generation workflow through the first loader; in the chart generation workflow, loading the fine-tuning model through a second loader and controlling the generation ratio of the fine-tuning model; introducing a text encoder into the chart generation workflow and introducing a control network to control the generated image content.

[0115] In this embodiment, the models that can be set in the comfyUI workflow may include CLIP text encoder, loRA, Stable Diffusion base model, etc., and are equipped with K sampler, VAE decoder, ControlNet, ControlNet loader, and Canny fine line preprocessor.

[0116] Specifically, a checkpoint loader is created according to the Stable Diffusion standard template to load the Stable Diffusion base model, the positive and negative cue words CLIP text encoder, and the LoRA model. The LoRA model's LoRA heap loader loads the LoRA and controls its generation ratio. The empty latent space has its width and height set to the same size as the base image. The K-sampler receives all preceding inputs and performs sampling calculations according to the settings. The VAE decoder outputs the pixel space.

[0117] Building upon the above, add a loading image to load the base map. Select control_v11p_sd15_canny or another Canny model from the ControlNet loader. The Canny fine-line preprocessor is used to control the Canny effect; add it to the ControlNet application and connect the parameter reference. Additionally, add a preview image to preview the Canny model processing effect.

[0118] In this embodiment, the generation of 2D and 3D effect charts can be achieved by selecting different LoRA models. Unlike general 3D chart production, it is possible to 3D simply by replacing the input model with the result generated by a LoRA model trained on a 3D image.

[0119] Step 250: In the chart generation workflow, the target fine-tuning model of the target dimension is selected based on the control text information, and prompt words are generated. The prompt words are processed by the text encoder to obtain vector information.

[0120] The target dimension includes two-dimensional and three-dimensional dimensions.

[0121] In this embodiment, the chart background image (as a graphic mask), text background image, and control text are input into the Stable Diffusion-based comfyUI workflow. Through multi-step collaborative processing, a chart that conforms to specifications and is both aesthetically pleasing and accurate is finally generated. The comfyUI workflow can be configured with multiple control options, such as enabling image control options like "Perfect Items" and "Hard Edges" (other parameters can be flexibly configured according to needs). The workflow then runs and generates structured image content. Figure 6 The corresponding flowchart is shown below.

[0122] Specifically, for the input control text, taking news text as an example, this text is crucial in guiding the AI ​​to generate a style that matches the context. The comfyUI workflow selects a base model and a LoRA model that fits the scenario based on the input control text. For example, if the text involves "technology," a LoRA model trained on a technology style is used to ensure the chart style matches the theme; if the text involves "3D," a LoRA model capable of achieving 3D effects is selected. Furthermore, based on the control text, a large language model (such as LLM) can be used to parse the control text and generate prompts, i.e., prompts that meet the requirements of the AIGC model or StableDiffusion. These prompts are converted into AI-understandable vectors by the CLIP text encoder and serve as one of the inputs to the StableDiffusion base model, guiding the AI ​​to generate artistic graphics consistent with the text context within the structural constraints of ControlNet (e.g., using cool colors and geometric textures for technology-themed charts), thus guiding the semantic direction of image generation.

[0123] Therefore, this embodiment controls the semantics of the image by controlling the text, thereby guiding AI to generate image content with a higher degree of semantic matching, thus enhancing the consistency and expressiveness of the image with the news context.

[0124] For example, the prompt word may consist of the following:

[0125] Description of the chart topic (e.g., "User growth data in the technology industry");

[0126] Artistic style requirements (e.g., "minimalism, high-saturation color scheme");

[0127] Detail constraints (e.g., "3D stereoscopic bar chart with a metallic texture").

[0128] Optionally, in the above-mentioned chart generation workflow, selecting the target fine-tuning model for the target dimension based on the control text information and generating prompt words, and processing the prompt words through a text encoder to obtain vector information, may include: parsing the control text information to determine the diffusion model matching the scenario and the target fine-tuning model for the target dimension, and processing the control text information through a language model to generate prompt words; inputting the prompt words into the text encoder for conversion processing to obtain vector information for semantic understanding.

[0129] Step 260: Import the text background image and the chart background image as a loading image pair into the chart generation workflow, and identify the graphic outline from the chart background image.

[0130] In the specific implementation, considering the highly random nature of AIGC graph generation and the requirement for accurate scale in data charts, this embodiment pre-generates a rough chart base image and then uses the ControlNet module to constrain the output range of the AI, thereby precisely controlling the generated results.

[0131] Specifically, refer to Figure 6 As shown, this embodiment introduces an image loading module to load text and chart background images respectively. The loaded background images are imported into the ComfyUI workflow as image pairs. After loading the chart background image, the ControlNet loader and Canny fine-line preprocessor can be used to extract the graphic outline from the background image, such as... Figure 5 The black outline of the chart's base image. This outline can be used to confine AI-generated content within its bounds, preventing the randomness of AIGC from disrupting the structure of the data graph. For example, in a bar chart, it can prevent AI from randomly altering the height ratio of the bars.

[0132] Optionally, the above-mentioned identification of graphic outlines from the chart background may include: outputting the chart background to the fine line preprocessor in the chart generation workflow, and extracting the graphic outlines in the background, wherein the graphic outlines are outlines in the chart that have a clear boundary distinction function.

[0133] For example, the height ratio and color of the bars in a bar chart, and the shape and size of the pie chart in a pie chart, can all serve as outlines with clear boundaries. This example does not limit this.

[0134] Step 270: Using vector information as the key guiding semantics for the diffusion model input, and utilizing the target fine-tuning model and control network, image content with controlled structure and semantic matching degree is generated based on the graphic contour.

[0135] In the processing of AI chart backgrounds, control network constraints are used to control the AI ​​processing to ensure the controllable structure of the data graphs. Specifically, the chart background is input as a graphic mask into the ComfyUI workflow. After loading the chart background, the ControlNet loader is used to ensure that the AI-generated graphics (such as the bar structure in an artistic bar chart) strictly follow the structural constraints of the background, guaranteeing the accuracy of data representation and generating image content with controlled structure and semantic matching.

[0136] In this embodiment, Stable Diffusion determines the guiding semantics based on the input vector information and outputs the corresponding parameters; the LoRA model controls the generation ratio of the base map based on the input base map; ControlNet constrains the input graphic contour and outputs constraint information. Finally, the image content is generated by combining the parameters.

[0137] Optionally, the aforementioned guiding semantics, which use vector information as the key input to the diffusion model, utilizes a target fine-tuning model and a control network to generate structurally controlled image content with semantic matching based on graphic contours. This can include: inputting vector information into the diffusion model to obtain latent space parameters output by the model, which serve as the semantic direction guiding the generation of image content; generating chart style control information through the target fine-tuning model, and using the diffusion model and the target fine-tuning model together to affect the scale of the generated base image, thus obtaining scale information; inputting the graphic contours as constraints into the control network to generate control constraint information; and generating image content in the sampler of the diffusion model based on the latent space parameters, chart style control information, scale information, and control constraint information.

[0138] As an example, in the comfyUI workflow, the collaborative processing of each part can be understood as follows:

[0139] The StableDiffusion base model is initialized through the checkpoint loader, which receives the CLIP-encoded Prompt vector.

[0140] The LoRA heap loader loads the selected LoRA model and controls its generation ratio (affecting the intensity of the chart style).

[0141] The empty latent space is set to have the same width and height dimensions as the base map to ensure that the generated image is size-matched.

[0142] The K-sampler receives all the above inputs (model, Prompt, ControlNet constraints, latent space parameters, etc.), performs sampling calculations, and generates latent space features that meet the constraints.

[0143] Step 280: The image content and text background are blended using the blending module to output the final chart.

[0144] In this embodiment, when generating the blended image, firstly, the VAE decoder in the comfyUI workflow converts the latent space features output by the K-sampler into a pixel-space image (i.e., constrained artistic graphic content). Then, the image blending module overlays the graphic content output by the VAE with the previously loaded text background image, ultimately generating a chart containing complete text information, with accurate structure and matching style. For example, the generated final image can be referenced... Figure 7 As shown, the bar chart illustrates the changes in the area of ​​lotus flowers in the park, thus generating a lotus flower map, which enriches the generated data charts. It is evident that, thanks to the randomness of AI, after controlling the base map requirements, the richness of the output data charts far exceeds the creative capabilities of an average designer in the same amount of time.

[0145] Furthermore, refer to Figure 8 As shown, loRA trained using 3D effects can generate corresponding 3D effect charts under the same scheme. Figure 8 In the example shown, a bar chart is used to display the changes in the export volume of hardware equipment in a certain country / region. To fit the theme, the generated 3D bar chart is constructed with 3D bars in the style of hardware equipment, enriching the chart and resulting in an aesthetically pleasing 3D chart.

[0146] In summary, this application's embodiments construct a unified chart generation workflow based on a diffusion model, integrating multiple modules and models. When generating chart illustrations, a standard chart template is constructed according to the chart illustration specifications, and this template is then broken down to obtain a baseline structure. The template is processed by a script to generate background images for text and charts. The input control text is combined with the background image and input into the workflow. Various models process the background image based on the control text, and a control network controls the generation of the background image, resulting in image content with semantic matching. Finally, the image content and text background image are mixed and output as the final chart. It is evident that the solution in this application's embodiments can significantly improve the automation and consistency of illustration production, reduce design costs, increase production efficiency, and significantly improve the usability and visual consistency of the final illustrations.

[0147] The data chart illustration method based on AIGC provided in this application has the following beneficial effects:

[0148] ① Automatically understands and matches chart types. It can accurately match chart types, avoiding data expression errors or misunderstandings.

[0149] ② This application resolves the issue of uncontrollable content in AIGC-generated data areas, controlling the content of AIGC-generated images within meaningful charts and graphics. It fully analyzes and resolves the contradiction between freely created AIGC and controllable information transmission, achieving optimal results in both information accuracy and aesthetic design during data visualization and transmission.

[0150] ③ Enrich the design elements of the chart area and enhance the artistic expression. Specifically, existing chart design tools offer relatively simple chart area designs, with chart blocks typically filled in a single color and rarely featuring more complex artistic designs. However, the technical solution provided in this application conducts an in-depth analysis of different chart types, distinguishing large-block graphics such as bar charts and pie charts, and deeply integrating the design of these large blocks with the advantages of AIGC image generation. This makes the data content in the chart area more artistic, and the technology also supports the rapid generation of artistically designed 3D charts, rather than ordinary 2D front-view charts. Relying on AI technology, the artistic design expression of the chart area is greatly enriched and can be combined with the theme of the data presentation, improving the artistic expressiveness of the data charts.

[0151] ④ The design of the chart area, text outline, and background can be completed in one integrated process, greatly improving the integration of elements. Specifically, in existing technologies, designers need to design the chart, font, and background image separately, spending a lot of time adjusting different elements to make them more aesthetically pleasing and integrated. In contrast, the technical solution provided in this application allows for the integrated generation of different elements of the entire chart, automatically matching colors, elements, and outline graphics, etc., quickly improving the overall integration of the chart.

[0152] ⑤ Lowering the design threshold for charts and significantly improving production efficiency and speed. Specifically, in existing chart design that relies on designers, a skilled chart designer takes approximately 1-3 hours to create a simple 2D chart. This is especially true for charts designed using the second or third path mentioned above, which typically require importing images into graphic design software and spending considerable time on artistic design and processing. All artistic design cannot be completed within a single software program. Furthermore, creating a beautiful 3D chart requires a skilled chart designer to first build a detailed 3D model of the chart, then adjust the angles to output a 2D result. This process not only takes at least 1-5 days but is also technically challenging, making it difficult for designers without modeling skills to complete. The technical solution provided in this application can significantly shorten the production time of high-quality 2D and 3D charts. According to usage verification, this technical solution takes only 10 minutes to create a 2D or 3D chart, and most chart design tasks can be completed within the corresponding tools, improving the overall quality of the charts.

[0153] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should know that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps may be performed in other orders or simultaneously.

[0154] like Figure 9 As shown in the figure, this application embodiment also provides a data chart and illustration system 900 based on AIGC, including:

[0155] The chart design module 910 is used to build standard chart templates based on preset chart specifications;

[0156] The chart splitting module 920 is used to split the standard chart template according to dimensions and measures to build the baseline structure data of the chart. The baseline structure data contains the baseline structure built by dimensions and measures.

[0157] The base map generation module 930 is used to process the chart standard template and baseline structure data through a preset script to generate text base maps and chart base maps, and uses the text base map, chart base map and the acquired control text information as input data;

[0158] Workflow building module 940 is used to build a graph generation workflow based on a diffusion model, and adds a text encoder, control network, and fine-tuning model to the graph generation workflow. The control network is used to control content generation.

[0159] The chart generation module 950 is used to encode and control the input data in the chart generation workflow, and through the selected target fine-tuning model and control network, process the input text background and chart background in the target dimension to generate structured image content with semantic matching degree and output the final chart.

[0160] It should be noted that the AIGC-based data chart mapping system provided in this application can execute the AIGC-based data chart mapping method provided in any embodiment of this application, and has the corresponding functions and beneficial effects of the execution method.

[0161] In practical implementation, the aforementioned AIGC-based data chart and graph illustration system can be integrated into a device, enabling the device to generate text and chart backgrounds, and, in conjunction with control text, generate structured image content using AIGC technology, and output the final image. As an electronic device, it achieves efficient and rapid generation of a large number of controllable charts using AI. This electronic device can consist of two or more physical entities, or it can consist of a single physical entity. For example, the electronic device can be a personal computer (PC), a computer, a server, etc. This application embodiment does not impose specific limitations on this.

[0162] like Figure 10 As shown, this application embodiment provides an electronic device, including a processor 111, a communication interface 112, a memory 113, and a communication bus 114. The processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114. The memory 113 is used to store computer programs. When the processor 111 executes the program stored in the memory 113, it implements the steps of the data chart illustration method based on AIGC provided in any of the aforementioned method embodiments. For example, the method may include the following steps: constructing a standard chart template based on a preset chart specification; splitting the standard chart template according to dimensions and measures to construct baseline structure data for the chart, the baseline structure data containing a baseline structure constructed by dimensions and measures; processing the standard chart template and baseline structure data through a preset script to generate text background images and chart background images, and using the text background images, chart background images, and acquired control text information as input data; constructing a chart generation workflow based on a diffusion model, and adding a text encoder, a control network, and a fine-tuning model to the chart generation workflow, the control network being used to control content generation; in the chart generation workflow, encoding and controlling the input data, and processing the input text background images and chart background images in the target dimension through the selected target fine-tuning model and control network to generate structured image content with semantic matching degree, and outputting the final chart.

[0163] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the AIGC-based data chart plotting method provided in any of the foregoing method embodiments.

[0164] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0165] The above are merely specific embodiments of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An AIGC-based data chart mapping method, characterized in that, The method comprises the following steps: Based on the preset chart specification, a chart standard template is constructed; According to the dimensions and the metrics, the chart standard template is split and processed to construct a baseline structure data of the chart, and the baseline structure data comprises a baseline structure constructed by the dimensions and the metrics; The chart standard template and the baseline structure data are processed through a preset script to generate a text base map and a chart base map, and the text base map, the chart base map and the obtained control text information are taken as input data; A chart generation workflow based on a diffusion model is constructed, and a text encoder, a control network and a fine-tuning model are added to the chart generation workflow, and the control network is used to control content generation; In the chart generation workflow, the input data is encoded and controlled, and the selected target fine-tuning model and the control network are used to process the input text base map and chart base map in the target dimension to generate image content with structural control and semantic matching degree, and finally output the chart; In the chart generation workflow, the input data is encoded and controlled, and the selected target fine-tuning model and the control network are used to process the input text base map and chart base map in the target dimension to generate image content with structural control and semantic matching degree, and finally output the chart, comprising: in the chart generation workflow, the target fine-tuning model of the target dimension is selected based on the control text information, and a prompt word is generated, the prompt word is processed by the text encoder to obtain vector information, the target dimension includes two dimensions and three dimensions; the text base map and the chart base map are taken as a pair of loaded images, and are imported into the chart generation workflow, and the graph contour is identified from the chart base map; the vector information is taken as the key guiding semantics of the diffusion model input, and the target fine-tuning model and the control network are used to generate image content with structural control and semantic matching degree based on the graph contour; the image content and the text base map are mixed through a mixing module to output the final chart; The vector information is taken as the key guiding semantics of the diffusion model input, and the target fine-tuning model and the control network are used to generate image content with structural control and semantic matching degree based on the graph contour, comprising: the vector information is input into the diffusion model to obtain the latent space parameters of the model output as the semantic direction of guiding image content generation; the target fine-tuning model generates chart style control information, and the diffusion model and the target fine-tuning model jointly act on the scale of the generated base map to obtain scale information; the graph contour is input into the control network as a constraint condition to generate control constraint information; in the sampler of the diffusion model, the latent space parameters, the chart style control information, the scale information and the control constraint information are used to generate image content.

2. The method of claim 1, wherein, A chart generation workflow based on a diffusion model is constructed, and a text encoder, a control network and a fine-tuning model are added to the chart generation workflow, comprising: A chart generation workflow is constructed, and a first loader is created according to a preset baseline template, and the diffusion model is loaded into the chart generation workflow through the first loader; In the chart generation workflow, the fine-tuning model is loaded through a second loader and the generation scale of the fine-tuning model is controlled; In the chart generation workflow, a text encoder is introduced, and a control network is introduced to control the generated image content.

3. The method of claim 1, wherein, In the chart generation workflow, a target fine-tuning model of a target dimension is selected based on the control text information, and a prompt word is generated, the prompt word is processed by a text encoder to obtain vector information, including: The content of the control text information is parsed to determine the diffusion model matching the scene and the target fine-tuning model of the target dimension, and the control text information is processed by a language model to generate a prompt word; The prompt word is input into the text encoder for conversion processing to obtain vector information for semantic understanding.

4. The method of claim 3, wherein, Identify the graphic outline from the chart base map, including: The chart base map is output to the detailed line preprocessor in the chart generation workflow, and the graphic outline in the base map is extracted, which is the outline with clear boundary distinguishing function in the chart.

5. The method of claim 1, wherein, The text base map and the chart base map are generated by processing the chart standard template and the reference structure data through the preset script, including: Input the chart standard template and the reference structure data into the preset automation script; In the automation script, load the reference structure data through the preset module, and use the preset drawing module to draw the base map to obtain the text base map and the chart base map.

6. The method of claim 5, wherein, In the automation script, load the reference structure data through the preset module, and use the preset drawing module to draw the base map to obtain the text base map and the chart base map, including: Through the automation script, the chart standard template and the reference structure data are parsed and processed to construct a base map framework, which includes key components; Based on the base map framework, use the preset drawing tool to draw the base map, and use the preset drawing function to set parameters at the corresponding positions of the key components based on the specified parameters in the chart standard template to obtain the text base map; Use the drawing tool to update the attribute calibration based on the key components to draw the chart base map.

7. An AIGC-based data chart mapping system, characterized by, Including: A chart design module is used to construct a chart standard template based on a preset chart specification; A chart splitting processing module is used to split the chart standard template according to dimensions and measures to construct reference structure data of the chart, which includes reference structures constructed by dimensions and measures; A base map generation module is used to process the chart standard template and the reference structure data through a preset script to generate a text base map and a chart base map, and the text base map, the chart base map, and the obtained control text information are used as input data; A workflow construction module is used to construct a chart generation workflow based on a diffusion model, and add a text encoder, a control network, and a fine-tuning model to the chart generation workflow, the control network is used to control content generation; A chart generation module is used to encode and control the input data in the chart generation workflow, and process the input text base map and chart base map in the target dimension through the selected target fine-tuning model and control network to generate structure-controlled image content with semantic matching degree, and output the final chart. In the chart generation workflow, the input data is encoded and controlled, and the input text base map and chart base map are processed in the target dimension through the selected target fine-tuning model and control network to generate structure-controlled image content with semantic matching degree, and output the final chart, including: in the chart generation workflow, the target fine-tuning model of the target dimension is selected based on the control text information, and the prompt word is generated, the prompt word is processed by the text encoder to obtain the vector information, and the target dimension includes two dimensions and three dimensions; the text base map and the chart base map are loaded as an image pair into the chart generation workflow, and the graph contour is identified from the chart base map; the vector information is used as the key guiding semantics of the diffusion model input, and the target fine-tuning model and the control network are used to generate structure-controlled image content with semantic matching degree based on the graph contour; the image content and the text base map are mixed through the mixing module to output the final chart; In the chart generation workflow, the input data is encoded and controlled, and the input text base map and chart base map are processed in the target dimension through the selected target fine-tuning model and control network to generate structure-controlled image content with semantic matching degree, and output the final chart, including: in the chart generation workflow, the target fine-tuning model of the target dimension is selected based on the control text information, and the prompt word is generated, the prompt word is processed by the text encoder to obtain the vector information, and the target dimension includes two dimensions and three dimensions; the text base map and the chart base map are loaded as an image pair into the chart generation workflow, and the graph contour is identified from the chart base map; the vector information is used as the key guiding semantics of the diffusion model input, and the target fine-tuning model and the control network are used to generate structure-controlled image content with semantic matching degree based on the graph contour; the image content and the text base map are mixed through the mixing module to output the final chart; 8. An electronic device, comprising: The communication interface, the memory and the communication bus complete the communication among each other through the communication bus. The memory is used to store computer programs. The processor is used to execute the programs stored on the memory to realize the steps of the AIGC-based data chart mapping method of any one of claims 1-6.

Images