Data chart illustration method, system and equipment based on AIGC

Abstract

The invention relates to an AIGC-based data chart illustration method, system and device, and relates to the technical field of AIGC data visualization, and the method comprises the steps: constructing a uniform chart generation workflow based on a diffusion model, and integrating a plurality of modules and models. When a chart illustration is generated, a chart standard template is constructed according to illustration chart specifications, and the chart standard template is split to obtain a reference structure. And generating a base map of characters and charts through the script processing template. And inputting the input control text and the base map into the workflow, processing the base map according to the control text by using each model, and controlling the generation of the base map through a control network to obtain image content with a semantic matching degree. And finally, mixing the image content and the character base map to output a final chart. According to the technical scheme, the automation degree and the consistency of illustration production can be greatly improved, the design cost is reduced, the production efficiency is improved, and the availability and the visual uniformity of final illustration are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of AIGC data visualization, in particular to an AIGC-based data chart mapping method, system and device. BACKGROUND

[0002] In related technologies, chart mapping is widely used in many fields such as news reporting, advertising marketing, PPT making, and paper writing. Chart mapping generally has three methods: ①, using the chart generation function in the graphic design software such as PS and Ai alone by a graphic designer to design data charts. The chart mapping method usually produces static charts; ②, using existing professional data visualization software or online tool library to design, such as tableau, flourish, D3 (full name Data-Driven Documents, a data visualization tool library based on JavaScript) and the like; ③, directly writing code by a front-end engineer to design complex data visualization graphics. Among them, chart mapping methods ② and ③ usually produce interactive charts to make static output processing using interactive charts. Figure One

[0003] However, in the actual application of news data mapping and various data mapping, the above chart mapping methods all need to rely on manual designers to create data table mapping, which has the following obvious defects: first, the manual design efficiency is low, which cannot meet the demand of real-time generation of mapping for a large amount of news or data content; second, the levels of different designers are uneven, which leads to the difficulty in unifying the quality and style of mapping and the problem of inconsistent visual effects; third, due to the lack of systematic design standards and process constraints, the mapping results are often difficult to meet the established visual specifications, especially in large-scale production environment, it is difficult to guarantee the design consistency and professionalism; finally, the design levels of designers are different, and excellent designers are few, and the charts designed by most designers cannot reach a high degree of beauty.

[0004] Therefore, the existing chart mapping methods mainly have the problems of low chart mapping efficiency, poor specification of generated mapping, and poor beauty. SUMMARY

[0005] The present application provides an AIGC-based data chart mapping method, system and device, which mainly based on structured or semi-structured input content (such as news data, chart data, etc.), according to the preset design specification and visual template, realizes the ability of efficient, standardized and batch generation of mapping, controls the generated charts, improves the quality of chart mapping, and effectively solves the problems of low chart mapping efficiency, poor specification of generated mapping, and poor beauty in the prior art.

[0006] ​In a first aspect, the present application provides a method, comprising:

[0007] constructing a chart standard template based on a preset chart specification;

[0008] splitting the chart standard template according to dimensions and metrics, constructing a reference structure data of the chart, the reference structure data comprising a reference structure constructed by the dimensions and the metrics;

[0009] processing the chart standard template and the reference structure data through a preset script, generating a text base map and a chart base map, and taking the text base map, the chart base map, and obtained control text information as input data;

[0010] constructing a chart generation workflow based on a diffusion model, and adding a text encoder, a control network, and a fine-tuning model to the chart generation workflow, the control network being used to control content generation;

[0011] in the chart generation workflow, performing encoding control processing on the input data, and processing the input text base map and chart base map in a target dimension through a selected target fine-tuning model and the control network, to generate image content with structure control and semantic matching degree, and output a final chart.

[0012] Optionally, the chart generation workflow based on the diffusion model is constructed, and the text encoder, the control network, and the fine-tuning model are added to the chart generation workflow, comprising:

[0013] constructing the chart generation workflow, and creating a first loader according to a preset reference template, and loading the diffusion model to the chart generation workflow through the first loader;

[0014] in the chart generation workflow, loading the fine-tuning model through a second loader and controlling the generation proportion of the fine-tuning model;

[0015] in the chart generation workflow, introducing the text encoder, and introducing the control network to control the generated image content.

[0016] Optionally, in the chart generation workflow, the input data is subjected to encoding control processing, and the input text base map and chart base map are processed in a target dimension through a selected target fine-tuning model and the control network, to generate image content with structure control and semantic matching degree, and output a final chart, comprising:

[0017] in the chart generation workflow, selecting a target fine-tuning model of a target dimension based on the control text information, and generating a prompt word, processing the prompt word through the text encoder to obtain vector information, the target dimension comprising two dimensions and three dimensions;

[0018] The text background and the chart background are imported into the chart generation workflow as a pair of loaded images, and a graphic contour is identified from the chart background;

[0019] The vector information is used as key guiding semantics for input of the diffusion model, and a target fine-tuning model and a control network are used to generate image content with structural control and semantic matching degree based on the graphic contour;

[0020] The image content and the text background are mixed through a mixing module to output a final chart.

[0021] Optionally, in the chart generation workflow, a target fine-tuning model of a target dimension is selected based on control text information, and a prompt word is generated, the prompt word is processed by a text encoder to obtain vector information, including:

[0022] The control text information is content-analyzed to determine a diffusion model matching a scene and a target fine-tuning model of a target dimension, and the control text information is processed by a language model to generate a prompt word;

[0023] The prompt word is input into the text encoder for conversion processing to obtain vector information for semantic understanding.

[0024] Optionally, the graphic contour is identified from the chart background, including:

[0025] The chart background is output to a detailed line preprocessor in the chart generation workflow, and a graphic contour in the background is extracted, the graphic contour being a contour in the chart having a clear boundary distinguishing function.

[0026] Optionally, the vector information is used as key guiding semantics for input of the diffusion model, and a target fine-tuning model and a control network are used to generate image content with structural control and semantic matching degree based on the graphic contour, including:

[0027] The vector information is input into the diffusion model to obtain latent space parameters output by the model as semantic directions for guiding image content generation;

[0028] Chart style control information is generated by the target fine-tuning model, and the diffusion model and the target fine-tuning model jointly act on a scale of the generated background to obtain scale information;

[0029] The graphic contour is input into the control network as a constraint condition to generate control constraint information;

[0030] In the sampler of the diffusion model, the image content is generated based on the latent space parameters, the chart style control information, the scale information, and the control constraint information.

[0031] Optionally, the text background and the chart background are generated by processing a chart standard template and reference structure data through a preset script, including:

[0032] inputting the chart standard template and the benchmark structure data into a preset automation script;

[0033] In the automation script, the benchmark structure data is loaded through a preset module, and a base map is drawn using a preset drawing module to obtain a text base map and a chart base map.

[0034] Optionally, in the automation script, the benchmark structure data is loaded through a preset module, and a base map is drawn using a preset drawing module to obtain a text base map and a chart base map, comprising:

[0035] Through the automation script, the chart standard template and the benchmark structure data are analyzed and processed to construct a base map framework, and the base map framework includes key components;

[0036] Based on the base map framework, a preset drawing tool is used to draw a base map, and using the specified parameters in the chart standard template as a benchmark, a preset drawing function is used to set parameters at the corresponding positions of the key components to obtain a text base map;

[0037] Using the drawing tool, attribute calibration is updated based on the key components to draw a chart base map.

[0038] In a second aspect, the present application provides an AIGC-based data chart mapping system, comprising:

[0039] A chart design module is configured to construct a chart standard template based on a preset mapping chart specification;

[0040] A chart splitting processing module is configured to split the chart standard template according to dimensions and measures to construct benchmark structure data of the chart, and the benchmark structure data includes a benchmark structure constructed by the dimensions and the measures;

[0041] A base map generation module is configured to process the chart standard template and the benchmark structure data through a preset script to generate a text base map and a chart base map, and to use the text base map, the chart base map, and obtained control text information as input data;

[0042] A workflow construction module is configured to construct a chart generation workflow based on a diffusion model, and to add a text encoder, a control network, and a fine-tuning model to the chart generation workflow, and the control network is configured to control content generation;

[0043] A chart generation module is configured to perform encoding control processing on the input data in the chart generation workflow, and to process the input text base map and the chart base map in a target dimension through a selected target fine-tuning model and the control network to generate image content with controlled structure and semantic matching degree, and to output a final chart.

[0044] In a third aspect, the present application provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete the communication among each other through the communication bus.

[0045] The memory is used to store a computer program.

[0046] The processor is used to execute the program stored on the memory, so as to realize the steps of the AIGC-based data chart mapping method according to any one of the embodiments of the first aspect.

[0047] In summary, the embodiments of the present application construct a unified diffusion model-based chart generation workflow for chart generation, integrating various modules and models. During chart generation, first, a chart standard template is constructed according to the mapping chart specification, and the chart standard template is split according to the dimensions and metrics to obtain a reference structure of the chart. Then, the design template and the reference structure are processed through a script to generate a text and chart base map. Finally, the user's control text is combined with the base map and input into the workflow, and the models integrated in the workflow process the two base maps according to the input text, and control the generation of the base map through a control network to obtain image content with semantic matching degree. Finally, the image content and the text base map are mixed to output the final chart. As can be seen, the technical solution of the present application can greatly improve the automation degree and consistency of mapping production, reduce design cost, improve production efficiency, and significantly improve the usability and visual uniformity of the final mapping, has good application and promotion prospects, and solves the problems of low chart mapping efficiency, poor specification of generated mapping and poor aesthetic appearance in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0048] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the present application.

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0050] Figure 1 A flowchart of a data chart mapping method based on AIGC provided by an embodiment of the present application is shown in the figure.

[0051] Figure 2 A flowchart of the steps of a data chart mapping method based on AIGC provided by an optional embodiment of the present application is shown in the figure.

[0052] Figure 3A column chart generation template example graph provided by an example of the present application;

[0053] Figure 4 A text background example graph provided by an example of the present application;

[0054] Figure 5 A chart background example graph provided by an example of the present application;

[0055] Figure 6 A chart generation flowchart provided by an example of the present application;

[0056] Figure 7 A 2D chart generation result example graph provided by an example of the present application;

[0057] Figure 8 A 3D chart generation result example graph provided by an example of the present application;

[0058] Figure 9 A structure block diagram of an AIGC-based data chart matching system provided by an example of the present application;

[0059] Figure 10 A structure diagram of an electronic device provided by an example of the present application. DETAILED DESCRIPTION

[0060] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0061] To facilitate the understanding of the embodiments of the present application, further explanation and description will be made in connection with the drawings and specific embodiments. The embodiments do not constitute a limitation on the embodiments of the present application.

[0062] Figure 1 A flowchart of an AIGC-based data chart matching method provided by an example of the present application. The method can specifically include the following steps:

[0063] Step 110, constructing a chart standard template based on a preset matching chart specification.

[0064] In the embodiment, the chart specification refers to a series of specifications designed in advance for different charts, which are mainly based on the attributes, composition, etc. of different chart specifications, and involve specifications according to matching chart types from the data dimension attribute, such as a column chart, which usually includes title, chart body, horizontal and vertical axes, labels, and data source description, etc.

[0065] In the embodiment, the chart standard template is designed in advance according to the chart specification, which mainly involves two parts, namely the composition structure of the chart and the parameters corresponding to the composition structure, such as the title, which usually includes font size, font format, position of the title in the chart, etc.

[0066] Therefore, the embodiment pre-generates the corresponding template to reduce the reliance on the designer's understanding and judgment of the data, so that the user no longer needs to understand the data and draw it, and can directly use the template or make simple modifications on the template to obtain the corresponding chart standard template.

[0067] It should be noted that the data charting method based on AIGC provided in the present application can be applied to the generation of charting of different charts, such as column chart, line chart, pie chart, heat map, etc. The embodiment does not limit the specific chart.

[0068] In step 120, the chart standard template is split according to the dimension and the measure to construct the baseline structure data of the chart.

[0069] The baseline structure data includes the baseline structure constructed by the dimension and the measure.

[0070] In the embodiment, the chart standard template is split according to the dimension and the measure, so that all the charting data can be processed according to the standard of the chart standard template. After processing, the baseline structure of the chart is successfully constructed. For example, taking a column chart as an example, the baseline structure includes horizontal coordinates and vertical coordinates, so the dimension can be used as the row and the measure can be used as the column. After processing, the dimension in the baseline structure can be used as the horizontal coordinate of the chart and the measure can be used as the vertical coordinate of the chart. Thus, the standardization processing of the chart structure is completed.

[0071] In step 130, the chart standard template and the baseline structure data are processed through a preset script to generate a text base map and a chart base map, and the text base map, the chart base map, and the obtained control text information are used as input data.

[0072] In the embodiment, the control text information is mainly used as one of the constraint conditions to enable the AI to generate accurate charting. The control text information can be a text description input by the user, or in the news industry, it can be news content, etc. The embodiment does not limit this.

[0073] The embodiment realizes an AIGC-based chart mapping generation method. In AIGC graph generation, the randomness of AI is very strong, while data charts require accurate proportions. Therefore, the embodiment uses a preset script (such as an automated script written in Python) to process chart standard templates and benchmark structure data to make a base map.

[0074] In making the base map, a reliable data chart base map is made using data visualization modules such as echarts and matplotlib. Specifically, first, the benchmark structure number is converted using a related function. For a column chart, for example, a function can be used to convert row and column data. Then, a drawing tool (such as a matplotlib template) is used to draw corresponding text base maps and chart base maps. Finally, the two base maps and control text information are used as reference inputs in the subsequent AI image generation process.

[0075] Thus, the embodiment makes a reliable data chart base map in advance, which can effectively constrain the output range of AI when using a control network, thereby accurately controlling the generation result of AI.

[0076] Step 140, constructing a chart generation workflow based on a diffusion model, and adding a text encoder, a control network, and a fine-tuning model to the chart generation workflow.

[0077] The control network is used to control content generation.

[0078] The diffusion model used in the embodiment can be Stable Diffusion, which belongs to a variant called Latent Diffusion Model (LDM). Its workflow involves three parts: training phase, diffusion process, and latent space operation. It is mainly used for text-to-image or image generation, and can automatically generate high-quality images according to user input text descriptions. The embodiment aims to solve the problem of relying on manual mapping in the news industry, and develops a standard AI automated mapping process to efficiently and quickly generate a large number of controllable charts using AI.

[0079] In a specific implementation, the embodiment is based on Stable Diffusion to construct a chart generation workflow (hereinafter referred to as: comfyUI workflow) for automatic chart generation. First, load the model of Stable Diffusion according to the standard template of Stable Diffusion, and set it in the comfyUI workflow. Then, set multiple modules, loaders and models in the comfyUI workflow, including: text encoder (such as CLIP text encoder), fine-tuning model (such as LoRA fine-tuning model), multiple image loading modules (for loading base maps), control network (such as ControlNet).

[0080] In actual implementation, considering that charts include two-dimensional (2D) charts and three-dimensional (3D) charts, and the differences in chart use scenarios (such as applications in different scenarios such as news, advertising marketing, PPT making, and paper writing), the comfyUI workflow can set different LoRAs, and can select LoRAs for processing images according to use scenarios or dimensions (2D or 3D charts) in actual application.

[0081] Step 150, in the chart generation workflow, the input data is encoded and controlled, and the input text base map and chart base map are processed in the target dimension through the selected target fine-tuning model and control network, to generate image content with controlled structure and semantic matching degree, and output the final chart.

[0082] The embodiment can realize 2D or 3D chart (usually 3D solid chart) generation. By analyzing the input control text, the required generated chart and matching chart can be identified as 2D or 3D, and the target dimension is determined. Based on the target dimension, the LoRA for chart generation is selected to adapt to different application scenarios and chart making requirements.

[0083] Specifically, in the comfyUI workflow, the fine-tuning model is selected according to the input control text, and the control network is selected. Then, the control text is parsed by the CLIP text encoder and input into Stable Diffusion. The LoRA controls the generated chart scale according to the two input base maps. According to the parameters output by each part, the text base map is generated, and the effect of the generated base map is controlled by ControlNet to obtain image content with controlled structure and semantic matching degree. Finally, the generated text base map and image content are mixed, and the output image is output as the final chart.

[0084] It can be seen that the embodiment of the application first designs a general chart template. Then, a text base map and a chart base map are generated through the template. Finally, the AI can generate content with a fixed structure through the control of the chart base map, and the text base map is superimposed by using an image superposition algorithm, so that a chart meeting the requirements can be generated. The embodiment greatly improves the automation degree and consistency of the production of matching charts, reduces the design cost, improves the production efficiency, and significantly improves the usability, visual uniformity and aesthetics of the final matching chart, and has good application and promotion prospects.

[0085] Referring to Figure 2 , a step flowchart of a data chart matching method based on AIGC is shown, which can specifically include the following steps:

[0086] In step 210, a chart standard template is constructed based on a preset chart matching specification.

[0087] In the related art, the existing technology depends on the understanding and judgment of the designer on the data, and the designer needs to actively set the chart type matching the data, so that the data expression is not accurate. Because some data have relatively fixed and common chart types to match, not all chart types can be matched. If it does not match, the data expression will be wrong or cause misunderstanding.

[0088] To this end, the embodiment uses standardized processing to automatically understand and match the chart type. Specifically, the embodiment pre-produces a template according to the matching chart type according to the data dimension attribute, and no longer needs to understand the data and draw, but modifies on the template, so that it does not need to depend on the understanding and judgment of the designer on the data.

[0089] Referring to Figure 3 , a column chart is taken as an example:

[0090] The matching chart design specification of the column chart in the embodiment includes: overall drawing board size: 750 × 1200 px, standard vertical column chart template, adaptive to mobile terminal graphic content and automatic chart platform output. Figure 3 In the column chart, the composition of the chart is arranged from top to bottom as follows: title area, chart main body area, horizontal axis label area, annotation explanation area (including but not limited to data source explanation area).

[0091] Among them, the corresponding attribute parameters include:

[0092] The title area font size is 44 px, the font is bold (Bold), the alignment is centered, it is close to the top of the drawing board, the line height is equal to the font height, and it is convenient for unified layout.

[0093] Bar chart area size width: 505 px (referring to the range from the leftmost column to the rightmost column), height: 600 px (used to accommodate the height of the largest column), columns are arranged in a horizontal average distribution within the 505 px column area, with equal spacing between columns, column tops aligned, and used to highlight the relative height differences of the data.

[0094] Spacing explanation, left margin: 155 px from the left edge of the drawing board to the start of the column area, right margin: 90 px from the right edge of the drawing board to the end of the column area, and the top of the column area is 75 px from the bottom of the title area. Auxiliary alignment lines (reference lines) are distributed within the column area for reference to the height scale of the columns. The lines are arranged at equal intervals for visual comparison of the annotation text position. If there are illustrations or arrow texts, they are uniformly arranged on the left and aligned with the left boundary of the column area, with a spacing of 34 px to the left of the start of the column area.

[0095] Horizontal axis label area text style Semibold font, size 20 pt, each label is centered aligned with the bottom of the corresponding column, label text spacing from the bottom of the column is 10 px, line height 80 px, suitable for Chinese and English label line breaks or annotation characters, etc.

[0096] The bottom data source explanation area has a top margin of 60 px from the bottom of the chart area, and the top and bottom margins within the block are each 75 px. The data source title size is 20 px, bold, and the data source content size is 18 px, regular, with the same text line height and size.

[0097] Using the above chart specification, a corresponding chart standard template can be generated. The parameters involved in the chart standard template can be consistent with the chart specification, or they can be adjusted according to actual needs.

[0098] It should be noted that the above chart template design is only one type of column chart, i.e., the chart template design is only one of the chart templates. Those skilled in the art can freely design chart templates according to different charts, including chart forms, chart specifications, and chart attribute parameters, without limitation. For example, different chart templates can be designed for different publishing channels, such as Weibo, WeChat, mobile devices, and PCs, to achieve better results. The font size, line spacing, etc. may change.

[0099] Step 220, split the chart standard template according to the dimensions and measures to construct the baseline structure data of the chart.

[0100] The baseline structure data includes the baseline structure constructed by the dimensions and measures.

[0101] In step 230, the text base map and the chart base map are generated by processing the chart standard template and the reference structure data according to a preset script, and the text base map, the chart base map and the obtained control text information are taken as input data.

[0102] Optionally, the generation of the text base map and the chart base map by processing the chart standard template and the reference structure data according to the preset script can include the following sub-steps:

[0103] In sub-step 2301, the chart standard template and the reference structure data are input into a preset automated script.

[0104] In sub-step 2302, the reference structure data is loaded in the automated script by a preset module, and the base map is drawn by using a preset drawing module to obtain the text base map and the chart base map.

[0105] The sub-step 2301 to the sub-step 2302 are uniformly described as follows:

[0106] In a specific implementation, since the template of the chart has been generated according to the standardized processing, and the reference structure of the chart is limited by the dimension and the measure, based on the preset chart mapping specification or the chart standard template, the corresponding automated script can be written, such as an automated script written by python. The base map is made by using the automated script. The automated script can integrate a plurality of related conversion functions, and can integrate or associate a drawing tool. By processing the chart standard template and the reference structure by the script, the text base map and the chart base map can be automatically drawn and generated by using the drawing tool, which are taken as one of the reference inputs of the AI image generation.

[0107] In an optional embodiment, in the automated script, the reference structure data is loaded by a preset module, and the base map is drawn by using a preset drawing module to obtain the text base map and the chart base map. Specifically, the chart standard template and the reference structure data are parsed and processed by the automated script to construct a base map framework, and the base map framework includes key components. Based on the base map framework, the base map is drawn by using a preset drawing tool, and based on the specified parameters in the chart standard template, the preset drawing function is used to set parameters at the positions corresponding to the key components to obtain the text base map. The drawing tool is used to update the attribute calibration based on the key components to draw the chart base map.

[0108] Specifically, the bottom frame, as one of the important components of the text bottom chart / table bottom chart, mainly includes a text bottom frame and a table bottom frame. The key components of the text bottom frame mainly include a title, a chart coordinate axis, a coordinate axis label, data sources, and source information, which correspond to the components in the chart standard template. The key components of the table bottom frame include a chart shape, rows, and columns. The parameters related to the bottom frame can correspond to the parameters related to the chart standard template.

[0109] As an example, the drawing function used in the embodiment can be the read_excel function of pandas, and the drawing tool can use matplotlib to draw. Taking a column chart as an example, the template data (i.e., the chart standard template and the reference structure) processed by the data template is loaded by using the pandas module, the read_excel function of pandas can directly convert the row and column data, and then the corresponding text bottom chart and table bottom chart are drawn by using the matplotlib module. The text bottom chart can be seen in FIG. 6, and the table bottom chart can be seen in FIG. 7. Figure 4 Figure 5

[0110] The following exemplary describes the parameters when drawing the text bottom chart and the table bottom chart:

[0111] The text bottom chart is composed of a title, a chart coordinate axis, a coordinate axis label, data sources, and source information. First, the chart is drawn by using matplotlib, the chart size figure is specified according to the template size, and then the coordinate axis position is determined by using the figure.add_axes function. The position needs to be set according to the parameters given by the template. The coordinate axis and the coordinate axis label are selected to be black, and the chart shape color is selected to be white. In this way, the white chart shape will disappear after the mask is generated because it is the same as the background color. The text information such as the title, the data sources, and the source information is drawn in the corresponding blank position by using the text function in the opencv module. The position is also obtained from the template. The data of the chart is loaded by using axes.bar from the pandas row and column data. The row is the horizontal coordinate and the label, and the column is the graph height or the proportion. Finally, the picture is output by using savefig. The table bottom chart is also drawn by using matplotlib. The graph color is selected to be black, and the show attribute of the chart components such as the coordinate axis, the label, and the title is set to false. Finally, the table bottom chart can be obtained by using savefig.

[0112] In step 240, a chart generation workflow based on a diffusion model is constructed, and a text encoder, a control network, and a fine-tuning model are added to the chart generation workflow.

[0113] The control network is used to control content generation. ​​

[0114] In a specific implementation, the embodiment of the present application constructs a graph generation workflow based on a diffusion model, and adds a text encoder, a control network, and a fine-tuning model to the graph generation workflow, which can specifically include: constructing a graph generation workflow, and creating a first loader according to a preset benchmark template, and loading the diffusion model into the graph generation workflow through the first loader; in the graph generation workflow, loading a fine-tuning model through a second loader and controlling the generation proportion of the fine-tuning model; introducing a text encoder in the graph generation workflow, and introducing a control network to control the generated image content.

[0115] In the embodiment, the model that can be set by the comfyUI workflow can include a CLIP text encoder, a loRA, a Stable Diffusion base model, etc., and is provided with a K sampler, a VAE decoder, a ControlNet, a ControlNet loader, and a Canny detailed line preprocessor.

[0116] Specifically, a checkpoint loader is created according to the standard template of Stable Diffusion to load the Stable Diffusion base model, the forward prompt word and the reverse prompt word CLIP text encoder, and the LoRA model. The LoRA stack loader of the LoRA model is used to load the loRA while controlling the generation proportion of the loRA. The empty latent space is set to the same size as the bottom map, the K sampler is used to receive various inputs in front of the sampling calculation parameters according to the settings, and the VAE decoder outputs the pixel space.

[0117] On the basis of the above, an image loading module is added to load a graphic bottom map, and the ControlNet loader selects control_v11p_sd15_canny or other Canny models. Among them, the Canny detailed line preprocessor is used to control the Canny effect, and the ControlNet application is added and connected with the parameter reference. In addition, a preview image is added to preview the processing effect of the Canny model.

[0118] Among them, in the embodiment, the graph generation of 2D effect and 3D effect can be realized by selecting different loRA models. Unlike general 3D graph making, as long as the input model is replaced by the result generated by the loRA model trained by the 3D picture, it can be 3D.

[0119] Step 250, in the graph generation workflow, a target fine-tuning model of a target dimension is selected based on control text information, and a prompt word is generated, the prompt word is processed by a text encoder to obtain vector information.

[0120] Among them, the target dimension contains two dimensions and three dimensions.

[0121] In this embodiment, the chart base map (as a graphic mask), the text base map, and the control text are input into the Stable Diffusion-based comfyUI workflow. Through multi-step collaborative processing, a chart that meets the specifications and has both aesthetic and accurate appearance is finally generated. The comfyUI workflow can set multiple control options, such as enabling "perfect chart item", "hard edge", and other image control options (the remaining parameters can be flexibly configured according to needs). Then run and generate the structure-controlled image content, Figure 6 The corresponding is the running flowchart.

[0122] Specifically, for the input control text, taking a news text as an example, the text is the key to guide the AI to generate a style and context that match. The comfyUI workflow selects the base and loRA model that meets the scene based on the input control text, such as the text involves "technology theme", then select the LoRA model trained with technology style to ensure that the chart style matches the theme; for example, the text involves "3D", then select the LoRA model that can realize 3D effect. And based on the control text, a large language model (such as LLM, etc.) can be used to parse the control text to generate a prompt (prompt word), that is, a prompt word that meets the requirements of AIGC model or Stable Diffusion is generated. The prompt word is converted into an AI understandable vector by a CLIP text encoder, which is one of the inputs of the Stable Diffusion base model, guiding the AI to generate artistic graphics consistent with the text context within the structure range of ControlNet constraint (such as cold color tone, geometric texture for technology theme chart), that is, the semantic direction for guiding image generation.

[0123] Therefore, the embodiment controls the semantics of the image through the control text, thereby guiding the AI to generate picture content with higher semantic matching degree, and further enhancing the consistency and expressiveness of the image and the news context.

[0124] For example, the prompt word can be composed of:

[0125] Chart theme related description (such as "technology industry user growth data");

[0126] Artistic style requirements (such as "minimalism, high saturation color matching");

[0127] Detail constraints (such as "3D solid column chart, surface with metal texture").

[0128] Optionally, in the chart generation workflow, the target fine-tuning model based on the control text information selects the target dimension, and generates a prompt word, the prompt word is processed by a text encoder to obtain vector information, which can include: performing content analysis on the control text information, determining a diffusion model matching the scene and a target fine-tuning model of the target dimension, and processing the control text information through a language model to generate a prompt word; the prompt word is input into the text encoder for conversion processing to obtain vector information for semantic understanding.

[0129] In step 260, the text base map and the chart base map are imported into the chart generation workflow as a pair of loaded images, and the graph contour is identified from the chart base map.

[0130] In a specific implementation, considering that AIGC graph generation is very strong due to the randomness of AI, while data charts require accurate proportions. Therefore, the embodiment pre-generates a general chart base map, and then uses the ControlNet module to constrain the output range of AI to accurately control the generation result of AI.

[0131] Specifically, referring to FIG. 8, Figure 6 As shown in the figure, the embodiment introduces a loaded image module for loading a text base map and a chart base map respectively. The loaded base maps are imported into the comfyUI workflow in the form of an image pair. After loading the chart base map, the ControlNet loader and the Canny detailed line preprocessor can be used to extract the graph contour in the base map, such as Figure 5 the black graph contour of the chart base map in FIG. 8. Using this graph contour, the AI-generated content can be limited to the contour range in the subsequent process, avoiding the randomness of AIGC to destroy the structure of data graphs, such as avoiding the randomness of AI to destroy the column height proportion in a column chart.

[0132] Optionally, the above-mentioned identification of the graph contour from the chart base map can include: outputting the chart base map to the detailed line preprocessor in the chart generation workflow to extract the graph contour in the base map. The graph contour is a contour with a clear boundary distinguishing function in the chart.

[0133] For example, in a column chart, the column height proportion, color, and in a pie chart, the pie shape size, etc. can all be contours with a clear boundary distinguishing function. The present embodiment does not limit this.

[0134] In step 270, the vector information is used as the key guiding semantics of the diffusion model, and the target fine-tuning model and the control network are used to generate image content with a controlled structure and a semantic matching degree based on the graph contour.

[0135] In the processing of the AI chart base map, the processing of the AI is controlled through the control network constraint to ensure the controllability of the structure of the data graph. Specifically, the chart base map is input as a graph mask to the ComfyUI workflow. After loading the chart base map, the ControlNet loader is used to ensure that the graph generated by the AI (such as the column structure in the artistic column chart) strictly follows the structure constraint of the base map, guarantees the accuracy of data expression, and generates image content with controlled structure and semantic matching degree.

[0136] In this embodiment, Stable Diffusion determines the guiding semantics based on the input vector information and outputs corresponding parameters; LoRA model controls the generation ratio of the base map based on the input base map; ControlNet is based on the input graph outline to generate constraint information. Finally, combined with the parameters, the image content is generated.

[0137] Optionally, the key guiding semantics of using vector information as input of the diffusion model to generate image content with controlled structure and semantic matching degree based on the graph outline by using the target fine-tuning model and the control network can include: inputting the vector information into the diffusion model to obtain the latent space parameters output by the model as the semantic direction of guiding image content generation; generating chart style control information through the target fine-tuning model, and through the diffusion model and the target fine-tuning model, the ratio of the generated base map is obtained; the graph outline is input into the control network as a constraint condition to generate control constraint information; in the sampler of the diffusion model, based on the latent space parameters, the chart style control information, the ratio information and the control constraint information, the image content is generated.

[0138] As an example, in the comfyUI workflow, the cooperative processing of each part can be understood as:

[0139] The Stable Diffusion base model is initialized by the checkpoint loader and receives the CLIP encoded Prompt vector;

[0140] The LoRA heap loader loads the selected LoRA model to control the generation ratio (affecting the strength of the chart style);

[0141] The empty latent space is set to the same width and height size as the base map to ensure that the size of the generated image matches;

[0142] The K sampler receives all the above inputs (models, prompts, ControlNet constraints, latent space parameters, etc.) to perform sampling calculation and generate latent space features that meet the constraints.

[0143] Step 280, mix the image content and the text base map through the mixing module to output the final chart.

[0144] In this embodiment, when generating a mixed image, first, the latent space features output by the K sampler are converted into an image in the pixel space (i.e., the constrained artistic graphic content) by the VAE decoder in the comfyUI workflow. Then, the image mixing module superimposes the graphic content output by the VAE on the previously loaded text base map, and finally generates a chart containing complete text information, accurate structure, and matching style. For example, the generated final image can refer to Figure 7 As shown in the bar chart, the bar chart shows the changes in the area of the park lotus in the region, so the generated map is a lotus map, which enriches the generated data chart. As can be seen, thanks to the randomness of AI, after using the base map to control the requirements of the base map, the richness of the output data chart far exceeds the creative ability of ordinary designers within the same time.

[0145] Further, as shown in Figure 8 Using loRA trained with 3D effects, the corresponding 3D effect chart can be generated under the same scheme. In the example shown in Figure 8 As shown in the example, the bar chart shows the changes in the export volume of hardware devices in a certain country / region. To fit the theme, the generated 3D bar chart constructs a 3D column in the style of hardware devices, enriching the chart. The final 3D effect chart has a beautiful appearance.

[0146] In summary, the embodiments of the present application construct a unified chart generation workflow based on a diffusion model, integrating multiple modules and models. When generating a chart, a chart standard template is constructed according to the chart specification, and the chart standard template is split to obtain a reference structure. The template is processed by a script to generate a text and chart base map. The input control text is combined with the base map and input into the workflow. The models process the base map according to the control text, and the generation of the base map is controlled by the control network to obtain image content with semantic matching degree. Finally, the image content and the text base map are mixed to output the final chart. As can be seen, the scheme of the embodiments of the present application can greatly improve the automation and consistency of chart production, reduce design costs, improve production efficiency, and significantly improve the usability and visual uniformity of the final chart.

[0147] The AIGC-based data chart matching method provided by the embodiments of the present application has the following beneficial effects:

[0148] ①, automatically understand and match the chart type. It can accurately match the chart type to avoid data expression errors or misunderstandings.

[0149] ② This application resolves the issue of uncontrollable content in AIGC-generated data areas, controlling the content of AIGC-generated images within meaningful charts and graphics. It fully analyzes and resolves the contradiction between freely created AIGC and controllable information transmission, achieving optimal results in both information accuracy and aesthetic design during data visualization and transmission.

[0150] ③ Enrich the design elements of the chart area and enhance the artistic expression. Specifically, existing chart design tools offer relatively simple chart area designs, with chart blocks typically filled in a single color and rarely featuring more complex artistic designs. However, the technical solution provided in this application conducts an in-depth analysis of different chart types, distinguishing large-block graphics such as bar charts and pie charts, and deeply integrating the design of these large blocks with the advantages of AIGC image generation. This makes the data content in the chart area more artistic, and the technology also supports the rapid generation of artistically designed 3D charts, rather than ordinary 2D front-view charts. Relying on AI technology, the artistic design expression of the chart area is greatly enriched and can be combined with the theme of the data presentation, improving the artistic expressiveness of the data charts.

[0151] ④ The design of the chart area, text outline, and background can be completed in one integrated process, greatly improving the integration of elements. Specifically, in existing technologies, designers need to design the chart, font, and background image separately, spending a lot of time adjusting different elements to make them more aesthetically pleasing and integrated. In contrast, the technical solution provided in this application allows for the integrated generation of different elements of the entire chart, automatically matching colors, elements, and outline graphics, etc., quickly improving the overall integration of the chart.

[0152] ⑤ Lowering the design threshold for charts and significantly improving production efficiency and speed. Specifically, in existing chart design that relies on designers, a skilled chart designer takes approximately 1-3 hours to create a simple 2D chart. This is especially true for charts designed using the second or third path mentioned above, which typically require importing images into graphic design software and spending considerable time on artistic design and processing. All artistic design cannot be completed within a single software program. Furthermore, creating a beautiful 3D chart requires a skilled chart designer to first build a detailed 3D model of the chart, then adjust the angles to output a 2D result. This process not only takes at least 1-5 days but is also technically challenging, making it difficult for designers without modeling skills to complete. The technical solution provided in this application can significantly shorten the production time of high-quality 2D and 3D charts. According to usage verification, this technical solution takes only 10 minutes to create a 2D or 3D chart, and most chart design tasks can be completed within the corresponding tools, improving the overall quality of the charts.

[0153] It should be noted that, for the method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the application embodiments are not limited to the order of the described actions, because according to the application embodiments, certain steps can be performed in other order or at the same time.

[0154] As shown in Figure 9 The application embodiments further provide an AIGC-based data chart mapping system 900, which comprises:

[0155] A chart design module 910 is configured to construct a chart standard template based on a preset mapping chart specification.

[0156] A chart splitting processing module 920 is configured to split the chart standard template according to dimensions and metrics, and construct benchmark structure data of the chart, wherein the benchmark structure data comprises a benchmark structure constructed by the dimensions and the metrics.

[0157] A base map generation module 930 is configured to generate a text base map and a chart base map by processing the chart standard template and the benchmark structure data through a preset script, and take the text base map, the chart base map and obtained control text information as input data.

[0158] A workflow construction module 940 is configured to construct a chart generation workflow based on a diffusion model, and add a text encoder, a control network and a fine-tuning model to the chart generation workflow, wherein the control network is configured to control content generation.

[0159] A chart generation module 950 is configured to perform encoding control processing on the input data in the chart generation workflow, and process the input text base map and the chart base map in a target dimension through a selected target fine-tuning model and the control network, to generate image content with structural control and semantic matching degree, and output a final chart.

[0160] It should be noted that the AIGC-based data chart mapping system provided by the application embodiments can perform the AIGC-based data chart mapping method provided by any of the application embodiments, and has the corresponding functions and advantages of performing the method.

[0161] In a specific implementation, the above-mentioned AIGC-based data chart mapping system can be integrated in a device, so that the device can generate text and chart base maps, combine control texts, generate structure-controlled image content through AIGC technology, and output a final image, as an electronic device, to realize efficient and rapid generation of a large number of controllable charts by using AI. The electronic device can be composed of two or more physical entities, or can be composed of one physical entity, such as a personal computer (PC), a computer, a server, etc., and the application embodiments do not make specific limitations thereto.

[0162] As Figure 10 shown, an embodiment of the present application provides an electronic device, comprising a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112 and the memory 113 complete mutual communication through the communication bus 114; the memory 113 is used to store a computer program; the processor 111 is used to execute the program stored on the memory 113 to implement the steps of the AIGC-based data chart mapping method provided by any one of the preceding method embodiments. Illustratively, the method can include the following steps: constructing a chart standard template based on a preset mapping chart specification; splitting the chart standard template according to dimensions and metrics to construct a baseline structure data of the chart, the baseline structure data containing a baseline structure constructed by the dimensions and the metrics; processing the chart standard template and the baseline structure data through a preset script to generate a text base map and a chart base map, and taking the text base map, the chart base map and the obtained control text information as input data; constructing a chart generation workflow based on a diffusion model, and adding a text encoder, a control network and a fine-tuning model to the chart generation workflow, the control network being used to control content generation; in the chart generation workflow, performing encoding control processing on the input data, and through a selected target fine-tuning model and the control network, processing the input text base map and chart base map in a target dimension to generate image content with a controlled structure and a semantic matching degree, and outputting a final chart.

[0163] An embodiment of the present application also provides a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the AIGC-based data chart mapping method provided by any one of the preceding method embodiments.

[0164] It should be noted that, in this document, relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the phrase "comprising a... " does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0165] The foregoing detailed description of the application has been presented for purposes of illustration and description. Various modifications and changes can be made to these embodiments without departing from the spirit and scope of the application. It is intended that the scope of the application should not be limited by the particular representative embodiments described above.

Claims

1. An AIGC-based data chart mapping method, characterized in that, The method comprises the following steps: Based on the preset chart specification, a chart standard template is constructed; According to the dimensions and the metrics, the chart standard template is split and processed to construct a baseline structure data of the chart, and the baseline structure data comprises a baseline structure constructed by the dimensions and the metrics; The chart standard template and the baseline structure data are processed through a preset script to generate a text base map and a chart base map, and the text base map, the chart base map and the obtained control text information are taken as input data; A chart generation workflow based on a diffusion model is constructed, and a text encoder, a control network and a fine-tuning model are added to the chart generation workflow, and the control network is used to control content generation; In the chart generation workflow, the input data is encoded and controlled, and the text base map and the chart base map are processed in the target dimensions through the selected target fine-tuning model and the control network to generate image content with structural control and semantic matching degree, and finally the chart is output.

2. The method according to claim 1, characterized in that, The chart generation workflow based on the diffusion model is constructed, and the text encoder, the control network and the fine-tuning model are added to the chart generation workflow, comprising: The chart generation workflow is constructed, and a first loader is created according to the preset baseline template, and the diffusion model is loaded into the chart generation workflow through the first loader; In the chart generation workflow, the fine-tuning model is loaded through the second loader and the generation proportion of the fine-tuning model is controlled; In the chart generation workflow, the text encoder is introduced, and the control network is introduced to control the generated image content.

3. The method according to claim 1, characterized in that, In the chart generation workflow, the input data is encoded and controlled, and the text base map and the chart base map are processed in the target dimensions through the selected target fine-tuning model and the control network to generate image content with structural control and semantic matching degree, and finally the chart is output, comprising: In the chart generation workflow, the target fine-tuning model of the target dimension is selected based on the control text information, and a prompt word is generated, the prompt word is processed by the text encoder to obtain vector information, and the target dimension includes two dimensions and three dimensions; The text base map and the chart base map are taken as a pair of loaded images, and are imported into the chart generation workflow, and the graph contour is identified from the chart base map; The vector information is taken as the key guiding semantics of the diffusion model input, and the target fine-tuning model and the control network are used to generate image content with structural control and semantic matching degree based on the graph contour; The image content and the text base map are mixed through a mixing module to output the final chart.

4. The method according to claim 3, characterized in that, In the chart generation workflow, the target fine-tuning model of the target dimension is selected based on the control text information, and a prompt word is generated, the prompt word is processed by the text encoder to obtain vector information, comprising: The control text information is content analyzed to determine the diffusion model matching the scene and the target fine-tuning model of the target dimension, and the control text information is processed by a language model to generate a prompt word; The prompt word is input into the text encoder for conversion processing to obtain vector information for semantic understanding.

5. The method according to claim 4, characterized in that, The graph contour is identified from the chart base map, comprising: The chart base map is output to a detailed line preprocessor in the chart generation workflow, and the graph contour in the base map is extracted, and the graph contour is a contour with clear boundary distinguishing function in the chart.

6. The method according to claim 5, characterized in that, The key guiding semantics of the vector information as the input of the diffusion model is used to generate the structure-controlled image content with semantic matching degree based on the graph contour, including: The vector information is input into the diffusion model to obtain the latent space parameters of the model output as the semantic direction of the guided image content generation; The graph style control information is generated through the target fine-tuning model, and the scale information is obtained by jointly acting on the generated base map through the diffusion model and the target fine-tuning model; The graph contour is input into the control network as a constraint condition to generate control constraint information; In the sampler of the diffusion model, the image content is generated based on the latent space parameters, the graph style control information, the scale information and the control constraint information.

7. The method of claim 1, characterized in that, The text base map and the graph base map are generated by processing the graph standard template and the benchmark structure data through the preset script, including: The graph standard template and the benchmark structure data are input into the preset automatic script; In the automatic script, the benchmark structure data is loaded through the preset module, and the text base map and the graph base map are obtained by using the preset drawing module for base map drawing.

8. The method of claim 7, wherein, In the automatic script, the benchmark structure data is loaded through the preset module, and the text base map and the graph base map are obtained by using the preset drawing module for base map drawing, including: The graph standard template and the benchmark structure data are parsed and processed through the automatic script to construct a base map framework, and the base map framework includes key components; Based on the base map framework, the text base map is obtained by using the preset drawing tool for base map drawing and using the specified parameters in the graph standard template as the benchmark to perform parameter setting at the corresponding positions of the key components by using the preset drawing function; The drawing tool is used to update the attribute calibration based on the key components to draw the graph base map.

9. An AIGC-based data chart mapping system, characterized by, Including: The graph design module is configured to construct a graph standard template based on a preset graph specification; The graph splitting processing module is configured to split the graph standard template according to dimensions and metrics to construct benchmark structure data of the graph, and the benchmark structure data includes a benchmark structure constructed by the dimensions and the metrics; The base map generation module is configured to generate a text base map and a graph base map by processing the graph standard template and the benchmark structure data through a preset script, and input the text base map, the graph base map and obtained control text information as input data; The workflow construction module is configured to construct a graph generation workflow based on a diffusion model, and add a text encoder, a control network and a fine-tuning model to the graph generation workflow, and the control network is configured to control content generation; The graph generation module is configured to perform encoding control processing on the input data in the graph generation workflow, and generate structure-controlled image content with semantic matching degree by processing the text base map and the graph base map in a target dimension through a selected target fine-tuning model and the control network, and output a final graph.

10. An electronic device, comprising: The device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is configured to store a computer program. A processor for implementing the steps of the AIGC-based data chart mapping method according to any one of claims 1-8 when executing a program stored on a memory.

Images