A chart data analysis text generation method

Abstract

The application belongs to the technical field of computer vision and natural language processing application, and particularly relates to a chart data analysis text generation method, which comprises an image coding module, a chart text contrast learning module and a script generation module. On the basis of combining a traditional chart text generation model, the model can obtain the features and change trends of charts in a finer granularity through a double-guiding mode of data analysis text and analysis feature query, and generate a popular script description in combination with a large language model. The application can well realize dynamic chart recognition and analysis, reduce subjectivity and singleness caused by manual participation, provide rich cases for a business simulation application based on data chart analysis, improve simulation training difficulty, and more effectively help experimenters learn more complex business changes.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision and natural language processing application, and particularly relates to a chart data analysis text generation method. BACKGROUND

[0002] At present, in the business intelligence decision simulation scene, a large amount of market fluctuation trend and its analysis often need to be provided, and the existing simulation system mostly analyzes the market trend by using fixed copywriting description, which requires manual analysis and writing of analysis copywriting for different market trend charts. On the one hand, this mode needs a large number of experts to manually process, consumes manpower and subjective inclination is difficult to avoid; on the other hand, market trends change variously, and the data chart analysis result preset by manual analysis in advance often cannot reflect the dynamics and diversity of the market. Therefore, the present method proposes to create a generative data chart analysis algorithm by combining image processing technology and natural language processing algorithm, so as to provide rich and dynamic market trend analysis copywriting for users.

[0003] The prior art generates text by taking an image as input, and the effect is good for images in nature (landscape, animals, etc.). Although the types of chart data are few, the features are obvious and do not conform to the regular image feature rules, so it is difficult for traditional methods to obtain real information and trend changes of the chart, and the subsequent text generation is also affected. SUMMARY

[0004] The present application proposes a chart data analysis text generation method which is reasonable in design, simple in method, strong in theory, and can automatically analyze data charts and generate corresponding text descriptions, aiming to reduce the subjective inclination of manual description by means of the automatic algorithm, provide generative market trend analysis copywriting for business decision simulation, improve the authenticity and diversity of simulation, and further increase the difficulty of training and learning effect.

[0005] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows: a chart data analysis text generation method, comprising an image encoding module, a chart text contrast learning module and a copywriting generation module;

[0006] The image encoding module uses a pre-training image encoding model (CLIP's ViT-L / 14) based on ViT (Vision Transfomer) as an image encoder, deletes the last layer of the ViT, and uses the output characteristics of the second layer of the last layer as the image encoding result, denoted as v c , and the original chart is denoted as chart, and the calculation process is as follows:

[0007] v c =ViT(chart)

[0008] The chart text contrast learning module sets an alignment judgment model between a learning input chart and text, which is constructed in a binary classification form. The core features of the chart are extracted through text information, and the features are output as new vectors aligned with the text embedding. A new calculation mode is introduced to complete the similarity calculation of image encoding and text encoding. The initial vector representation of the text is embedded by the Bert pre-training model, denoted as v t The calculation process is as follows:

[0009] v t =Bert(text)

[0010] During the training process of the alignment judgment model, a feature query intervention mode is proposed to enable the model to fully extract effective features in the chart. The query is a labeled data composed of chart type, chart element name, and chart field. The interaction calculation process of the three vectors is denoted as:

[0011] v si =QATT(v c ,v t ,v qi )

[0012] where v si represents the extracted chart-text pair feature vector, v qi represents the embedding value of the query, i represents the i-th query, and the detailed calculation process of QATT() is as follows:

[0013]

[0014] The similarity between v c and v qi is calculated by the sim() similarity calculation function, and the similarity calculation result is used as a weighted balance of the traditional attention mechanism. The result under the best query is selected as the final output result by the Top() sorting function.

[0015] The script generation module obtains the feature vector v s of the chart-text pair, and uses a large language model LLM to complete the text generation. The formula is:

[0016] Output sentence=LLM(FC(v s ))

[0017] where FC() represents a fully connected layer, and the original parameters of ViT, Bert, and LLM are sealed during the training process. The final output generates the script Output sentence.

[0018] As preferred, the calculation mode QATT() of the chart text comparison module is guided by query and text description, so that the model learns how to obtain chart features.

[0019] As preferred, the v qi It can be represented by Bert or other models, and neural network random initialization can also be used.

[0020] As preferred, the FC() full connection layer of the text generation module adjusts v si to adapt to the dimension of the LLM.

[0021] Compared with the prior art, the application has the advantages and positive effects that:

[0022] On the basis of combining traditional chart text generation models, the model can obtain chart features and trends in more fine-grained manner through the double-guided mode of data analysis text and feature query, and generate popular text descriptions combined with large language models. The application can well realize dynamic chart recognition and analysis, reduce subjectivity and singularity caused by manual participation, provide rich cases for business simulation applications based on data chart analysis, improve simulation training difficulty, and more effectively help learners learn more complex business changes. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0024] Figure 1 A chart data analysis text generation method model structure diagram is provided for the embodiment.

[0025] Figure 2 An original input image example diagram is provided for the embodiment. DETAILED DESCRIPTION

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0027] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the application. However, it will be apparent to one skilled in the art that the application can be practiced without the specific details set forth in this description, and that the application is not limited to the particular embodiments described in detail below.

[0028] In the prior art, images are input to generate text. In nature, images such as landscapes and animals have good effects, and although the types of chart data are few, the features are obvious and do not conform to the conventional image feature rules, so it is difficult for traditional methods to obtain chart real information and trend changes therefrom, and subsequent text generation is also affected. To achieve the above purpose, the method is divided into the following three modules, which are image encoding module, chart text contrast learning module and copywriting generation module.

[0029] First, the image encoding module uses a pre-trained image encoding model (ViT (Vision Transfomer) based CLIP ViT-L / 14) as an image encoder, removes the last layer of ViT, and uses the output characteristics of the second layer of the last layer as the image encoding result, denoted as v c , the original chart is denoted as chart, and the calculation process is as follows:

[0030] v c = ViT (chart)

[0031] Second, the chart text contrast learning module, due to the substantial difference between images and text in embedding process and principle, direct text generation of image features is difficult to obtain good results, and the training of this process will also increase the training difficulty due to the non-symmetry of the two vectors. Therefore, after obtaining the initial encoding of the chart, this module sets a learning input alignment judgment model between the chart (chart) and the text (text), which is constructed in a binary classification form, so that the model can correctly judge whether the current input image and text are consistent with the description or not. The core function is to extract the core features of the chart through the text information, and output the features as new vectors aligned with the text embedding. The application uses attention mechanism to complete the similarity calculation of image encoding and text encoding. The initial vector of the text is embedded by the BERT pre-training model, denoted as v t , and the calculation process is as follows:

[0032] v t = Bert (text)

[0033] On this basis, we want to realize a v cKey feature extraction and feature and semantic alignment mapping, due to the difference between chart features and conventional natural image features, too single color and chart text and numbers will make it difficult for ordinary image embedding model to capture comprehensive chart information, therefore, in the training process of the alignment module, the mode of data analysis feature query intervention is proposed, so that the model can fully extract the effective features in the chart, the query is a labeled data, composed of chart type, chart element name, chart field and other contents, the three vector interaction calculation process is recorded as:

[0034] v si =QATT(v c ,v t ,v qi )

[0035] Where v si Indicates the extracted chart-text pair feature vector, v qi Indicates the embedding value of query, i indicates the ith query, and the detailed calculation process of QATT() is as follows:

[0036]

[0037] v qi Can be represented by Bert or other models, and can also be randomly initialized using neural network. Considering the difference between chart image data and conventional image, including limited chart color, usually with fixed color to represent a certain expression, resulting in color change is not obvious, therefore, the traditional image processing mode is difficult to extract features under this data, the present application introduces a new calculation mode QATT(), aims to learn how to obtain chart features under the guidance of query and text description. The detailed calculation process of QATT() is as follows:

[0038]

[0039] The similarity between v c And v qi Is calculated by the sim() similarity calculation function, and the similarity calculation result is used as the weighted balance of traditional attention mechanism, and the result under the best query is selected as the final output result by the Top() sorting function;

[0040] Finally, the text generation module, after obtaining the chart-text pair feature vector v si , the large language model LLM is used to complete the text generation work, and its formula is:

[0041] Output sentence=LLM(FC(v si ))

[0042] Where FC() represents a fully connected layer, which will connect v si The dimensions were adjusted to fit LLM, and the original parameters of ViT, BERT, and LLM were preserved during training. The final output is a text sentence.

[0043] like Figure 2 The original image input shown is as follows: Figure 1 In the model shown, the graph encoding model is converted into an image vector, and queries (e.g., European market, market prospects, enterprise users, etc.) and text descriptions (e.g., high-end family users account for the highest proportion in the European market, but the opposite is true in Asia) are input. After calculation, the calculated vector is transformed in dimension through FC and input into the language large model LLM to generate text descriptions.

[0044] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments for application in other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for generating text from chart data analysis, characterized in that, It includes an image encoding module, a chart and text comparison learning module, and a copywriting generation module; The image encoding module uses the ViT-L / 14 pre-trained image encoding model CLIP as the image encoder, while removing the last layer of ViT and using the output characteristics of the second layer of the last layer as the image encoding result, denoted as . The original chart is denoted as chart, and the calculation process is as follows: ； The chart-text comparison learning module sets up an alignment judgment model between the input chart and text, constructed in a binary classification manner. While extracting the core features of the chart from the text information, it outputs the features as a new vector aligned with the text embedding. A new calculation mode is introduced to complete the similarity calculation between image encoding and text encoding. The initial vector representation of the text is embedded using a BERT pre-trained model, denoted as... The calculation process is as follows: ； During the training process of the alignment judgment model, a mode of intervention through data analysis feature query is proposed, enabling the model to fully extract effective features from the chart. The query is labeled data, consisting of chart type, chart element name, chart domain, etc. The three vector interaction calculation process is denoted as follows: ； in This represents the extracted chart-text pair feature vector. This represents the embedding value of the query, where i represents the i-th query. The detailed calculation process is as follows: ； The similarity calculation function sim() is used to calculate the similarity. and Similarity calculation is performed, and the similarity calculation result is used as a weighted balance for the traditional attention mechanism. The best result under the query is selected as the final output by the Top() sorting function. The copy generation module obtains the feature vectors of the chart-text pairs. Then, the Large Language Model (LLM) is used to complete the text generation task, and its formula is as follows: ， Here, FC() represents a fully connected layer, and the three modules are sealed during training. , LLM raw parameters, final output generated copy .

2. The method for generating text from chart data analysis according to claim 1, characterized in that, The calculation mode QATT() of the chart-text comparison module, guided by both query and text description, enables the model to learn how to obtain chart features.

3. The method for generating text from chart data analysis according to claim 1, characterized in that, The chart text comparison module It can be represented by BERT or other models or randomly initialized using a neural network.

4. The method for generating text from chart data analysis according to claim 1, characterized in that, The FC() fully connected layer of the copy generation module will Adjusted to fit the dimensions of LLM.

Images