Big language model assisted literature visual analysis method and system

Abstract

The invention relates to the technical field of literature measurement, and discloses a literature visual analysis method and system assisted by a large language model, and the method comprises the steps: matching a chart and an introduction document corresponding to the chart in a visual knowledge base according to an input problem; combining the input question and the introduction document to generate a first cue word, and inputting the first cue word into a large language model; the big language model outputs a structured language and a visual image reconstructed by a four-step method; interpreting the visual image by the visual large language model, and outputting visual interpretation data; visual interpretation and question input are combined to generate a third cue word, the third cue word is input into the large language model, and the large language model generates a final visual chart and chart visual interpretation; based on a large language model, by combining a customized knowledge base and a general knowledge base, the visualization diversity is increased, and high professional performance and strong generalization ability of unconventional problems are shown in the field of literature metering; and by adopting abnormal feedback, closed-loop control of the intelligent agent on a visual result is realized, and the reliability is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of bibliometrics, and particularly relates to a method and system for literature visual analysis assisted by a large language model. Background Art

[0002] In the field of bibliometrics, in-depth analysis of a literature database is crucial for researchers. Whether it is a novice scholar or a senior expert in the field, they all need to conduct literature analysis to grasp the development context of the field, track research hotspots, lock in important literature and academic leaders. Traditional literature visualization tools, such as HistCite, Citespace, VOSviewer, etc., provide a variety of chart templates to help users understand data from different perspectives. However, these tools have some deficiencies. For example, they have strict requirements for the input data format, a high threshold for users to get started, complex parameter settings, and the need to install multiple third-party toolkits to obtain satisfactory chart results.

[0003] With the improvement of the code generation ability of large language models, new large language model-driven visualization tools have emerged. They allow users to create visualization charts through natural language descriptions. Such tools liberate users from cumbersome coding work and make data visualization analysis easier and more efficient. For example, the DataScience tool of the Open large language model can automatically generate data summaries and prompt the large language model to generate Python code for data conversion and visualization. The Minard tool, through a better interactive interface, enables users to modify charts by asking questions and view the modification records. The Data Formulator tool further splits the visualization task into two parts: data processing and chart specification, improving the user's operation accuracy and the richness of chart types.

[0004] Although these new tools have achieved remarkable success in simplifying visualization analysis work, they still have some deficiencies. First, due to the insufficient knowledge reserve of large language models, most of these tools use simple Python matplotlib libraries or specifications such as Vega-lite, resulting in limited expressive ability of visualization results. Second, except for DataFormulator, other tools, due to the preference problem of large language models, will classify visualization tasks where users do not specify chart types into a limited number of simple charts, restricting the richness of visualization results. Finally, these tools lack a closed-loop feedback mechanism and do not guarantee the reliability of visualization results. Summary of the Invention

[0005] Aiming at the deficiencies of the existing technology, the purpose of the present invention is to provide a method and system for literature visual analysis assisted by a large language model.

[0006] To achieve the above object, the present invention provides the following technical solutions: A method for literature visual analysis assisted by a large language model, comprising the steps of: Comprising the steps of: Match charts and the corresponding introduction documents in the visual knowledge base according to the input question; Merge the input question with the introduction document to generate a first prompt word and input it into the large language model; The large language model outputs a structured language and a visual image reconstructed by the four-step method; Merge the structured language, the introduction document, and the visual image to generate a multi-modal prompt word and input it into the visual large language model. The visual large language model interprets the visual image and outputs visual interpretation data; Merge the visual interpretation and the input question to generate a third prompt word and input it into the large language model. The large language model combines the corresponding relationships between the feature quantities, statistical quantities, and the concepts involved in the user's question, and the subjective insights corresponding to the identities of scientific researchers to generate the final visual chart and the visual interpretation of the chart.

[0007] In the present invention, preferably, the step of generating the first prompt word includes: Judge the question type, the types include new questions and follow-up questions. When the type is a new question, integrate the type and the question into a prompt word; When the type is a follow-up question, integrate the type, the question, and the corresponding existing question as a prompt word.

[0008] In the present invention, preferably, the structured language reconstructed by the four-step method includes multi-dimensional feature quantity matching, statistical quantity determination and solution, data structure design, and visualization form selection and detail determination.

[0009] In the present invention, preferably, according to the first prompt word, it also includes the content description document of the source database.

[0010] In the present invention, preferably, the visual large language model uses the introduction document of the input chart as knowledge to comprehensively describe the visual phenomena presented by each visual element in the visual image, and according to the structured language reconstructed by the four-step method corresponding to the chart, corresponds the visual phenomena to the properties shown by the selected external features and statistical quantities of the literature to obtain the corresponding visual interpretation.

[0011] In the present invention, preferably, it further includes feedback adjustment. The case codes and introduction documents of the four-step method for paraphrasing programming problems and the selected charts, as well as the detailed requirements for the roles and tasks of the large language model, are combined to generate an initial prompt word, which is input into the large language model to make it output the action selection and code for the first iteration. In each subsequent iteration, the initial prompt word and the context prompt words recorded in each previous iteration are combined and then input into the large language model for iterative debugging. Under the guidance of the four-step description of the programming problem, the intelligent body will automatically carry out the programming and debugging of the data processing process first, then the programming and debugging of the visualization rendering code, and after the visualization chart is successfully rendered, it will independently decide whether to re-perform the programming and debugging of the data processing or visualization rendering according to the result feedback, or end the programming and output the result.

[0012] In the present invention, preferably, it further includes exception capture: during the operation of the large language model, while capturing the exception information, the exception stack is traced to achieve timely traceability, and the complete exception chain is converted into an easy-to-read text format and sent to the large language model to guide the large language model to trace and correct the error or warning information that occurs during the program operation.

[0013] A literature visual analysis system assisted by a large language model, comprising: A visualization knowledge base, the visualization knowledge base includes a customized knowledge base and a general knowledge base. The customized knowledge base stores customized chart templates, and the general knowledge base is used to store general chart templates; A bibliometric structure body, which is used to process the input problem and generate a prompt word; A large language model, the large language model is used to output the structured language, visualization chart or chart visual interpretation reconstructed by the corresponding four-step method after arithmetic analysis according to the input problem; A visual large language model, which is used to interpret the visualization image and output visual interpretation data.

[0014] In the present invention, preferably, it further includes an interaction interface. The interaction interface includes an interactive question area and a visualization result display area. The interactive question area is used to input natural language, and the visualization result display area is used to present the output structured language, visualization chart and chart visual interpretation data.

[0015] In the present invention, preferably, the interaction interface further includes a display area mode switching control area, an upload data area and a visualization result operation area.

[0016] Compared with the prior art, the beneficial effects of the present invention are: The method of the present invention is based on a large language model. By combining the customized knowledge base and the general knowledge base, it increases the diversity of visualization, can show high professional performance in the field of bibliometrics, and has strong generalization ability for unconventional problems; By adopting anomaly feedback, closed-loop control of the visualization results by the agent is achieved, significantly improving the reliability of encoding. This closed-loop control ensures that the system can self-detect and correct errors during the process of autonomously reasoning to solve user problems, thus providing more accurate and reliable visualization results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic flowchart of the method described in the present invention.

[0018] Figure 2 It is a schematic flowchart of the structured language reconstructed by the four-step method described in the present invention.

[0019] Figure 3 It is a flowchart of the visual interpretation of the visualization described in the present invention.

[0020] Figure 4 It is a schematic flowchart of the feedback adjustment process of the large language model described in the present invention.

[0021] Figure 5 It is an example diagram of the chart effect described in the present invention.

[0022] Figure 6 It is a schematic flowchart of the content of the visualization knowledge base and the process of constructing the vector database described in the present invention.

[0023] Figure 7 It is a schematic diagram of the system interaction interface described in the present invention.

[0024] Figure 8 It is a schematic diagram of the enlarged display chart of the system interaction interface described in the present invention.

[0025] Figure 9 It is the preview mode of the visualization result display area of the system interface described in the present invention.

[0026] Figure 10 It is a schematic diagram of the visualization interface of Example 1.

[0027] Figure 11 It is a schematic diagram of the interface of the histogram with modified colors in Example 1.

[0028] Figure 12 It is a schematic diagram of the interface modified to a stacked river chart in Example 1.

[0029] Figure 13 It is a schematic diagram of the interface modified to a hierarchical edge bundling chart in Example 1.

[0030] Figure 14 It is a schematic diagram of the enlarged effect interface of the hierarchical edge bundling chart in Example 1.

[0031] Figure 15 Schematic diagram of the interface of the bubble chart generated in the second embodiment.

[0032] Figure 16 Schematic diagram of the interface of the magnified effect of the bubble chart generated in the second embodiment.

[0033] Figure 17 Schematic diagram of the interface of the modified bubble chart in the second embodiment.

[0034] Figure 18 Schematic diagram of the interface of the generated diverging stacked bar chart in the second embodiment.

[0035] Figure 19 Schematic diagram of the interface of the generated force-directed graph in the second embodiment.

[0036] Figure 20 Schematic diagram of the interface of the modified force-directed graph in the second embodiment. Detailed implementation manners

[0037] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0039] Please refer to Figure 1 , a preferred embodiment of the present invention provides a large language model-assisted literature visual analysis method. Based on the large language model, through natural language dialogue interaction, corresponding visual charts are generated to achieve complex visual analysis tasks, specifically including the following steps: S1. Match the chart and the introduction document corresponding to the chart in the visual knowledge base according to the input question; S2. Combine the input question with the introduction document to generate a first prompt and input it into the large language model; S3. The large language model outputs a structured language and a visual image reconstructed by the four-step method; S4. Combine the structured language, the introduction document, and the visualization image to generate a multi-modal prompt, and input it into the vision-language model. The vision-language model interprets the visualization image and outputs visual interpretation data. S5. Combine the visual interpretation and the input question to generate a third prompt and input it into the language model. The language model combines the corresponding relationships between the feature quantities, statistical quantities, and each concept involved in the user's question, as well as the subjective insights corresponding to the identities of scientific researchers to generate the final visualization chart and chart visual interpretation.

[0040] Specifically, in steps S1 - S3, as Figure 2 shown, the structured language reconstructed by the four-step method includes multi-dimensional feature quantity matching, statistical quantity determination and solution, data structure design, and visualization form selection and detail determination. The matching of multi-dimensional feature quantities needs to describe the selection of several externally defined literature feature quantities in the source database according to the user's needs, such as literature ID, author, year, citation array, etc., and the selected feature quantities are extracted by indexing and other methods in the subsequent language model programming part; the determination and solution of statistical quantities need to describe the language definitions of some necessary data that require secondary processing to obtain, and are solved one by one in the language model programming part, such as counting the number of papers published by a certain author in a certain year, the total number of citations between two countries, the keyword with the most co-occurrence times at a certain time, etc.; the data structure design needs to describe the language definition of the data structure used to store the above-obtained feature quantities and statistical quantities, such as choosing year, author, or country as the unit entry, and choosing a linear list or a directed graph as the structure form, and corresponding data integration operations are performed in the language model programming part; the visualization form selection and detail determination need to describe the chart name selected from the knowledge base and emphasize the requirements for chart details in the user's needs.

[0041] Retrieve several charts with relatively high semantic similarity matching degrees in the visualization knowledge base according to the user's input question. Then, merge the user's question, the content description document of the source database, and the introduction documents of the top-ranked charts pre-matched in the knowledge base to generate a first prompt and input it into the language model, requiring the language model to use a structured output format to give the result of restating the user's question in the structured language reconstructed by the four-step method at one time, so that the system can extract the description of each step and the selected chart name from the output text using formatted strings to provide guidance for the subsequent part.

[0042] In step S4, the visual large language model uses the introduction document of the input chart as knowledge. The introduction document includes the expression characteristics of the chart itself, comprehensively describes the visual phenomena presented by each visual element in the visualization image, and according to the structured language reconstructed by the four-step method corresponding to the chart, corresponds the visual phenomena to the properties shown by the selected external features and statistics of the literature, obtaining the corresponding visual interpretation. Assemble the output result of the visual large language model and the current user's question into a prompt to call the general large language model, and further explain the above visual interpretation result according to the user's original question, focusing on explaining the corresponding relationship between the feature quantity, statistic and each concept involved in the user's question; finally, the large language model will also give corresponding subjective insights in the capacity of a scientific researcher. The overall process is as Figure 3 shown.

[0043] In this embodiment, the step of generating the first prompt further includes: Judging the question type, the types include new questions and follow-up questions. When the type is a new question, integrate the type and the question into a prompt; When the type is a follow-up question, integrate the type, the question and the corresponding existing question as a prompt.

[0044] In this embodiment, it also includes feedback adjustment, as Figure 4 shown, paraphrase the four-step method into programming questions, combine the case code and introduction document of the selected chart, and the detailed requirements for the role and tasks played by the large language model to generate an initial prompt, input it into the large language model to make it output the action selection and code for the first iteration. For each subsequent iteration, the initial prompt and the context prompts recorded in each previous iteration will be combined and input into the large language model for iterative debugging. Under the guidance of the four-step method description of the programming question, the large language model will self-drive to first program and debug the data processing process, then program and debug the visualization rendering code, and after the visualization chart is successfully rendered, independently decide whether to re-program and debug the data processing or visualization rendering according to the result feedback, or end the programming and output the result.

[0045] In this embodiment, whether it is the data processing process or the visualization rendering coding process, the large language model submits the written code in the form of a function with a fixed prototype. For example, in the data processing process, the complete source literature database is passed into the function as a parameter, and the processed data structure is passed out of the function in a return manner; in the visualization rendering coding process, the processed data structure is passed into the function as a parameter, and the program interface related to visualization is passed in by means of a callback function, without any return value. The large language model only needs to write the definition part of the function body according to the requirements, which is simple, reliable and easy to understand.

[0046] In this embodiment, for the data processing process, the large language model is only imported, and the imported knowledge is a detailed explanation of the data structure corresponding to the selected chart, as well as the field naming and indexing method of the source file database; for the visual rendering encoding process, the imported knowledge is an introduction to the input data structure and an example of the visual rendering code corresponding to the selected chart, as well as detailed annotations to the code to guide the large language model to write rendering code. The visualization knowledge base contains chart case codes and corresponding introduction documents that are carefully designed for common problems in the field of bibliometrics. The introduction document is accompanied by corresponding code explanations, data structure descriptions, descriptions of expected visualization effects, and visual elements that need to be explained. The index vector is calculated using the classification and specific problem sets that can be solved to form a vector database. Examples of chart effects are shown in the figure below. Figure 5 As shown in Figure 2, the content of the visual knowledge base and the process of constructing the vector database are as follows: Figure 6 shown.

[0047] In this implementation, the problems that may occur in the large language model are divided into priorities. In order to fully ensure the effectiveness, fit and readability of the output results, a targeted approach to the hierarchical definition of the review standards is proposed. Different levels of resolution are given according to the severity of the hierarchical definition problems, and users are informed of possible defects in a realistic manner.

[0048] Specifically, the quality and safety of the output results of the large language model are divided into three levels of abnormalities. The first level must be achieved. If there is a problem, it will be solved. If it cannot be solved, it will be thrown as a major exception; the second level is achieved with all efforts. If problems are found, they will be solved. If they cannot be solved, they will be classified as severe warnings and thrown out; the third level is pursued with efforts. If problems are found and improvement methods can be directly proposed, they will be modified. Otherwise, no improvement methods will be proposed. If they cannot be solved, they will be thrown out as warm reminders.

[0049] For the large language model and the vision large language model in the present invention, the program that meets the first-level standard must be able to run normally, and the corresponding abnormal situation is that the program submitted by the large language model encounters an error during execution, resulting in the interruption of execution; for the program that meets the second-level standard, the data processing program is required to have correct running results, and the visual rendering encoding program is required to have complete and clear elements on the chart, which needs to be reviewed by comparing with the predicted ideal results; the program that meets the third-level standard requires that the chart results of the visual rendering meet the user's needs and the picture is beautiful and harmonious. The compliance assessment is achieved by intelligently visually interpreting the text of the visual results and the original user question, and guiding the vision large language model to make subjective judgments. The specific rating criteria are shown in Table 1. The rating criteria and the traceability method are passed into the large language model so that it can perform autonomous closed-loop debugging. The large language model discovers errors independently during the programming process and corrects them in a timely manner. To ensure that the large language model checks all program problems without omission, it is first necessary to correctly predict the ideal output result to ensure the correct target direction; secondly, it is necessary to define the review criteria at different levels and grasp the scale of the review; at the same time, it is also necessary to trace the source, from the result to the cause, discover the source of the problem and modify it in a timely manner.

[0050] Table 1. Rating Criteria

[0051] In this embodiment, it also includes exception capture: during the operation of the large language model, while capturing the exception information, the exception stack is tracked to achieve timely traceability, and the complete exception chain is converted into an easy-to-read text format and sent to the large language model to guide the large language model to trace and correct the error or warning information that occurs during the program operation.

[0052] Another preferred embodiment of the present invention provides a literature visual analysis system assisted by a large language model, including: A visual knowledge base, the visual knowledge base includes a customized knowledge base and a general knowledge base. The customized knowledge base stores customized chart templates, and the general knowledge base is used to store general chart templates; A bibliometric structure for processing the input question and generating a prompt word; A large language model, the large language model is used to output the restructured structured language, visual chart or chart visual interpretation corresponding to the four-step method after computing and analyzing according to the input question; A vision large language model for interpreting the visual image and outputting visual interpretation data.

[0053] Such as Figure 7As shown in the figure, in this embodiment, the literature visual analysis system further includes an interactive interface, which includes an interactive question area and a visualization result display area. The interactive question area is used to input natural language, and the visualization result display area is used to present the output structured language, visualization charts, and chart visual interpretation data. In list mode, this area will present the visualization results and corresponding interpretation texts given by the system after each interaction in a list form. There is a candidate box on the left side of each chart. When selected, the user is allowed to export a visualization report using the generate report button, and the preview effect of the report will be given in preview mode.

[0054] In this embodiment, the interactive interface further includes a display area mode switching control area, an upload data area, and a visualization result operation area. The display area mode switching control area can switch the visualization result display area between list mode and preview mode. The upload data area allows the user to import source data files or replace the knowledge base template. The visualization result operation area allows the user to operate on the selected generated charts, including deletion operations and generating reports and exporting operations.

[0055] As Figure 8 shown in the figure, in this embodiment, in order to allow users to clearly observe the generated visualization charts, all the charts in the visualization result display area in list mode can be full-screen displayed and previewed after being clicked. At this time, other irrelevant interface information will be grayed out, and the user can focus on observing the information on the chart or interacting with the chart.

[0056] As Figure 9 shown in the figure, in this embodiment, in order to enable users to more conveniently utilize the visualization results generated by the system, the system allows users to export the generated visualization charts and insight texts as a visualization report in the form of a word document. Select the chart to be exported in the visualization result display area in list mode and switch to preview mode to view the content effect of the exported report. The exported report only needs to be simply modified and improved to become a complete literature analysis report, which plays a guiding role in the user's future development plan in the current scientific research field.

[0057] Example 1: Xiao Li is a researcher who has just entered the field of visualization. Although he is very interested in literature analysis, he doesn't have much experience in visualization. Recently, he is preparing to write a review on the research trends in a specific field and hopes to deeply understand the cooperation relationship network among authors in the field. Facing a large amount of literature data and complex author relationships, Xiao Li hopes to use our system to achieve literature data visualization to help him complete this review. So he input a large amount of literature data of the IEEE VIS conference for visualization into the system.

[0058] First, he hoped to simply and intuitively understand the trend of the number of literature publications in the field of visualization through a histogram. So he entered in the dialog box, "I request to draw a histogram to show the publication trend of articles in the field of visualization." The system generated a visualization through analysis, as Figure 10 shown. This visualization shows the trend of the number of literature publications between 2000 and 2024.

[0059] It can be observed from the histogram that the number of literature publications shows an obvious upward trend between 2000 and 2024. Especially during the period from 2011 to 2022, the number of publications increased significantly and reached the peak. The data for 2023 and 2024 also show relatively high numbers of publications, although there is a slight decline. This indicates that during this period, the activity of literature publication has increased significantly, which may be related to the expansion of the research field, the increase in research funds, or the frequency of academic exchanges.

[0060] However, he felt that the colors of the histogram were not vivid enough and wanted to change them to red. So he proposed to the system, "Please change the color of the histogram to red." Subsequently, the color of the histogram was set to red to enhance the visual effect and highlight the change trend of the data, as Figure 11 shown.

[0061] Having obtained the visualization of the overall article publication trend in this field between 2000 and 2024, he then wanted to get the trend of the number of articles published by different countries in this field. So he requested from the system, "Please recommend another suitable visualization that can show the trend of the number of articles published by different countries in this field between 2000 and 2024." The output of the system is as Figure 12 shown.

[0062] This visualization shows the trend of the number of articles published by different countries in this field between 2000 and 2024. The horizontal axis is the time axis, and the vertical axis is the number of articles published by each country. Each country is represented by a line of a different color, and the height of the line represents the number of articles published by that country in a specific year.

[0063] It can be seen from the visualization that the overall number of articles shows a significant upward trend. Especially after 2010, the number of articles published by Country A has increased rapidly, indicating a substantial increase in its research activity in this field. In contrast, the number of publications by Country B has declined after reaching the peak in 2019, but still remains at a relatively high level. This shows that although Country B still occupies an important position in the research of this field, the rapid growth of Country A may have an important impact on the global research pattern in the future.

[0064] He hoped to understand the cooperation relationships among the authors of these papers and the degree of their closeness, and to find the core nodes and isolated authors in the cooperation network. So he asked the system, "What is the cooperation among the authors?" The system matched the template of the cooperation network and automatically generated the corresponding visual encoding. This encoding uses grouped colors to identify authors from the same country, and presents the cooperation relationships through nodes and curves. The connections of the curves represent the cooperation among the authors, and the number of connecting lines reflects the degree of closeness of the cooperation. Based on these encodings, the system outputs a network diagram of the cooperation relationships, thus visually showing the distribution of the core nodes and the marginal nodes, such as Figure 13 , 14 as shown

[0065] This visualization shows what the cooperation among the authors in this field is like. Among them, the words forming the circles are the names of the top 100 authors with the largest number of articles participated in among all the authors involved in uploading conference data in the data, grouped according to the countries to which the authors belong, and the names of authors from the same country are in the same color. We can see that the number of authors from Country A is the largest. In particular, the cooperation of "Huamin Qu" with many international authors shows the activity of Country A in this field. At the same time, there are connections between the author names of different colors, indicating that there is cooperation among authors from different countries in this field, reflecting the importance of international cooperation in academic research.

[0066] So far, Xiao Li has successfully completed his research. This system not only saved him the cumbersome programming work, but also helped him achieve complex data analysis when he had no visualization foundation.

[0067] Example 2: Xiao Zhang is a data analyst who often uses visualization to analyze data. Therefore, she is very familiar with the chart design and basic terms of visualization and has a certain sense of visualization aesthetics. Because she often uses visualization, she is very interested in the situation in the academic field of visualization and hopes to quickly and simply understand the development of each country in the field of visualization through our system and find the authors she should pay attention to, etc. These information will help her understand the field of visualization more quickly and accurately. So she input a large amount of literature data in the field of visualization into the system.

[0068] First, she hopes to understand the overall situation in the field of visualization. She enters the requirement "Please use a bubble chart to show the distribution of the number of papers in different countries, where the y-axis represents the countries, the x-axis represents the time of article publication from 2000 to 2024, the size of the bubble represents the number of articles in that year, and the size of the bubble is proportional to the number of articles. The larger the number, the larger the bubble. Pay attention to controlling an appropriate scale. And there should be a note on the right side of the figure to introduce the order of magnitude of bubbles of different sizes. The overall color of the figure is blue with a transparency of 50%. At the same time, I hope to add an interactive effect and add a tooltip to display the detailed data corresponding to the bubble." as a prompt to understand the overall distribution of the literature quantity.

[0069] The system successfully gives a bubble chart that meets the user's requirements, allowing for an intuitive observation and understanding of the changing trends in the number of papers published by different countries in each year through the interactive effect, as Figure 15 、 16 shown. Over time, the overall number of papers shows an upward trend, which may reflect the annual growth and development of global academic research.

[0070] However, at this time, due to the excessive number of countries, she finds that some bubbles are too small, there is overlap between the bubble charts, and it is not convenient to observe. So she hopes to improve the previous figure. "Based on the previous figure, help me redraw a bubble chart. I hope to only retain the countries where the number of articles in a certain year exceeds 100. If the number of articles of a country does not exceed 100 every year, then ignore that country." as a prompt.

[0071] Then the system recognizes it as a follow-up question to the previous one and makes corresponding improvements to the visualization, as Figure 17 shown. The bubble chart at this time is retained according to the requirements, and only four countries remain. She believes that these four countries are relatively important countries in the field of visualization. At the same time, she finds that the number of papers of Country A and Country B in multiple years is significantly higher than that of the other two countries. She speculates that the academic research of these two countries in the field of visualization is very active and needs to be focused on.

[0072] At this time, she hoped to compare the number of papers in the two countries in detail, but the comparison between Country A and Country B in the bubble chart was not intuitive. So she input into the system, "Please help me redraw a graph. I hope to use a diverging stacked bar chart to compare the number of papers in Country A and Country B. The x-axis extends to the left and right, representing the number of papers. Note that the scale values on both ends of the x-axis are the same. The y-axis represents the years when the articles were published, from 2000 to 2024. I hope Country A is on the left, with the color #CD9B1D, and Country B is on the right, with the color #7CCD7C. Also, add an interactive effect, and there should be a tooltip to see the corresponding detailed data." As a prompt, the diverging design of the diverging stacked bar chart can effectively display the comparison between the two countries, making it intuitive to understand the academic contributions of the two countries.

[0073] The system gave a diverging stacked bar chart as required, as Figure 18 shown. It can be seen from the graph that the overall trend of the number of papers published in both Country A and Country B is increasing year by year. Among them, the number of papers in Country A has increased significantly in recent years, especially reaching a peak in 2022 and 2024. While the number of papers published in Country B has decreased after reaching a peak in 2019.

[0074] She thought this might indicate that the output of Country A in the field of academic research is growing rapidly, reflecting its continuous efforts in scientific research investment and talent cultivation; then she hoped to find the relatively important authors in this field. According to the prompts of the questions she might care about given by the system, and knowing that the system already had a template for the force-directed graph, she input, "Please use a force-directed graph to show the cooperation relationship among the top 100 authors with more published articles. Group them according to the countries to which the authors belong, and the author names from the same country have the same color. Help me add a legend to introduce the countries represented by different colors." After understanding the user's input, the system adjusted the existing template to meet the user's needs. And on the basis of meeting the user's requirements for the graph, it also adjusted the size of the circles representing the authors. One can intuitively see the number of collaborations of the authors according to the size of the circles, as Figure 19 shown.

[0075] According to the force-directed graph, she found that in the cooperation network, some scholars not only cooperate with scholars from their own countries but also actively establish connections with scholars from other countries. These "middlemen" play an important role in promoting exchanges among scholars from different countries: for example, Huamin Qu, who has a high number of published papers, has established cooperation relationships with many scholars from different countries, indicating that in visual research, there are some influential scholars who play an active role in promoting international cooperation.

[0076] However, since the number of authors at this time was too large to observe easily, she asked the system, "Help me redraw a graph. I hope to understand the collaboration relationships among the top 20 authors with the most published articles." as a prompt.

[0077] The system successfully made the correct modification to the visualization, and the number of authors shown decreased, as Figure 20 shown. She believes that these 10 scholars should be the authors worthy of key attention in the field of visualization. At the same time, she learned that Chinese scholars play an important role in the research of the visualization field: among the top 5 scholars with the most published papers, four are Chinese scholars, namely Huamin Qu, Wei Chen, Shixia Liu, and Nan Cao. This may indicate that the Chinese have played an important role in the research of the visualization field and promoted the development of this field. And at this time, the connections between the authors are clearer, and it can be seen that the collaboration relationship between Huamin Qu and Yingc Large Language Model Wu is particularly close, showing a strong academic collaboration network.

[0078] Since then, she has generally understood the overall situation of the visualization field and obtained the author information that should be concerned about and the collaboration situation between the authors.

[0079] In some other preferred embodiments of the present invention, there is provided a computer-readable storage medium storing a computer program, which when executed by a processor causes the processor to execute the steps of the method as described in the above embodiments.

[0080] If the above functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. And the aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs and other various media that can store program codes.

[0081] The above description is a detailed description of the preferred and feasible embodiments of the present invention, but the embodiments are not used to limit the patent application scope of the present invention. Any equivalent changes or modifications made under the technical spirit prompted by the present invention should fall within the patent scope covered by the present invention.

Claims

1. A document visual analysis method assisted by a large language model, characterized in that: Includes steps: Match the chart and the corresponding introduction document in the visual knowledge base according to the input question; Combining the input question with the introduction document to generate a first prompt word and inputting it into the large language model; The large language model outputs structured language and visual images reconstructed in four steps; The structured language, the introduction document and the visual image are combined to generate multimodal prompt words, which are input into a visual large language model, and the visual large language model interprets the visual image and outputs visual interpretation data; The visual interpretation and the input question are combined to generate a third prompt word which is input into the large language model. The large language model combines the correspondence between the feature quantities, statistics and the concepts involved in the user's question and the subjective insights corresponding to the identity of the scientific researcher to generate the final visualization chart and visual interpretation of the chart.

2. The method for document visual analysis assisted by a large language model according to claim 1, characterized in that: The step of generating the first prompt word comprises: Determine the question type, which includes new questions and follow-up questions. When the question type is a new question, integrate the question type and question into a prompt word; When the type is a follow-up question, integrate the type, question and corresponding existing question as prompt words.

3. The method for document visual analysis assisted by a large language model according to claim 1, characterized in that: The structured language reconstructed by the four-step method includes multi-dimensional feature quantity matching, statistical quantity determination and solution, data structure design and visualization form selection and detail determination.

4. The method for document visual analysis assisted by a large language model according to claim 1, characterized in that: According to the first prompt word, a content description document of the source database is also included.

5. The method for document visual analysis assisted by a large language model according to claim 1, characterized in that: The visual big language model uses the introduction document of the input chart as knowledge to comprehensively describe the visual phenomena presented by each visual element in the visualization image, and reconstructs the structured language according to the four-step method corresponding to the chart, corresponds the visual phenomena to the properties shown by the external features and statistics of the selected documents, and obtains the corresponding visual interpretation.

6. A method for document visual analysis assisted by a large language model according to claim 5, characterized in that: It also includes feedback adjustment, which combines the four-step method of retelling programming problems, the case code and introduction documents of the selected charts, and the detailed requirements for the roles and tasks of the large language model to generate initial prompt words, which are input into the large language model to output the action choices and codes for the first iteration. Each subsequent iteration will merge the initial prompt words with the context prompt words recorded in the previous iterations and input them into the large language model for iterative debugging.

7. The method for document visual analysis assisted by a large language model according to claim 6, characterized in that: It also includes exception capture: during the operation of the large language model, the exception information is captured and the exception stack is tracked to achieve timely tracing, and the complete exception chain is converted into a text format and sent to the large language model, guiding the large language model to trace and correct the error or warning information that occurs during the program running.

8. A document visual analysis system assisted by a large language model, characterized in that: include: A visualization knowledge base, the visualization knowledge base includes a customized knowledge base and a general knowledge base, the customized knowledge base stores customized chart templates, and the general knowledge base stores general chart templates; Bibliometric structure, used to process input questions and generate prompt words; A large language model, wherein the large language model is used to output a structured language, a visual chart or a visual interpretation of a chart reconstructed by the corresponding four-step method after operation and analysis based on an input question; The visual language model is used to interpret visual images and output visual interpretation data.

9. A document visual analysis system assisted by a large language model according to claim 8, characterized in that: It also includes an interactive interface, which includes an interactive question area and a visual result display area. The interactive question area is used to input natural language, and the visual result display area is used to present the output structured language, visual charts and chart visual interpretation data.

10. A document visual analysis system assisted by a large language model according to claim 9, characterized in that: The interactive interface also includes a display area mode switching control area, an upload data area and a visualization result operation area.

Images