A subjective city performance visualization question and answer method based on a large language model

Abstract

The application discloses a subjective city performance visualization question and answer method based on a large language model, which is characterized by using a large language model (LLM) to perform query rewriting, knowledge base retrieval and prompt integration on the natural language query of a user, effectively identifying the user's intention, combining semantic information and spatial distribution of points of interest (POIs), generating an accurate semantic density map through a semantic self-adaptive kernel density estimation algorithm, positioning a significant area based on a density-based spatial clustering algorithm, analyzing geographical relationships of different scales, and providing a detailed interactive visual answer. Compared with the prior art, the application solves the complex visualization automatic question and answer problem of city performance, enhances the adaptability of subjective city performance analysis to different scenarios through the large language model, and improves the accuracy and generalization ability of the analysis. The application can be widely applied to various practical application scenarios such as city site selection and travel planning, and has high practical value and good development prospect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of urban data analysis, and particularly to a subjective urban performance visualization question-answering method based on a large language model. BACKGROUND

[0002] Urban performance is an important indicator for measuring community well-being and sustainable development capacity, and subjective urban performance (SUP) is increasingly concerned as it evaluates the quality of life from the perspective of residents' subjective perception. However, the measurement of subjective urban performance faces a series of challenges. First, traditional methods of collecting residents' opinions through questionnaires and interviews are time-consuming and labor-intensive, and difficult to implement on a large scale. Second, analysis based on a single influencing factor often ignores the complexity of the urban environment and the diversity of residents' needs, leading to biased analysis results. In addition, existing methods lack natural language understanding capabilities and cannot adapt to specific scenarios of subjective differences.

[0003] Existing question-answering techniques mainly provide answers in text form and simple charts, and the visualization representation lacks interpretability and intuitiveness, making it difficult for the public to understand complex analysis results and professional terms. Although there have been advances in visual natural language interface technology, there is still insufficient research on processing geographic data for urban visualization. Meanwhile, traditional visualization question-answering systems rely on toolkits that are limited by rules and patterns, and large language models have advantages but face problems such as inability to update knowledge in real time, potential hallucinations, and providing incorrect answers. In view of this, the present application proposes a subjective urban performance (SUP) visualization question-answering method based on a large language model. SUMMARY

[0004] The present application aims to provide a subjective urban performance visualization question-answering method based on a large language model to solve the problems in the background art.

[0005] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0006] A subjective urban performance visualization question-answering method based on a large language model, which preprocesses user natural language text queries, uses a large language model to infer and generate definitions in specific scenarios of subjective urban performance, and generates density maps and locates significant areas based on geographic analysis to display answers in a visual form; specifically including the following steps:

[0007] S1, input the question q and the text instruction INST to generate the pseudo document d':

[0008] d' = LLM(q, INST)

[0009] S2, repeat the pseudo document d' with the original question q n times and connect them using the [SEP] separator to form a new question q + :

[0010] q + =concat({q}×n,[SEP],d')

[0011] S3. Slice and vectorize the collected domain knowledge into d-dimensional vectors v. k Stored as a knowledge base, and retrieved using similarity calculations and vectorized new questions v q The top-K most relevant knowledge fragments k i The similarity calculation expression is:

[0012] sim + ,k i >= <v q ,v k >

[0013] S4. Through the prompt word integration module, domain knowledge, pseudo-documents, and questions are integrated into a carefully designed prompt word template, which consists of the following parts:

[0014] ① Instructions: These outline the tasks that need to be completed and set the framework for the operation of the model.

[0015] ② Domain knowledge: Decomposing the subjective definition of urban performance requires following some basic urban performance norms. These norms are specific to a particular domain and involve retrieving relevant knowledge fragments from a knowledge base to provide the context and factual information needed by the model.

[0016] ③ Reasoning steps: Break down the task into several smaller, easier-to-handle tasks.

[0017] ④ Input / Output Format: Input consists of the user's original query and a pseudo-document. Output is specified in JSON format.

[0018] S5. In the geographic subjective city performance analysis module, collect Points of Interest (POIs) data in the city and apply the Semantic Adaptive Kernel Density Estimation (SAKDE) method to estimate the density of POI scatter points:

[0019]

[0020] Where f(x) is the estimated density value at point x, n is the total number of points of interest (POIs), and d(xx) i ) is x and x i The Euclidean distance between them h is the Gaussian kernel function a , which is adaptive bandwidth.

[0021] S6. Quantify the semantic contribution of Points of Interest (POIs) to the metric through semantic text similarity, and generate a class-level density map D.​t Point of Interest (POI) category p i and index m j The similarity is its embedding vector v pi v mj Cosine similarity:

[0022]

[0023] S7. Use semantic similarity normalized to a probability distribution to analyze class-level density maps. Perform a weighted average to generate an index-level density plot D. m Similarly, the index-level density map D is generated for the index. m The final answer density map is obtained by weighting and aggregating the indicators defined by Subjective Urban Performance (SUP). a .

[0024]

[0025] S8. Analyze the urban area using the density-based spatial clustering algorithm (DBSCAN). Divide the urban area into a 4200×4000 grid, with each grid representing one pixel block. Use the quantile method to select the top 10% of grid blocks that are strongly related to the problem and cluster them to locate significant areas.

[0026] S9. In the answer generation module, a rule-based tree-structured narrative template is used to organize the generated results according to three levels: answer overview, indicator checking, and point of interest (POI) category exploration, to generate the final visual answer. The narrative template consists of three parts:

[0027] ① Answer Overview: This provides a holistic perspective by combining density maps, masked areas, and information cards, based on the answer data generated after question analysis.

[0028] ② Indicator Check: Identify relevant evaluation indicators based on user query needs. Each indicator is displayed through visualization methods, such as using a heat map to represent population density or a bar chart to compare educational resources in different regions.

[0029] ③ Exploration of Points of Interest (POI) Categories: This section focuses on the impact of POIs on metrics, showcasing which types of locations have a significant influence on specific metrics through a categorization view.

[0030] Preferably, the visualization forms include heatmaps, masked positioning maps, and tree diagrams.

[0031] Preferably, the prompt word integration module in S4 contains instructions, domain knowledge, reasoning steps, input and output, and prompt word templates of their formats; the instructions are descriptive languages of tasks, the domain knowledge covers relevant articles, discussions, and documents of urban performance, including but not limited to text fragments of aspects such as infrastructure conditions, social service levels, and environmental quality, the reasoning steps are descriptive languages of analysis subtasks, and the input and output specify the input and output formats of the large language model.

[0032] Preferably, the geographic subjective urban performance analysis module in S5 generates a density heat map using the kernel density estimation method and locates the significant area using machine learning methods such as clustering methods based on the collected urban geographic data, including but not limited to point of interest (POI) information, urban infrastructure, air pollution quality, etc.

[0033] Preferably, the answer generation module in S9 generates a tree structure visualization narrative template based on rules to organize analysis results according to three levels of answer overview, indicator check, and point of interest (POI) category exploration; the root node of the tree structure is the answer overview level, the second level node is each indicator of the subjective urban performance generated by the large language model, and the leaf node is the point of interest (POI) category related to the parent node indicator.

[0034] Compared with the prior art, the present application provides a subjective urban performance visualization question and answer method based on a large language model, which has the following beneficial effects:

[0035] The present application solves the complex visualization automatic question and answer problem of urban performance, especially by using a large language model to decompose problem definition and using geographic analysis and machine learning technology to generate a visualization answer. In addition, the pre-processing step of the problem in the present application effectively solves the illusion problem and improves the accuracy of the answer, and the large language model enhances the adaptability of subjective urban performance analysis to different scenarios and improves the generalization ability of the analysis. The present application can be widely applied in urban construction, geographic site selection, travel planning and other practical application scenarios, and has high practical value and good development prospect. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 It is a flowchart of the subjective urban performance visualization question and answer method based on a large language model mentioned in embodiment 1 of the present application;

[0037] Figure 2 It is a principle diagram of the subjective urban performance visualization question and answer method mentioned in embodiment 1 of the present application;

[0038] Figure 3 It is an implementation example diagram of the geographic subjective urban performance analysis module mentioned in embodiment 1 of the present application;

[0039] Figure 4 The subjective urban performance visualization question and answer process diagram is taken as an example of the question and answer about exploring community vitality in embodiment 2 of the present application. DETAILED DESCRIPTION

[0040] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present application, and gives a detailed implementation and specific operation process, but the protection scope of the present application is not limited to the following embodiments.

[0041] The present application is a method for solving the question and answer about urban performance, and proposes a framework SUPQA (Subjective Urban Performance Question Answering) to answer the question about urban performance and automatically generate visual answers. First, the question is rewritten for query to generate a pseudo document, and then the relevant domain knowledge is retrieved from the pre-processed knowledge base as supplementary materials, and the subjective urban performance definition under the context of the question is inferred by using a large language model according to a prompt word template. Based on the definition, a multi-dimensional and multi-aspect density heat map is generated by using semantic adaptive kernel density estimation method and clustering technology. Finally, the final visual answer is generated according to the rule-based narrative template.

[0042] Embodiment 1:

[0043] Please refer to the accompanying Figure 1 , the present application generates answers to subjective urban performance questions in the following steps:

[0044] Step 1: input question q and collected multiple data;

[0045] Step 2: output large language model for q and text instruction INST, and construct pseudo document d';

[0046] Step 3: retrieve knowledge base, and construct knowledge fragment k i ;

[0047] Step 4: construct complete prompt P, input large language model, and infer subjective urban performance definition;

[0048] Step 5: generate density map D t , D m , D a by using SAKDE method for the definition;

[0049] Step 6: locate the significant area by using DBSCAN technology based on density map D t , D m , D a ;

[0050] Step 7: generate visual answer A sAnd output together with the literal answer.

[0051] In combination with the above, a subjective city performance visualization question and answer method based on a large language model is described in detail, which specifically includes the following content:

[0052] Please refer to Figures 1-2 The present application provides a subjective city performance visualization question and answer method based on a large language model, which preprocesses user natural language text queries, uses a large language model to infer and generate definitions of subjective city performance in specific scenarios, and generates density maps and locates significant areas based on geographic analysis to display answers in a visual form, including heat maps, mask positioning maps, and tree maps. The specific steps include:

[0053] S1, input question q and text instruction INST to generate pseudo document d':

[0054] d' = LLM(q, INST)

[0055] S2, concatenate the pseudo document d' with the original question q for n times and use the [SEP] separator to form a new question q + :

[0056] q + = concat({q}×n,[SEP],d')

[0057] S3, slice and vectorize the collected domain knowledge into a d-dimensional vector v k , store it as a knowledge base, and retrieve the TOP-K most relevant knowledge fragments k q from the vectorized new question v i by similarity calculation, and the similarity calculation expression is:

[0058] sim<q + ,k i > = <v q ,v k >

[0059] S4, through the prompt word integration module, integrate the domain knowledge, pseudo document and question in the carefully designed prompt word template, which is composed of the following parts:

[0060] ① Instruction: summarizes the content to be completed, setting the framework for the operation of the model.

[0061] ② Domain knowledge: decomposing the subjective city performance definition needs to follow some basic city performance specifications, which are specific to the field and are retrieved from the knowledge base to provide the context and factual information required by the model.

[0062] ③Reasoning steps: break down the task into smaller, more manageable subtasks.

[0063] ④Input and output formats: input consists of raw queries from users and pseudo-documents. Output is specified in JSON format.

[0064] The above prompt word integration module includes instructions, domain knowledge, reasoning steps, input and output, and prompt word templates of their formats; the instructions are descriptive languages for tasks, the domain knowledge covers relevant articles, discussions, and literature on urban performance, including but not limited to text segments on infrastructure conditions, social service levels, and environmental quality, the reasoning steps are descriptive languages for analyzing subtasks, and the input and output specify the input and output formats of large language models.

[0065] S5、Please refer to Figure 3 In the geographic subjective urban performance analysis module, the point of interest (POIs) data in the city is collected, and the semantic adaptive kernel density estimation (SAKDE) method is used to estimate the density of the point of interest (POI) scatter:

[0066]

[0067] where f(x) is the estimated density value of point x, n is the total number of points of interest (POIs), and d(x-x i ) is the Euclidean distance between x and x i , is the Gaussian kernel function h a , and a is the adaptive bandwidth.

[0068] The above geographic subjective urban performance analysis module generates a density heat map using the kernel density estimation method and locates the significant area using machine learning methods such as clustering methods based on the collected city geographic data, including but not limited to point of interest (POIs) information, urban infrastructure, air pollution quality, etc.

[0069] S6, quantify the semantic contribution of points of interest (POI) to indicators by semantic text similarity, generate class-level density map D t , the similarity between point of interest (POI) category p i and indicator m j is its embedding vector v pi , and the cosine similarity of v mj :

[0070]

[0071] S7, use the semantic similarity normalized to a probability distribution to weighted average the class-level density map to generate the indicator-level density map D mLikewise, the indicators are weighted aggregated to generate the final answer density map D m and the subjective urban performance (SUP) defined indicator weights. a .

[0072]

[0073] S8, analyze the urban area using a density-based spatial clustering algorithm (DBSCAN), divide the urban area into 4200x4000 grids, each grid represents a pixel block, and filter out the top 10% of grid blocks strongly related to the problem and cluster them to locate the significant area.

[0074] S9, in the answer generation module, use a rule-based tree structure narrative template to generate the final visual answer according to the three levels of answer overview, indicator check and point of interest (POI) category exploration. The narrative template includes three parts:

[0075] ① Answer overview: Provide an overall perspective by combining density map, mask area and information card based on the generated answer data after problem analysis.

[0076] ② Indicator check: Identify relevant evaluation indicators according to user query requirements. Each indicator is displayed through visualization, such as population density through heat map, or comparison of different areas of educational resources through bar chart.

[0077] ③ Point of interest (POI) category exploration: Focus on the impact of points of interest (POI) on indicators, and show which types of places have an important impact on specific indicators through classification view.

[0078] The above answer generation module organizes the analysis results based on rules according to the three levels of answer overview, indicator check and point of interest (POI) category exploration, and generates a tree structure visual narrative template; The root node of the tree structure is the answer overview level, the second level node is each indicator of the subjective urban performance generated by the large language model, and the leaf node is the point of interest (POI) category related to the parent node indicator.

[0079] Example 2:

[0080] Based on example 1 but with some differences, the following will further illustrate the invention with the question and answer of exploring community vitality as an example, which includes the following contents:

[0081] Step 1: input question q "I want to find a place suitable for family living, can you help me analyze the area with good schools, parks and community centers?" (as shown in Figure 4 A);

[0082] Step 2: output q and text instruction INST to large language model, construct pseudo document d' (as shown in Figure 4 E) ;

[0083] Step 3: retrieve knowledge base, construct knowledge fragment k i ; build complete prompt word P, input large language model, and infer to obtain subjective city performance definition. The definition under this embodiment consists of three indexes: economy, society, and infrastructure. The influence weight and influence factor of each index are also obtained (as shown in Figure 4 B and Figure 4 D) ;

[0084] Step 4: generate density map D t , D m , and D a using SAKDE method (as shown in Figure 4 C). Different density maps can be switched and viewed through interface navigation;

[0085] Step 5: based on density map D t , D m , and D a , use DBSCAN technology to locate significant areas (as shown in c4 area in Figure 4 ). The interface highlights the significant areas in the form of circular masks, and each area clearly defines the community service radius and center point;

[0086] Step 6: generate visual answer A s based on rule-based narrative template, and output together with the text answer.

[0087] The above is only the preferred specific implementation of the present application, but the protection scope of the present application is not limited to this. Any skilled person in the art, according to the technical solution and improvement concept of the present application, makes equivalent replacement or change within the technical range disclosed by the present application, which should be covered in the protection scope of the present application.

Claims

1. A subjective urban performance visualization question-answering method based on a large language model, characterized in that, By preprocessing user natural language text queries, a large language model is used to infer and generate a definition of subjective urban performance under specific scenarios. Based on geographic analysis, a density map and salient areas are generated and the answer is presented in a visual form. Specifically, the following steps are included: S1, Input Problem q Use the INST text command to generate pseudo-documents d ': d ' = LLM( q , INST) S2, The pseudo-document d 'With the original problem q repeat n This time, using the [SEP] delimiter to connect creates a new problem. q + : q + = concat({ q }× n ,[SEP], d ') S3. The collected domain knowledge is sliced ​​and vectorized to obtain a d-dimensional vector. v k Stored as a knowledge base, and retrieved using similarity calculations and vectorized new questions. v q The top-K most relevant knowledge fragments k i The similarity calculation expression is: try< q + , k i >=< v q , v k > S4. Through the prompt word integration module, domain knowledge, pseudo-documents, and questions are integrated into a carefully designed prompt word template, which consists of the following parts: ① Instructions: Used to outline what needs to be accomplished and to set the framework for the model's operation; ② Domain knowledge: Decompose the urban performance norms followed by the subjective definition of urban performance, and retrieve relevant knowledge fragments from the knowledge base to provide the context and factual information needed by the model; ③ Reasoning steps: Break down the task into several smaller, easier-to-handle tasks; ④ Input / output format: Input consists of the user's original query and pseudo-document; output is specified in JSON format; S5. In the geographic subjective city performance analysis module, collect interest point data in the city and apply the semantic adaptive kernel density estimation method to estimate the density of interest points: in, f ( x ) represents a point x The estimated density value at that location; n This represents the total number of points of interest data. d ( x - x i )express x and x i The Euclidean distance between them; The Gaussian kernel function; h a Indicates adaptive bandwidth; S6. Quantify the semantic contribution of interest points to the index through semantic text similarity, and generate a class-level density map. D t Points of interest categories p i and indicators m j The similarity is its embedding vector v pi , v mj The expression for calculating cosine similarity is: S7. Use semantic similarity normalized to a probability distribution to analyze class-level density maps. D t Perform a weighted average to generate an index-level density plot. D m Similarly, for generating index-level density maps D m The final answer density map is obtained by weighting and aggregating the index weights defined by subjective city performance. D a : S8. Analyze the urban area using a density-based spatial clustering algorithm. Divide the urban area into a 4200×4000 grid, with each grid representing one pixel block. Use the quantile method to select the top 10% of grid blocks that are relevant to the problem and cluster them to locate salient areas. S9. In the answer generation module, a rule-based tree-structured narrative template is used to organize the generated results according to three levels: answer overview, indicator check, and interest point category exploration, in order to generate the final visual answer; the narrative template includes: ① Answer Overview: This provides a holistic perspective by combining density maps, masked areas, and information cards, based on the answer data generated after question analysis; ② Indicator Check: Identify relevant evaluation indicators based on user query needs; each indicator is displayed through visualization. ③ Exploration of Points of Interest Categories: Focus on the impact of points of interest on indicators, and use a category view to show the types of locations that affect specific indicators.

2. The subjective city performance question-answering method based on a large language model according to claim 1, characterized in that, The visualization formats include heatmaps, masked positioning maps, and tree diagrams.

3. The subjective city performance question-answering method based on a large language model according to claim 1, characterized in that, The prompt word integration module described in S4 includes prompt word templates for instructions, domain knowledge, reasoning steps, inputs and outputs, and their formats. Among them, instructions are descriptive language for the task; domain knowledge refers to relevant articles, discussions, and literature on urban performance, including text fragments on infrastructure status, social service levels, and environmental quality; reasoning steps are descriptive language for analyzing sub-tasks; and inputs and outputs are used to specify the input and output formats of the large language model.

4. The subjective city performance question-answering method based on a large language model according to claim 1, characterized in that, The geographic subjective urban performance analysis module described in S5 uses the collected urban geographic data to generate a density heat map using the kernel density estimation method, and uses clustering methods to locate significant areas. The urban geographic data includes point of interest information, urban infrastructure, and air pollution quality.

5. The subjective city performance question-answering method based on a large language model according to claim 1, characterized in that, The answer generation module described in S9 generates a tree-structured visual narrative template based on rule-based analysis results, according to three levels: answer overview, indicator check, and interest point category exploration. The root node of the tree structure is the answer overview level, the second-level nodes are the various indicators of subjective city performance generated by the large language model, and the leaf nodes are the interest point categories related to the parent node indicators.

Images