Community updating public opinion collection method based on knowledge graph and multi-modal large model

Abstract

The invention discloses a community updating public opinion collection method based on a knowledge graph and a multi-mode large model, and belongs to the field of urban planning. The method comprises the following steps: constructing a community updating knowledge graph based on various elements, attributes and relationships of urban community updating; entities related to community updating and user opinions in a user language input text are extracted through a big language model, and interested entities in a community planning effect picture selected by a user through a mouse are extracted through a visual big model; querying entities or attributes related to the entities, and giving feedback according to a query result and user input; the user performs multiple rounds of interaction according to the feedback so as to perfect and confirm user suggestions; and generating a community updating effect picture by utilizing the image local editing capability of the visual large model and combining with user opinions. According to the method, accurate, efficient and interactive collection of public opinions can be realized by combining a knowledge graph and a large model technology, and the method has relatively high adaptability and expansibility.

Description

Technical Field

[0001] The present invention belongs to the field of urban planning, and specifically relates to a method for collecting public opinions on community renewal based on a knowledge graph and a multimodal big model. The method is characterized in that the knowledge graph and the multimodal big model are combined to realize interactive and accurate collection of public opinions. Background Art

[0002] Community renewal is a common governance and improvement method in urban planning in recent years. Its main purpose is to improve the quality of life of residents and meet diverse community needs through subtle adjustments and improvements to the local environment of the community. Under the guidance of humanistic values, the importance of public participation in community renewal design has become increasingly prominent. The collection and feedback mechanism of public opinions is a very critical link in community renewal. Public opinions can not only reflect the views of community residents on the existing environment, but also provide direction for subsequent planning and design. However, the current way of collecting public opinions faces some problems and challenges.

[0003] In the collection of public opinions, question-and-answer interaction is a very important method. It can dynamically obtain residents' views and needs on community environment, facilities, services, etc. through direct dialogue between people and the system. Compared with traditional questionnaires and public meetings, question-and-answer interaction has the advantages of strong flexibility, immediate feedback, and in-depth content. Through question-and-answer interaction, the system can continuously adjust the direction of questions according to the specific answers of residents, deeply explore the real needs of the public, and avoid the singleness and superficiality of information. In addition, question-and-answer interaction can also organize and supplement the vague and scattered opinions of residents, effectively improving the breadth and depth of public opinion collection.

[0004] However, the current question-and-answer interaction method still has significant problems, which are mainly manifested in the following aspects: First, the existing question-and-answer system has limited ability to understand public opinions and has difficulty processing complex, unstructured natural language expressions, resulting in the inability to accurately analyze and feedback the detailed opinions of many residents; second, the interaction mode of the existing system is mostly a preset question-and-answer path, which lacks flexibility and cannot dynamically adjust questions based on the actual feedback of residents, resulting in insufficient depth of information collection; finally, the existing system lacks effective support from domain knowledge and cannot perform efficient semantic analysis and judgment on residents' opinions, which affects the accuracy and reliability of the collection results.

[0005] Based on this, the introduction of knowledge graphs and multimodal large models can effectively make up for the above shortcomings of existing question-and-answer interactions. Knowledge graphs provide systematic and structured knowledge support by constructing entities and their relationships in the fields of community environment, facilities, services, etc., which can enhance the system's ability to understand and analyze public opinions; multimodal large models have powerful natural language processing and image processing capabilities, etc., which can dynamically generate questions and deeply understand residents' unstructured answers, thereby achieving efficient question-and-answer interactions. This combination can not only improve the accuracy of public opinion collection, but also ensure the comprehensiveness and flexibility of information collection. Therefore, using the combination of knowledge graphs and multimodal large models to build an intelligent and interactive public opinion collection system is a necessary means to promote public participation and improve the efficiency of opinion collection in community renewal projects. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects in the prior art and provide a method for collecting public opinions on community updates based on knowledge graphs and multimodal large models, aiming to solve the problems of lack of flexibility, insufficient understanding and feedback of user opinions when collecting public opinions through interactive questions and answers in community updates, and to improve the public's participation in community updates and the efficiency and accuracy of opinion collection.

[0007] The specific technical solutions adopted by the present invention are as follows:

[0008] In the first aspect, the present invention provides a method for collecting public opinions on community updates based on a knowledge graph and a multimodal large model, which is as follows:

[0009] S1. Construct a knowledge graph of community renewal based on various elements, attributes and their relationships of urban community renewal;

[0010] S2. Based on the community update knowledge graph, a large language model is used to extract entities and user opinions related to community updates in the user's language input text, and a large visual model is used to extract entities of interest in the community planning renderings selected by the user with the mouse;

[0011] S3, in the community updated knowledge graph, query S2 to obtain entities or attributes related to the entity, and give feedback based on the query results and user input;

[0012] S4. The user conducts multiple rounds of interactions based on the feedback in S3 to improve and confirm the user opinions in S2; the local image editing capability of the visual large model is used in combination with the improved and confirmed user opinions to generate a community update rendering; if the community update rendering does not meet the user requirements, iteratively execute S2 to S4 until the improved and confirmed user opinions are met.

[0013] Preferably, in S1,

[0014] Elements include buildings and supporting facilities, roads and ancillary facilities, and living environments; attributes include name, location, area, color, material, slope, width, and pavement; relationships include spatial relationships, hierarchical relationships, and functional associations;

[0015] The construction of the community renewal knowledge graph is based on the "Urban Residential Area Planning and Design Standards", and its structured expression meets the requirements of urban planning and design;

[0016] The community updated knowledge graph is stored and managed using databases including the Neo4J graph database.

[0017] Preferably, in S2,

[0018] The large language model extracts entities and user opinions related to community updates in user language input text from the community update knowledge graph through a predefined prompt word template; the prompt word template includes model role definition, user input text, output format definition, domain knowledge dictionary and a few sample cases;

[0019] The output format is defined as: community update operations, user requirements for community updates, and entities related to community updates;

[0020] The domain knowledge dictionary is a dictionary consisting of community updated knowledge graph entities and attributes in S1;

[0021] The user opinion is in a specified format, including: update area, update action and update content; if the relevant field in the user opinion does not exist, it is a default value.

[0022] Preferably, in S2, the visual big model understands the community planning renderings through semantic segmentation, and the visual big model used for semantic segmentation is Dino V2;

[0023] The entity elements in the community planning renderings selected by the user are extracted based on their proportion in the selected area. If the proportion is greater than the preset threshold T R Entity extraction is considered valid when .

[0024] Preferably, in S3,

[0025] Retrieving entities and their related attributes in the community updated knowledge graph through Cypher query language;

[0026] Based on the query results and user input, the large language model generates corresponding feedback through a predefined prompt word template; the prompt word template includes: model role definition, user input text, community updated knowledge graph query results, output format definition and a few sample cases.

[0027] Preferably, in S4,

[0028] The large visual model used for local image editing is StabilityAI;

[0029] The visual big model generates a community update effect diagram by constructing a prompt word template; the prompt word template includes: an image to be edited, an editing area, an editing action and user needs.

[0030] In a second aspect, the present invention provides a community updated public opinion collection system based on a knowledge graph and a multimodal large model, which is used to implement the method as described in any one of the first aspects;

[0031] The system comprises:

[0032] The knowledge graph construction module is used to construct a community renewal knowledge graph based on various elements, attributes and their relationships of urban community renewal;

[0033] An information extraction module is used to extract the user's language input text through a large language model or to extract the image entity selected by the user through a large visual model;

[0034] The opinion analysis module is used to query the entities and their attributes related to user input in the community update knowledge graph, and combine the large language model to infer the user's real needs and generate feedback suggestions;

[0035] The visual effect graph generation module is used to generate a community update effect graph based on the confirmed user opinions, and perform multiple rounds of interaction and iteration based on user feedback suggestions until the user opinion requirements are met.

[0036] In a third aspect, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can implement the community-updated public opinion collection method based on the knowledge graph and the multimodal large model as described in any one of the first aspects.

[0037] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for collecting public opinions on community updates based on a knowledge graph and a multimodal large model is implemented as described in any one of the first aspects.

[0038] In a fifth aspect, the present invention provides a computer electronic device, comprising a memory and a processor;

[0039] The memory is used to store computer programs;

[0040] The processor is used to implement the community updated public opinion collection method based on knowledge graph and multimodal large model as described in any one of the first aspects when executing the computer program.

[0041] In order to achieve efficient and accurate collection of public opinions in community updates, the present invention combines the advantages of community update knowledge graph and multimodal large model, and provides a dynamic and interactive public opinion collection method. Compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) Flexible modeling of community update knowledge graph: By constructing a flexible and extensible community update knowledge graph, the elements related to community update are structured, so that the system can accurately capture and analyze different types of community elements in user opinions, adapt well to the diverse user needs in the community update process, and facilitate system expansion and automation.

[0043] (2) Accurate processing capabilities of multimodal large models: The present invention uses multimodal large models (including large language models and large visual models) to achieve efficient collection of public opinions. The large language model can accurately extract and understand the natural language information input by users, and the large visual model can effectively process the image information in the community update renderings. In this way, it can ensure that the collection of public opinions is comprehensive, especially in the scenario of multimodal information input, to ensure the accuracy and efficiency of information processing.

[0044] (3) Dynamic question-and-answer interaction: By using a multi-round interactive question-and-answer system, the present invention can dynamically adjust the direction of questions and provide real-time feedback, effectively improving the interactivity and flexibility of public opinion collection. This method can deeply explore public needs, avoid the limitations of traditional fixed question-and-answer paths, and make the collected opinions more real and rich.

[0045] (4) Combining multiple rounds of interaction with rendering generation: By combining the visual macro model, the present invention can generate renderings of community updates based on multiple rounds of user interaction opinions, and make adjustments based on user feedback until a final rendering that meets the requirements is generated. This approach ensures real-time reflection and visual expression of user needs, greatly improving the efficiency of user experience and design feedback.

[0046] The present invention constructs an intelligent and flexible public opinion collection system through the organic combination of knowledge graph and multimodal large model. The system can realize the accurate and efficient collection of public opinions, and provides important technical support for the smooth implementation of community renewal projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of the process of the present invention;

[0048] Figure 2 A local example of the community update knowledge graph constructed for the present invention;

[0049] Figure 3 It is the semantic segmentation result map; among them, (a) is the original image, and (b) is the semantic segmentation map. DETAILED DESCRIPTION

[0050] The present invention is further described and illustrated below in conjunction with the accompanying drawings and specific embodiments. The technical features of each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0051] like Figure 1 As shown, a method for collecting public opinions on community updates based on a knowledge graph and a multimodal big model provided by the present invention comprises: first, constructing a knowledge graph related to community updates to express elements within the community such as buildings, roads, green spaces and other elements and their relationships; then, extracting entities related to community updates from the user's natural language input using a big language model, or extracting entities from the image selected by the user through a big visual model; then, querying the extracted entities and their related attributes in the knowledge graph, and analyzing the user's true intentions in combination with the big language model to provide feedback or suggestions; finally, generating a community update effect map for user reference through a big visual model based on the user's confirmed opinions. If the effect does not meet the user's needs, the system supports the user to conduct multiple rounds of interaction and iteration until an update plan that meets the user's requirements is generated.

[0052] The method for collecting public opinions on community updates of the present invention will be described in detail below.

[0053] Step S1: Based on various elements, attributes and their relationships of urban community renewal, a community renewal knowledge graph is constructed, such as Figure 2 shown.

[0054] As a preferred embodiment of the present invention, in this step, the elements of the community renewal knowledge graph nodes include three categories: buildings and supporting facilities, roads and ancillary facilities, and living environment, which can fully express the core elements of community renewal. The attributes of the community renewal knowledge graph nodes include name, location, area, color, material, slope, width, and pavement. The relationships between the edges of the community renewal knowledge graph include spatial relationships, hierarchical relationships, and functional associations. The combination of these elements can comprehensively describe the physical and social elements in the community renewal process, and provide support for knowledge association queries in subsequent opinion collection.

[0055] In addition, the construction of the community renewal knowledge graph is based on the "Urban Residential Area Planning and Design Standards" to ensure that its structured expression meets the requirements of urban planning and design. In other words, the construction of the community renewal knowledge graph is based on the "Urban Residential Area Planning and Design Standards" to ensure that the covered community renewal elements meet the specifications and standards of urban planning and design. The community renewal knowledge graph can be expanded according to actual needs, further introducing dynamically updated regional data and municipal information to enhance the adaptability of the graph. The community renewal knowledge graph can be stored and managed using a graph database, including but not limited to Neo4J and other graph databases.

[0056] S2. Based on the community update knowledge graph in S1, a large language model is used to extract entities and user opinions related to community updates in the user's language input text, and a large visual model is used to extract entities of interest in the community planning renderings selected by the user with the mouse.

[0057] As a preferred embodiment of the present invention, in this step, to ensure the extraction of key information in the user text input, the large language model used extracts the entities and user opinions related to the community update in the user language input text from the community update knowledge graph through a predefined prompt word template. The prompt word template includes the following contents:

[0058] 1) Model role definition: limit the context and interpretation method of the large language model.

[0059] 2) User input text: community update requirements entered by users through the interactive interface.

[0060] 3) Output format definition: Specify the structure of the feedback content to ensure that the generated information meets the user's understanding needs.

[0061] Specifically, the output format in the large language model prompt word template is defined as: community update operation, user needs of community update and entities related to community update. Community update operation and community update needs will be combined to form community update opinions.

[0062] 4) Domain knowledge dictionary: Combined with the entities, attributes and relationships in the community updated knowledge graph as a reference for language model understanding.

[0063] Specifically, the domain knowledge dictionary in the large language model prompt word template is a dictionary consisting of the community updated knowledge graph entities and attributes in step S1. This design ensures that when the large language model parses user input, it can automatically match the keywords mentioned by the user with the relevant entities and relationships in the community updated knowledge graph, thereby improving the accuracy and efficiency of opinion collection.

[0064] 5) Few-sample cases: used to guide the large language model to understand and process the few-sample example data input by the user, where the few-sample cases are specific cases related to community updates.

[0065] Through the above templates, the large language model can efficiently extract the specific needs of users and match them with relevant entities in the community updated knowledge graph.

[0066] As a preferred embodiment of the present invention, in this step, the generated user opinion is in a prescribed format, including: update area (clearly indicating the part the user wants to modify), update action (specific modification operation, such as "add" or "modify") and update content (specific opinion details, such as "use permeable bricks for paving"); if the relevant field does not exist, it is a default value.

[0067] As a preferred embodiment of the present invention, in this step, the visual big model understands the community planning effect map through semantic segmentation, such as Figure 3 As shown, the visual large model used for semantic segmentation includes but is not limited to Dino V2. The entity elements in the community planning renderings selected by the user are extracted based on their proportion in the selected area. If the proportion is greater than the preset threshold T R Entity extraction is considered valid when .

[0068] That is to say, in order to effectively extract the entities in the community planning renderings selected by the user, it is preferred to segment the community planning renderings through a large visual model (including but not limited to DinoV2), match the segmented entities with the user interaction area (mouse selection area), and combine the proportion R of entity i in the selection area to extract the entities in the community planning renderings. i , determine the user's focus. Specifically, to ensure the accuracy of the user's selection of entities, a threshold T needs to be set R , when R i >TR, the entity is considered as a valid choice of the user. This step can effectively filter out one or more community update elements that the user cares about most, and improve the accuracy of information collection. In addition, the user selects the area [x min ,y min ,x max ,y max ] will be integrated into the community update opinions formed during the text interaction phase as an update area.

[0069] S3. In the community update knowledge graph of S1, query the entities or attributes related to the entities obtained in S2 (including using the large language model to extract entities related to community updates in the user's language input text and using the large visual model to extract entities of interest in the community planning renderings selected by the user with the mouse), and give feedback based on the query results and user input.

[0070] As a preferred embodiment of the present invention, in this step, in order to better understand the user's true intention, the Cypher query language is used to retrieve entities and their related attributes in the community update knowledge graph of S1. The system will use the large language model to generate corresponding feedback through the predefined prompt word template based on the user input and the community update knowledge graph query results. Among them, the prompt word template includes the following content:

[0071] 1) Model role definition: limit the context and interpretation method of the large language model.

[0072] 2) User input text: community update requirements entered by users through the interactive interface.

[0073] 3) Community updates knowledge graph query results: Based on the entities extracted in step S2, the relevant entities and attributes in the knowledge graph are obtained through the Cypher query language. This language is designed specifically for knowledge graph queries and can efficiently find the entities extracted from user input and their associated attributes and relationships to ensure the accuracy of the query results.

[0074] 4) Output format definition: Specify the structure of the feedback content to ensure that the generated information meets the user's understanding needs.

[0075] 5) Few-shot examples: Few-shot examples used to guide large language models to understand and process user input.

[0076] Through the above-mentioned prompt word template, the system can accurately generate feedback based on user needs, making the process of collecting public opinions more interactive and personalized.

[0077] In actual use, users can conduct multiple rounds of interactions by clicking on feedback suggestions or re-entering text or using the mouse to select areas of interest in the community planning renderings. During the interaction process, feedback will be continuously provided and community update opinions will be updated.

[0078] S4. Users conduct multiple rounds of interactions based on the feedback in S3, continuously improve the user opinions in S2, and confirm the final user opinions. The local image editing capability of the visual large model is used to generate a community update rendering in combination with the improved and confirmed user opinions. If the generated community update rendering does not meet user requirements, iteratively execute S2 to S4 until the improved and confirmed user opinions are obtained.

[0079] As a preferred embodiment of the present invention, in this step, the visual big model generates a community update effect map through a local image editing function, and the visual big model used for local image editing includes but is not limited to StabilityAI.

[0080] As a preferred embodiment of the present invention, in this step, the visual big model generates a community update effect diagram by constructing a prompt word template, wherein the prompt word template includes:

[0081] 1) Image to be edited: the path and current status of the community update rendering.

[0082] 2) Edit area: the area that the user wishes to modify.

[0083] 3) Editing action: specific operation, such as "add" or "delete".

[0084] 4) User needs: used to determine the details of the renewal plan, such as the building materials used, green area, etc.

[0085] In this step, users can continue to adjust the community update opinions through multiple rounds of interaction, and generate updated renderings through the visual big model until the user's final needs are met.

[0086] When calling the visual big model for processing, the validity of the user's update opinion must also be verified. An effective community update opinion must contain three elements: update area (clearly define the part the user wants to modify), update action (specific modification operation, such as "add" or "modify"), and update content (specific opinion details, such as "use permeable bricks for paving"). After these contents are converted through the prompt word template, they can be used as input for the visual big model to ensure that the generated renderings can meet user requirements.

[0087] The present invention also provides a community update public opinion collection system based on knowledge graph and multimodal large model, which is used to implement the above-mentioned community update public opinion collection method based on knowledge graph and multimodal large model; specifically, the system includes the following modules:

[0088] 1) Knowledge graph construction module, which is used to construct a community renewal knowledge graph based on various elements, attributes and their relationships of urban community renewal;

[0089] 2) an information extraction module, which is used to extract the user's language input text through a large language model or to extract the image entity selected by the user through a large visual model;

[0090] 3) Opinion analysis module, which is used to query the entities and their attributes related to user input in the community update knowledge graph, and combine the large language model to infer the user's real needs and generate feedback suggestions;

[0091] 4) A visual effect diagram generation module, which is used to generate a community update effect diagram based on the confirmed user opinions, and perform multiple rounds of interaction and iteration based on user feedback suggestions until the user opinion requirements are met.

[0092] The method for collecting public opinions on community updates of the present invention will be specifically described below through examples.

[0093] Example

[0094] This embodiment provides a method for collecting public opinions on community updates based on a knowledge graph and a multimodal large model, which is as follows:

[0095] Step S1: Construct a knowledge graph related to community renewal based on various elements, attributes and relationships between various elements of urban community renewal. See the attached graph for an example. Figure 1 .

[0096] This embodiment first constructs a knowledge graph related to community micro-renewal to achieve structured management of community renewal elements, and interacts with the domain knowledge auxiliary large model of community renewal. Nodes include three categories: buildings and supporting facilities, roads and ancillary facilities, and living environment elements, which can fully express the core elements of community renewal. Node attributes include name, location, area, color, material, slope, width, pavement, etc. Relationships include spatial relationships, hierarchical relationships, functional associations, etc., which are used to describe the connection and interaction between different elements in the community.

[0097] In the knowledge graph, the attributes of a building node can include its name, location, building type, material, etc.; the attributes of a road node can include width, material, etc. The construction process of the knowledge graph can be expanded based on the "Urban Residential Area Planning and Design Standards" to ensure that it adapts to the needs of different community renewal projects.

[0098] The constructed knowledge graph is stored and managed using a graph database (Neo4J).

[0099] Step S2: Use the large language model to extract entities and user opinions related to community updates in the user's language input text; use the large visual model to extract entities in the community planning renderings selected by the user with the mouse.

[0100] This embodiment supports users to input text to describe their opinions on community updates; this embodiment also supports users to determine the community update area by selecting an area in the community update effect diagram by using a mouse.

[0101] When users interact through text input, the big language model (GPT) automatically extracts key information about community updates and community update opinions from the user's text input based on the prompt word template. The prompt word template defines the role, input text, output form, domain knowledge, and few-shot examples of the big language model. The domain knowledge of the prompt word template is the dictionary constructed by the knowledge graph entities and attributes in step S1. The output form of the prompt word template is: community update operations, community update-related entities, and user update requirements.

[0102] The large language model, combined with the dictionary in the knowledge graph, can automatically identify the keywords entered by the user, match the relevant community update elements, and give the corresponding community update operations and user update requirements. For example, if the user enters "I hope to increase the green area in the community", the large language model will automatically identify the key entity "green space", identify the "increase" update operation, and identify the "increase green area" user requirement. The large language model will output: update operation [increase], update related entity [green space], user demand [increase green area]. In addition, the system will record the user's update opinion as: update operation [increase], user demand [increase green area].

[0103] When the user interacts with the image, that is, uses the mouse to select a certain area in the community update rendering, the visual model (DinoV2) will first segment the community update rendering. The semantic segmentation results are as follows: Figure 3 As shown; secondly, extract the specific entities in the image area selected by the user. For example, when the user selects the road part, the visual model will extract the "road" entity and judge its importance based on its proportion in the selected area. The extracted entity needs to meet a certain ratio threshold T R , to ensure the accuracy of extraction.

[0104] For the selected entity, the user clicks on the selected area text interaction, the system will convert it into text input and switch to text interaction, and give feedback in step S3. In addition, based on the area selected by the user, the system will update the user's update opinion as follows: Update area [x min ,y min ,x max ,y max ], Update operation [Increase], User demand [Increase green area], [x min ,y min ,x max ,y max ] indicates the area selected by the user.

[0105] Step S3: Query step S2 in the knowledge graph to obtain entities or attributes related to the entity, and give feedback based on the query results and user input. The feedback is used to further improve user opinions.

[0106] After extracting relevant entities from user opinions, the system performs in-depth analysis through the knowledge graph. This embodiment uses the Cypher query language to retrieve entities and their attributes related to user opinions in the knowledge graph. Through this process, the system can accurately understand the user's needs and generate feedback based on existing data.

[0107] For example, if the user wants to increase the width of a road, the system will query the knowledge graph for information about the current width of the road, paving materials, and other information, and generate suggestions through the large language model. The large language model will combine user input and attributes in the graph to generate update plans, such as "In which part of the community do you want to increase the green area?", "How much green area do you want to increase?", "For the increased green area, are there any special design requirements or facility requirements." This process is implemented based on a prompt word template, which includes large model role definition, user input, knowledge graph query results, output format definition, and a few sample examples to ensure the accuracy of feedback.

[0108] Step S4: After multiple rounds of interaction based on the suggestions, the user confirms his or her opinion on the community update, and a rendering of the community update is generated by combining the local image editing capability of the visual large model with his or her community update opinion. If the effect does not meet the user's requirements, steps 2, 3, and 4 are iterated until a community update opinion that meets the user's requirements is obtained.

[0109] After the public opinions are confirmed, the system generates an updated community rendering through the visual big model. The visual big model (StabilityAI) will adjust the community rendering according to the specific update requirements input by the user. The generated rendering shows the updated plan of the community, such as increasing the green area or adjusting the road layout.

[0110] If the user is not satisfied with the generated renderings, he can continue to modify his opinions, and the system will iteratively generate new renderings based on the new input. For example, if the user wants to further increase the green area, the system will adjust the renderings based on the user's new input until the user's needs are met. The prompt word template of the visual large model includes the image to be edited, the editing area, and the editing operation to ensure that the renderings generated in each iteration meet the user's specific requirements.

[0111] The present invention can realize accurate, efficient and interactive collection of public opinions by combining knowledge graph and big model technology, and has strong adaptability and scalability.

[0112] It should be understood that parts not elaborated in detail in this specification belong to the prior art.

[0113] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A method for collecting public opinions on community updates based on knowledge graph and multimodal large model, characterized in that: The details are as follows: S1. Construct a knowledge graph of community renewal based on various elements, attributes and their relationships of urban community renewal; S2. Based on the community update knowledge graph, a large language model is used to extract entities and user opinions related to community updates in the user's language input text, and a large visual model is used to extract entities of interest in the community planning renderings selected by the user with the mouse; S3, in the community updated knowledge graph, query S2 to obtain entities or attributes related to the entity, and give feedback based on the query results and user input; S4, the user conducts multiple rounds of interactions based on the feedback in S3 to improve and confirm the user opinions in S2; Using the local image editing capability of the visual big model, combined with the improved and confirmed user opinions, a community update effect map is generated; If the community update effect diagram does not meet the user's requirements, S2 to S4 are iteratively executed until the user's opinion is satisfied, improved and confirmed.

2. The method for collecting public opinions on community updates based on knowledge graph and multimodal large model according to claim 1 is characterized in that: In S1, Elements include buildings and supporting facilities, roads and ancillary facilities, and living environments; attributes include name, location, area, color, material, slope, width, and pavement; relationships include spatial relationships, hierarchical relationships, and functional associations; The construction of the community renewal knowledge graph is based on the "Urban Residential Area Planning and Design Standards", and its structured expression meets the requirements of urban planning and design; The community updated knowledge graph is stored and managed using databases including the Neo4J graph database.

3. The method for collecting public opinions on community updates based on knowledge graph and multimodal large model according to claim 1 is characterized in that: In S2, The large language model extracts entities and user opinions related to community updates in user language input text from the community update knowledge graph through a predefined prompt word template; the prompt word template includes model role definition, user input text, output format definition, domain knowledge dictionary and a few sample cases; The output format is defined as: community update operations, user requirements for community updates, and entities related to community updates; The domain knowledge dictionary is a dictionary consisting of community updated knowledge graph entities and attributes in S1; The user opinion is in a specified format, including: update area, update action and update content; if the relevant field in the user opinion does not exist, it is a default value.

4. The method for collecting public opinions on community updates based on knowledge graph and multimodal large model according to claim 1 is characterized in that: In S2, the visual big model understands the community planning renderings through semantic segmentation, and the visual big model used for semantic segmentation is Dino V2; The entity elements in the community planning renderings selected by the user are extracted based on their proportion in the selected area. If the proportion is greater than the preset threshold T R Entity extraction is considered valid when .

5. The method for collecting public opinions on community updates based on knowledge graph and multimodal large model according to claim 1 is characterized in that: In S3, Retrieving entities and their related attributes in the community updated knowledge graph through Cypher query language; Based on the query results and user input, the large language model generates corresponding feedback through predefined prompt word templates; The prompt word template includes: model role definition, user input text, community updated knowledge graph query results, output format definition and few sample cases.

6. The method for collecting public opinions on community updates based on knowledge graph and multimodal large model according to claim 1 is characterized in that: In S4, The large visual model used for local image editing is StabilityAI; The visual big model generates community update effect diagrams by constructing prompt word templates; The prompt word template includes: an image to be edited, an editing area, an editing action and user needs.

7. A community updated public opinion collection system based on knowledge graph and multimodal large model, characterized in that: Used to implement the method according to any one of claims 1 to 6; The system comprises: The knowledge graph construction module is used to construct a community renewal knowledge graph based on various elements, attributes and their relationships of urban community renewal; An information extraction module is used to extract the user's language input text through a large language model or to extract the image entity selected by the user through a large visual model; The opinion analysis module is used to query the entities and their attributes related to user input in the community update knowledge graph, and combine the large language model to infer the user's real needs and generate feedback suggestions; The visual effect graph generation module is used to generate a community update effect graph based on the confirmed user opinions, and perform multiple rounds of interaction and iteration based on user feedback suggestions until the user opinion requirements are met.

8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, it can implement the method for collecting public opinions on community updates based on a knowledge graph and a multimodal large model as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method for collecting public opinions on community updates based on the knowledge graph and the multimodal large model is implemented as described in any one of claims 1 to 6.

10. A computer electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the community updated public opinion collection method based on knowledge graph and multimodal large model as described in any one of claims 1 to 6 when executing the computer program.

Images