Index system creation method and device, equipment, medium and program product

Abstract

The embodiment of the invention provides an index system creation method and device, equipment, a medium and a program product. The method comprises the steps that if a query request requesting for creating a target index system is detected, a large language model is controlled to convert the query request into a semantic query request for a knowledge graph, and the target index system is an index system used for evaluating a target object; the control chart retrieval enhancement model queries a target community hit by the query request in a target knowledge graph; the target knowledge graph is a knowledge graph created according to a triple extracted from target knowledge data, the triple comprises entities, attributes of the entities and relationships among the entities, the target knowledge data is knowledge data containing a target index, and the target index is an index used for evaluating a target object; and controlling the large language model to generate a target index system according to the searched triple in the target community and the searched abstract of the target community.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a method, apparatus, device, medium, and program product for creating an indicator system. Background Art

[0002] With the acceleration of urbanization and the deep integration of information technology, smart cities have become the core path to solving urban governance problems and improving the level of public services. Building a scientific and comprehensive smart city evaluation index system is the key foundation for measuring the effectiveness of urban smart construction and guiding subsequent optimization directions.

[0003] Currently, the mainstream approach to constructing smart city evaluation index systems is based on theoretical research and case studies. Its implementation process is divided into two phases: the first phase is the construction of an index framework. This phase first identifies the core evaluation dimensions and indicators for smart cities based on existing urban science theories, relevant literature, and policy documents. The urban operation system is then broken down into a tree-like structure: "goal layer - criterion layer - indicator layer." This forms a basic index framework that intuitively reflects the relationships between various indicators. Subsequently, the definition, calculation method, and theoretical basis of each evaluation indicator are clarified by combining existing literature research findings with the actual operation experience of case cities. Finally, through verification methods such as expert scoring, indicators with low scores are eliminated based on their mean importance, ultimately finalizing the smart city evaluation index framework. The second phase is the weighting of measurement indicators. This phase selects alternative measurement indicators that meet the aforementioned index framework from public data sources. Using methods such as hierarchical analysis, chain-comparison scoring, and principal component analysis, the importance of the alternative measurement indicators is compared and analyzed to determine the relative weights of each indicator, ultimately forming a complete smart city evaluation index system.

[0004] The above-mentioned existing technical solutions have certain rationality and practicality. Their advantage lies in that they fully integrate the experience and wisdom of experts in the field of urban science, effectively avoiding the problem of the indicator system being divorced from the existing theoretical basis and the actual operation of the city; at the same time, through the combination of expert experience judgment and mathematical deduction methods, it can take into account the qualitative judgment and quantitative analysis needs in the process of indicator selection and empowerment, and to a certain extent ensure the scientific nature of the indicator system.

[0005] However, in actual application, existing technical solutions still have many technical problems that need to be solved, which seriously restrict the construction efficiency and application adaptability of the smart city evaluation index system: First, theoretical and literature research is labor-intensive and inefficient. Smart cities are a typical multidisciplinary field, encompassing multiple disciplines such as urban planning, computer science, public administration, and environmental science. The number of related theoretical literature and policy documents is vast and complex. Existing technical solutions primarily rely on manual work for literature reading, information extraction, and selection of alternative indicators. This not only consumes a significant amount of manpower and time, but also has extremely low efficiency. Furthermore, because the same indicator may be expressed in multiple forms in different documents and policies, a significant amount of manual verification and unification work is required, further increasing the workload.

[0006] Second, the indicator selection and weighting process relies on human judgment, which carries a significant risk of subjective bias. During the indicator selection phase, the selection of candidate indicators relies heavily on the researcher's personal experience and subjective judgment, which can easily lead to the omission of key indicators that appear frequently in the literature but are relatively implicit, affecting the comprehensiveness of the indicator system. During the indicator weighting phase, if the expert sample structure involved in scoring is simple (for example, it only includes scholars in technical fields and lacks groups such as government urban governance managers and citizen representatives), the weighting results are likely to be biased towards the needs of a particular field, out of touch with the actual livelihood and governance needs of smart city construction, and fail to objectively reflect the actual importance of each indicator.

[0007] Third, the indicator system is difficult to dynamically adjust and rapidly iterate. Smart city construction is influenced by policy trends, technological innovation (such as breakthroughs in artificial intelligence and the Internet of Things), and changes in urban development stages. Its evaluation requirements and core dimensions require timely updates, such as adding new evaluation objectives or dimensions like "smart emergency response" and "digital twin applications," or deleting outdated traditional indicators. However, existing technical solutions rely on manual indicator screening, framework adjustments, and weight recalculation, which cannot efficiently process large amounts of data. When the indicator system needs to be adjusted and updated, manual research and analysis throughout the entire process is often required. This results in a long iteration cycle and slow response time for the indicator system, making it difficult to adapt to the rapid development of smart cities.

[0008] In summary, the existing methods for constructing smart city evaluation index systems based on theoretical research and case experience have obvious deficiencies in efficiency, objectivity, and dynamic adaptability. There is an urgent need for a new construction solution that can solve the above technical problems, so as to improve the construction efficiency of the smart city evaluation index system, reduce subjective bias, and achieve dynamic and rapid iteration. Summary of the Invention

[0009] The purpose of the embodiments of the present application is to provide a method, apparatus, device, medium and program product for creating an indicator system, which can improve the efficiency of creating the indicator system to a certain extent.

[0010] A first aspect of an embodiment of the present application provides a method for creating an indicator system, the method comprising: If a query request for creating a target indicator system is detected, controlling the large language model to convert the query request into a semantic query request for the knowledge graph, wherein the target indicator system is an indicator system for evaluating the target object; The control graph retrieval enhancement model searches for a target community hit by the query request in a target knowledge graph; the target knowledge graph is a knowledge graph created based on triples extracted from target knowledge data, the triples including: entities, attributes of entities, and relationships between entities; the target knowledge data is knowledge data containing target indicators, and the target indicators are indicators used to evaluate the target object; The large language model is controlled to generate the target indicator system according to the triples in the found target community and the summary of the found target community.

[0011] A second aspect of an embodiment of the present application provides a device for creating an indicator system, the device comprising: A first conversion module is configured to, upon detecting a query request requesting the creation of a target indicator system, control the large language model to convert the query request into a semantic query request for the knowledge graph, wherein the target indicator system is an indicator system for evaluating a target object; A first query module is configured to control the graph retrieval enhancement model to query a target community hit by the query request in a target knowledge graph; the target knowledge graph is a knowledge graph created based on triples extracted from target knowledge data, the triples including: entities, attributes of entities, and relationships between entities; the target knowledge data is knowledge data containing target indicators, and the target indicators are indicators used to evaluate the target objects; The first generating module is used to control the large language model to generate the target indicator system according to the triples in the found target community and the summary of the found target community.

[0012] A third aspect of an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method for creating an indicator system as described in the first aspect are implemented.

[0013] A fourth aspect of an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method for creating an indicator system as described in the first aspect are implemented.

[0014] The fifth aspect of an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps of the method for creating an indicator system as described in the first aspect.

[0015] A sixth aspect of an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps of the method for creating an indicator system as described in the first aspect.

[0016] In an embodiment of the present application, if a query request for creating a target indicator system is detected, the large language model is controlled to convert the query request into a semantic query request for the knowledge graph: the large language model has a strong language understanding ability, and by converting the query request into a semantic query request for the knowledge graph, it can accurately understand the user's intention to create a target indicator system. In this way, the structured characteristics of the knowledge graph can be used to quickly locate and obtain relevant knowledge. Compared with the traditional method of manually reading and understanding documents and extracting indicators, the efficiency of information acquisition is greatly improved, and the problem of low efficiency of manually creating an indicator system is solved. The control graph retrieval enhancement model queries the target community hit by the query request in the target knowledge graph: the graph retrieval enhancement model performs retrieval based on the graph data structure, and can efficiently find the target community matching the query request in the huge knowledge network of the target knowledge graph. The target knowledge graph is created by triples extracted from the target knowledge data, covering a wealth of knowledge about the target indicator. This retrieval method can quickly focus on the knowledge community closely related to the creation of the target indicator system, further improving the pertinence and efficiency of information retrieval, avoiding the tedious process of manually screening massive knowledge data, and helping to speed up the creation of the indicator system. The large language model is controlled to generate the target indicator system based on the triples and summaries of the target communities found. The large language model combines the structured and refined information of the triples (including entities, entity attributes, and relationships between entities) and community summaries in the target communities, leveraging its powerful generation capabilities to quickly generate the target indicator system. This approach fully leverages the structured knowledge of the knowledge graph and the generation advantages of the large language model. Compared to relying on manual judgment to select and construct the indicator system, it not only improves generation efficiency but also reduces subjective bias to a certain extent, generating a more comprehensive and accurate target indicator system. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of a method for creating an indicator system according to an embodiment of the present application; Figure 2 This is a flowchart of the steps of the method for creating a smart city indicator system provided by an embodiment of the present application; Figure 3 The user enters the prompt word and optimizes the adjustment; Figure 4 It is the workflow of the smart city evaluation factor query system driven by GraphRAG and LLM; Figure 5 It is an example of a smart city evaluation index system generated through knowledge query; Figure 6 This is a schematic diagram of the structure of an indicator system creation device provided in an embodiment of the present application; Figure 7 This is a schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application; Figure 8 This is another hardware structure diagram of an electronic device implementing an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0019] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0020] The method, apparatus, device, storage medium, chip, and computer program product for creating an indicator system provided in the embodiments of the present application can effectively solve the above-mentioned technical problems. The following, in conjunction with the accompanying drawings, describes in detail the method, apparatus, device, storage medium, chip, and computer program product for creating an indicator system provided in the embodiments of the present application through specific embodiments and their application scenarios.

[0021] like Figure 1 As shown, Figure 1This is a flow chart of the method for creating an indicator system provided in the embodiment of the present application. The method for creating an indicator system can be applied to electronic devices, which can be computers, smart phones, tablet computers, wearable smart devices and other devices, or servers in distributed systems, cloud servers, intelligent cloud computing servers with artificial intelligence technology, or intelligent cloud hosts and other servers. It can also be applied to Figure 6 The creation device of the indicator system shown, Figure 7 or Figure 8 For details about the electronic equipment shown, please refer to the relevant instructions in the following text. Figure 1 The method for creating the indicator system includes the following steps S11 to S13: S11, if a query request for creating a target indicator system is detected, the large language model is controlled to convert the query request into a semantic query request for the knowledge graph, and the target indicator system is an indicator system for evaluating the target object.

[0022] S12, the control graph retrieval enhancement model queries the target community hit by the query request in the target knowledge graph; the target knowledge graph is a knowledge graph created based on triples extracted from the target knowledge data, and the triples include: entities, attributes of entities and relationships between entities, and the target knowledge data is knowledge data containing target indicators, and the target indicators are indicators used to evaluate the target object.

[0023] S13, controlling the large language model to generate the target indicator system according to the triples in the found target community and the summary of the found target community.

[0024] A "big language model" is a natural language processing model based on deep learning. It features a large number of model parameters, extensive training data, and powerful language understanding capabilities. In this application, the big language model is used to convert query requests into semantic queries on the knowledge graph and to generate a target indicator system based on the triples and summaries found in the target community. This leverages its powerful language understanding and generation capabilities to achieve efficient indicator system creation.

[0025] The "Graph Retrieval Enhancement Model" is a composite technology that combines graph retrieval and generation models. Its core logic is to provide more accurate and structured information support for the generation process through the retrieval capabilities of graph structures. In this application, the Graph Retrieval Enhancement Model is used to search the target community hit by the query request in the target knowledge graph. Leveraging its efficient graph-based retrieval capabilities, it quickly locates knowledge communities relevant to the creation of the target indicator system.

[0026] A knowledge graph is created from triples (entities, their attributes, and relationships) extracted from target knowledge data. It models data and its relationships in the form of a graph, with nodes representing entities and edges representing relationships. In this application, the knowledge graph serves as a vehicle for knowledge storage and query, providing a structured source of knowledge for the creation of an indicator system, enabling large language models and graph-enhanced retrieval models to retrieve the required information.

[0027] For example, assume that the target object is a smart city, and the target indicator system is a smart city evaluation indicator system. When a query request to create a smart city evaluation indicator system is detected, the large language model converts the request into a semantic query request for the smart city-related knowledge graph. The graph retrieval enhancement model queries the constructed smart city knowledge graph for the target communities hit by the request, such as communities related to smart city security, intelligent transportation, etc. The large language model then generates a smart city evaluation indicator system based on the triples in these target communities (such as the entity "monitoring equipment" in the security field, the attribute "coverage rate", the relationship "related to public safety protection", etc.) and the community summary. For example, specific indicators such as the coverage rate of monitoring equipment are generated as part of the smart city evaluation indicator system.

[0028] By converting query requests through a large language model, we can accurately understand user intentions, leverage the structured characteristics of knowledge graphs to quickly acquire relevant knowledge, improve information acquisition efficiency, and address the low efficiency of manually created indicator systems.

[0029] The graph retrieval enhancement model efficiently queries the target community in the knowledge graph, improves the targeted nature of information retrieval, avoids manual screening of massive knowledge data, and further accelerates the creation of the indicator system.

[0030] The large language model combines the target community information to generate an indicator system, reducing the subjective bias when manually constructing the indicator system, making the generated indicator system more comprehensive and accurate, and improving the quality of the indicator system.

[0031] In some optional implementations, after controlling the large language model to generate the target indicator system based on the triples in the found target community, the method further includes: If an evaluation attribute input by the user for the i-th indicator is received, the i-th indicator is corrected according to the evaluation attribute input by the user for the i-th indicator and the preset correction rule of the i-th indicator, where the i-th indicator is any indicator in the target indicator system.

[0032] The "i-th indicator" refers to any indicator in the target indicator system. In general indicator system construction scenarios, a complete indicator system often includes multiple indicators from different aspects, which together constitute the evaluation system for the target object. In this application, the i-th indicator is a specific indicator in the target indicator system. For example, in the smart city evaluation indicator system, the "intelligent monitoring and management rate of municipal pipelines" can be used as an example of the i-th indicator.

[0033] "Evaluation attributes" are some characteristic descriptions given by users for the i-th indicator, which are used to determine whether the indicator meets actual needs. Under normal circumstances, evaluation attributes may include aspects such as the availability and applicability of the indicator. In this application, evaluation attributes specifically refer to the evaluation information given by users in terms of the availability of indicators (such as whether they can be accurately obtained corresponding to existing statistical calibers, whether they can be obtained through surveys, etc.) and applicability (such as whether they comply with domestic policy orientations, whether they have evaluation value, etc.). For example, for the indicator "Intelligent Monitoring and Management Rate of Municipal Pipeline Networks", users can evaluate whether its data can be obtained from relevant government departments in terms of availability, and evaluate whether it complies with the policy orientation of smart city construction in terms of applicability.

[0034] "Correction rules" are a set of pre-set criteria for processing the i-th indicator based on the evaluation attributes. In general indicator system optimization scenarios, correction rules are used to ensure the scientificity and practicality of the indicator system. In this application, the correction rules decide whether to retain, merge, adjust or delete the indicators based on different combinations of indicator availability and applicability. For example, when the indicator availability is A (can be accurately obtained corresponding to the existing statistical caliber) and the applicability is a (fully in line with domestic policy orientation and has evaluation value), the correction method is "Aa complete retention".

[0035] For example, let's continue using the smart city evaluation index system. Assume that the i-th indicator is "Punctuality of Urban Public Transport." The user enters the following evaluation attributes for this indicator: Regarding availability, the relevant data is accurately available through the bus company's operational record system, resulting in an A rating. Regarding applicability, the indicator aligns with current policies for efficient smart city transportation operations and possesses significant evaluation value, resulting in an A rating. Based on the preset correction rules, the indicator is corrected to "Aa Full Retention," meaning that "Punctuality of Urban Public Transport" remains unchanged and remains in the smart city evaluation index system.

[0036] By obtaining the evaluation attributes input by the user for the i-th indicator, the indicator can be evaluated in combination with the actual situation, making the indicator system more suitable for actual application scenarios and enhancing the practicality of the indicator system.

[0037] The i-th indicator is corrected according to the preset correction rules, which ensures the scientificity and rationality of the indicator system, avoids the existence of inapplicable or difficult-to-obtain data indicators in the system, and improves the quality of the indicator system.

[0038] This indicator correction method based on user evaluation attributes and correction rules can perform targeted optimization on the automatically generated indicator system, make up for the possible deficiencies in the automatic generation process, and further improve the target indicator system.

[0039] In some optional implementations, after receiving the evaluation attribute input by the user for the i-th indicator, and correcting the i-th indicator according to the evaluation attribute input by the user for the i-th indicator and a preset correction rule for the i-th indicator, the method further includes: Obtain the Cronbach's alpha coefficient and item-level content validity index of the target indicator system; Verify the reliability and validity of the target indicator system based on the Cronbach's alpha coefficient and the item-level content validity index; If the verification is passed, the target indicator system is output.

[0040] The Cronbach's alpha coefficient is a statistic used to measure the internal consistency of an indicator system. In general research and evaluation, the Cronbach's alpha coefficient is often used to test the degree of consistency between multiple measurement items. In this application, the Cronbach's alpha coefficient of the target indicator system is calculated to determine whether the indicators within the system are well-consistent. If Cronbach's alpha is greater than 0.7, the internal consistency of the indicator system is good.

[0041] The "Item-Level Content Validity Index" is used to assess the content validity of each item in the indicator system. In related fields, it measures the correlation between a single indicator and the concept being measured. In this application, when the number of experts is ≥6, an I-CVI ≥ 0.78 indicates good item content validity. This index is used to determine whether each indicator is valid for the target assessment object.

[0042] "Reliability and validity verification" is the process of determining the scientific validity of the target indicator system through analysis of Cronbach's alpha coefficient and item-level content validity index. In general indicator system construction, reliability and validity verification is a key step in ensuring the quality of the indicator system. In this application, the target indicator system is only output after it has passed verification based on these two indices, ensuring that the final indicator system is reliable and scientific.

[0043] For example, let's assume the target indicator system is an evaluation system for a particular enterprise's digital transformation. To obtain the Cronbach's alpha coefficient and item-level content validity index for this indicator system, 10 experts in the field of enterprise digital transformation were invited to perform a 5-point Likert-scale quantitative rating of the indicator system. Subsequently, professional statistical software was used to calculate the Cronbach's alpha coefficient and item-level content validity index. If the calculated Cronbach's alpha coefficient is greater than 0.7 and the item-level content validity index is greater than 0.78, indicating that the indicator system has good internal consistency and good item content validity, the verification is passed. At this point, the evaluation indicator system for the enterprise's digital transformation level is output for subsequent evaluation of the enterprise's digital transformation level.

[0044] Obtaining the Cronbach's α coefficient and item-level content validity index of the target indicator system provides a quantitative basis for judging the scientificity and effectiveness of the indicator system, making the evaluation of the indicator system more objective and accurate.

[0045] By analyzing the Cronbach's α coefficient and the item-level content validity index to verify the reliability and validity of the target indicator system, it is possible to promptly discover possible problems in the indicator system, such as poor consistency between indicators or low validity of a single indicator, so as to optimize the indicator system and improve its quality.

[0046] The target indicator system is output only after verification, which ensures that the indicator system finally applied has high reliability and scientificity. It is of great significance for accurately evaluating the target object and avoids the erroneous evaluation results caused by the use of unreasonable indicator systems.

[0047] In some optional implementations, if a query request requesting the creation of a target indicator system is detected, before controlling the large language model to convert the query request into a semantic query request for the knowledge graph, the method further includes: Acquire target knowledge data; Controlling the text embedding model to convert the target knowledge data into a vector; If a user prompt word for extracting the target indicator is received, controlling the large language model to extract a triplet related to the user prompt word from the vector; Controlling the graph retrieval enhancement model to create the target knowledge graph according to the extracted triples related to the user prompt word; The graph retrieval enhancement model is controlled to perform community analysis on the knowledge graph to cluster at least one community from the target knowledge graph and generate a summary of each community in the at least one community.

[0048] "Target knowledge data" refers to knowledge data containing target indicators used to evaluate target objects. In various evaluation scenarios, target knowledge data is the fundamental information source for building the evaluation system. In this application, for example, when building a smart city evaluation indicator system, target knowledge data can include structured and unstructured text materials such as policy documents, research reports, planning documents, and academic literature in the field of smart city evaluation. These data contain various information related to smart city evaluation.

[0049] A "text embedding model" is a model that converts textual material into vectors. In the field of natural language processing, text embedding models can convert textual information into a vector form that computers can better process, facilitating subsequent retrieval, analysis, and other operations. In this application, a text embedding model (such as the gte-qwen2-7b-instruct model) converts the organized target knowledge data into vectors and stores them in a vector database, providing a suitable data format for subsequent large language model extraction.

[0050] "User prompts" are keywords or key phrases entered by the user to extract target indicators. During the interaction with the model, users provide prompts to guide the model to extract information based on their needs. In this application, users enter prompts related to smart city evaluation, such as "smart city security-related indicators," and the large language model extracts relevant triples from the vector data.

[0051] "Community analysis" is an operation that performs cluster analysis on knowledge graphs to cluster at least one community from the target knowledge graph and generate a summary of each community. In knowledge graph analysis, community analysis helps to rationally divide complex knowledge networks, making it easier to locate and understand relevant knowledge more accurately. In this application, the graph retrieval enhancement model uses a hierarchical clustering algorithm and community detection technology to perform community analysis on the knowledge graph. For example, in the smart city knowledge graph, communities related to different aspects of the smart city (such as safety, transportation, etc.) are clustered and community summaries are generated to provide a structured knowledge basis for the subsequent creation of an indicator system.

[0052] For example, if the target object is a smart factory, a smart factory evaluation index system needs to be constructed. First, target knowledge data related to smart factories is obtained, such as smart factory construction standard documents and production efficiency research reports. Then, a text embedding model is used to convert this data into vectors and store them in a vector database. Next, the user enters "smart factory production process optimization indicators" as the user prompt word, and the large language model extracts triples related to the prompt word from the vector, such as (smart factory production process, optimization direction, automated operation), etc. The graph retrieval enhancement model creates a target knowledge graph based on the extracted triples and performs community analysis on the knowledge graph, clustering communities such as production process optimization and equipment management communities, and generating summaries for each community. For example, the summary of the production process optimization community may be a summary of key information about the automation and intelligence of smart factory production processes.

[0053] Acquiring target knowledge data provides a rich information basis for building a target indicator system, enabling the indicator system to fully reflect the characteristics and evaluation needs of the target object.

[0054] The target knowledge data is converted into vectors through the text embedding model, which facilitates the subsequent information extraction of large language models and the construction and analysis of knowledge graphs, thereby improving the efficiency of data processing and information extraction.

[0055] User prompt words guide the large language model to extract specific relevant information, making the extraction process more targeted, and can accurately obtain knowledge closely related to the construction of the indicator system, thereby improving the accuracy of the indicator system construction.

[0056] The graph retrieval enhancement model performs community analysis on the knowledge graph and generates a community summary, structuring the complex knowledge graph to facilitate the subsequent rapid positioning and utilization of relevant knowledge to generate an indicator system, further improving the efficiency and quality of indicator system creation.

[0057] In some optional embodiments, the method further comprises: If the verification fails and a new user prompt word is received from the user, the method returns to the step of controlling the large language model to extract triples related to the user prompt word from the vector.

[0058] "Failed verification" means that when the reliability and validity of the target indicator system are verified, its Cronbach's alpha coefficient does not reach the standard of greater than 0.7, or the item-level content validity index does not meet the requirement of I-CVI ≥ 0.78 when the number of experts is ≥ 6, indicating that the indicator system has problems in internal consistency or item content validity and does not meet the scientific and effective standards. In the quality control link of indicator system construction, failed verification means that the current indicator system needs further optimization. In this application scenario, the reliability and validity of the target indicator system generated by a series of operations is verified and the conclusion that it does not meet the requirements is drawn.

[0059] "New user prompt words" are keywords or sentences that users re-enter after the indicator system fails verification to guide the large language model to extract information. In the process of interacting with the model to optimize the indicator system, users change the prompt words in the hope that the model can extract more appropriate information, thereby building a more scientific and effective indicator system. In this application, for example, after the smart city evaluation indicator system fails verification, the user may enter a more specific "quantifiable evaluation indicators in smart city ecological security protection" as a new user prompt word to guide the model to extract relevant information.

[0060] For example, let's assume a system is being constructed to evaluate the scientific research strength of universities. During reliability and validity testing of this system, the Cronbach's alpha coefficient was found to be 0.65, less than 0.7, and thus failed the validation. The user then believes that the previously entered prompt may have extracted incomplete and inaccurate information, so they enter a new prompt: "Evaluation indicators related to the transformation of university scientific research results." After receiving the new prompt, the system returns to the step of controlling the large language model to extract triples related to the new prompt from the vector. Based on the new prompt, the large language model extracts triples such as (university scientific research projects, transformation results, and market application value). Subsequently, a series of operations, including rebuilding the knowledge graph and conducting community analysis, are performed in the hope of generating a more scientific and effective system for evaluating the scientific research strength of universities.

[0061] When the verification fails, the operation of returning to re-extract information provides a way to optimize the indicator system. It can make adjustments to the problems existing in the indicator system, which helps to improve the quality of the indicator system and make it more in line with scientific and effective standards.

[0062] The input of new user prompt words gives users the ability to actively intervene in and optimize the indicator system construction process. Users can guide the model to extract information more accurately based on the verification results and their own understanding of the target object evaluation, so that the final generated indicator system is more in line with actual needs.

[0063] This cyclic optimization mechanism continuously adjusts the key information extraction process for the construction of the indicator system, which can gradually improve the indicator system, increase its accuracy and reliability in evaluating the target objects, and avoid using unreasonable indicator systems for evaluation.

[0064] In some optional embodiments, the method further comprises: Control the large language model to generate the definition, calculation method and extraction basis of each indicator in the target indicator system.

[0065] As mentioned above, the "big language model" is a natural language processing model based on deep learning, with powerful language understanding and generation capabilities. In this scenario, the big language model uses its capabilities to generate the definitions, calculation methods, and extraction basis for each indicator in the target indicator system based on information from the target community. For example, in the construction of a smart city evaluation indicator system, the big language model can generate an indicator such as "the proportion of days with good urban air quality" based on relevant information from the target community in the smart city knowledge graph. The definition of this indicator is the ratio of days with good urban air quality to the total number of days during the statistical period; the calculation method is to count the number of good days and divide it by the total number of days; the extraction basis may be knowledge related to the impact of air quality on the smart city's ecological environment.

[0066] "Indicator definition" is a clear explanation of the meaning of each indicator in the target indicator system. In various indicator systems, clearly defined indicators are the basis for accurate understanding and application of indicators. In this application, the indicator definition generated by the large language model can accurately describe the specific content represented by the indicator when evaluating the target object, so that users can clearly know the direction of indicator measurement. For example, for the "urban traffic congestion index", it is clearly defined as a quantitative value reflecting the degree of urban traffic congestion, which is calculated by a specific algorithm combined with factors such as traffic flow and vehicle speed.

[0067] The "calculation method" determines how to obtain the specific numerical value of an indicator. In the application of the indicator system, a unified and reasonable calculation method ensures the quantification and comparability of the indicator. In this application, the calculation method generated by the large language model provides users with an operational guide for obtaining indicator data. For example, the calculation method for "urban public transportation share rate" is to divide the urban public transportation trip volume by the total urban trip volume and multiply by 100%.

[0068] "Extraction basis" is to explain the reasons why the indicator is extracted from the knowledge data and the relevant knowledge support. When constructing an indicator system, clarifying the extraction basis can enhance the scientificity and rationality of the indicator. In this application, the extraction basis generated by the large language model can help users understand the intrinsic connection between the indicator and the target object evaluation. For example, the extraction basis of the "urban green space coverage rate" indicator may be the importance of green space to improving the urban ecological environment and improving the quality of life of residents. These bases come from knowledge data such as policy documents, research reports, etc. related to smart cities.

[0069] For example, let's consider the construction of an indicator system for evaluating the development level of rural e-commerce. When generating the indicator system, the large language model defines the "rural e-commerce sales growth rate" as an indicator reflecting the growth rate of rural e-commerce sales over a given period. The calculation method is (current period rural e-commerce sales - previous period rural e-commerce sales) ÷ previous period rural e-commerce sales × 100%. The analysis is based on the fact that rural e-commerce sales growth is a key factor in measuring the vitality and growth trends of rural e-commerce development. This information is derived from statistical reports on rural e-commerce development, industry research materials, and other knowledge and data.

[0070] The definition of indicators generated by the large language model makes the meaning of each indicator in the target indicator system clear and unambiguous, which helps users accurately understand the meaning of each indicator when evaluating the target object, avoids ambiguity in understanding the indicators, and improves the readability and ease of use of the indicator system.

[0071] The calculation method for generating each indicator provides a standardized operating method for obtaining the specific numerical value of the indicator, ensures the quantifiability and comparability of the indicator, and enables different entities to adopt a unified calculation method when applying the indicator system, thereby improving the accuracy and reliability of the indicator system in actual evaluation.

[0072] Clarifying the basis for extracting each indicator enhances the scientificity and rationality of the indicator system. Users can clearly understand the reasons why each indicator is included in the system and the relevant knowledge support, so that they can more confidently apply the indicator system to evaluate the target objects. It also facilitates further optimization and improvement of the indicator system.

[0073] The following is an explanation of how to create a smart city indicator system.

[0074] Unlike existing technologies, this invention does not construct an index system through manual research and expert scoring. It has the advantages of quickly and massively acquiring existing literature and policies and generating new information through reasoning. It is suitable for multi-objective and multi-dimensional smart city development status assessment. The invention is divided into the following steps: Figure 2 ): 1. Build a knowledge query system driven by a large language model Implementation steps: (1) Data collection and organization. Collect and organize existing policy documents, research reports, planning documents, academic literature, and other structured and unstructured text materials in the field of smart city evaluation as knowledge data for the large language model to learn and understand.

[0075] (2) Build a vector database. Use the text embedding model to convert the structured and unstructured text materials into vectors and store them in the vector database.

[0076] (3) Constructing a knowledge graph. The first step is to locally deploy the open-source Ollama large language model platform and GraphRAG, install a large language model that is good at Chinese natural language processing tasks (NLP Task) on the Ollama platform, and then associate the constructed vector database with GraphRAG. The second step is that users input prompts related to smart city evaluation indicators through natural language dialogue and continuously optimize and adjust them (e.g. Figure 3 As shown in the figure), the selected large language model will automatically extract all entities, relations and attributes related to the prompt word from the constructed vector database.

[0077] The third step is to deploy a graph database locally and associate it with GraphRAG. All the information extracted by the large language model is loaded into the graph database in the form of knowledge graph "triples" (entity, relationship, entity; entity, attribute, attribute value) as a knowledge source for subsequent question-answering queries (e.g. Figure 4 shown).

[0078] See Figure 4 Large Language Model (LLM): A general AI model with powerful text generation and comprehension capabilities, which is the core component of "generating the final response" here.

[0079] Response: The system's final answer to the user's query.

[0080] Query: A question raised by a user (e.g., “What are the evaluation indicators for smart city transportation?”).

[0081] AI Query System For Evaluation Indicators of Smart City: The core of the entire system, responsible for coordinating "query processing, knowledge retrieval, and model interaction."

[0082] Context: Domain-related information (such as indicator definitions, policy texts, and similar cases) retrieved from data sources to assist large language models in understanding queries.

[0083] Domain Adapted Prompt Tuning: Customize general queries for smart city evaluation, enabling large language models to more accurately understand specialized questions.

[0084] Retrieval: The process of "screening and searching for relevant content" from various data sources.

[0085] Smart City Evaluation Dataset: A collection of information required for evaluation, including two types of sub-data: Statistical Indicators: Quantitative data (e.g., “5G base station coverage”, “smart community penetration”).

[0086] Policies & Reports, Corpus: Unstructured text (e.g., smart city policy documents, research reports, news corpus).

[0087] Graph Database (Smart City Graph Database): A database that stores domain knowledge in an entity-relationship structure, containing three core elements: Entities: Key objects in a smart city (e.g., “smart transportation system,” “energy management platform,” “citizen users”).

[0088] Relationships: associations between entities (e.g., “transportation system → support → economic development”, “policy → guidance → indicator setting”).

[0089] Theme Clusters: Group entities and relationships by theme (e.g., "Public Service Theme Cluster" and "Infrastructure Theme Cluster").

[0090] Indexing Time: The pre-processing stage (non-real-time query stage) of the system "processing data sources and building search indexes".

[0091] Text Chunks: Split long text (e.g., policy reports) into small “snippets” (for easier embedding and retrieval).

[0092] Smart City Theme Vectors: These are high-dimensional numerical vectors of text / entities converted through embedding technology (semantically similar content has closer vectors for faster retrieval).

[0093] Embedding: The technology of converting non-numerical information such as text and entities into "numerical vectors" is the core of realizing "semantic retrieval".

[0094] Smart City Evaluation Elements: core objects of smart city evaluation (such as evaluation dimensions "economy, society, environment" and specific indicators "smart medical response time").

[0095] Figure 4 The system shown works as follows: Phase 1: Preprocessing (Indexing Time – Preparing for Search) The purpose is to "convert scattered domain knowledge into a 'quickly searchable' index" to support the efficiency and accuracy of subsequent queries.

[0096] Step 1: Split and vectorize “evaluation elements” Input: Smart City Evaluation Elements (core objects of smart city evaluation, such as indicators, dimensions, and cases).

[0097] Process 1: Split Smart City Evaluation Elements into Text Chunks (break long text into small segments, such as splitting a policy report into "General Principles," "Transportation," "Energy," and other segments).

[0098] Process 2: Use embedding technology to convert text chunks (or Smart City Evaluation Elements themselves) into Smart City Theme Vectors (converting text semantics into high-dimensional vectors. For example, the vectors of "smart transportation indicators" and "transportation system evaluation" will be very close).

[0099] Step 2: Associate the graph database with the vector index Smart City Theme Vectors are associated with the Smart City Graph Database (including Entities, Relationships, and Theme Clusters).

[0100] Effect: Both "structured entity-relationship knowledge" (such as "intelligent transportation system → association → congestion index indicator" in the database) and "unstructured text vector knowledge" (such as the vector of policy fragments) can provide support for subsequent retrieval.

[0101] Phase 2: Real-time query generation (user interaction and intelligent response) When a user initiates a query, the system outputs accurate domain responses through "retrieval enhancement + large language model generation".

[0102] Step 1: User initiates a query Users submit queries to the AI ​​Query System For Evaluation Indicators of Smart City (e.g., “How to evaluate the environmental sustainability of smart cities”).

[0103] Step 2: Domain Adaptation and Query Tuning Through Domain Adapted Prompt Tuning, the system converts "general natural language queries" into "professional prompts tailored to the field of smart city evaluation" (for example, converting "Is the environment good?" into "Please analyze the environmental sustainability evaluation dimensions based on smart city environmental statistical indicators and policy requirements").

[0104] Step 3: Multi-source retrieval to obtain context After tuning, the query obtains context from two data sources through the Retrieval module: Retrieve structured knowledge from the Smart City Graph Database (e.g., “environmental entity (wastewater treatment plant, photovoltaic power station) → relationship (support → ‘carbon neutrality indicator’) → topic cluster (environmental sustainability cluster)”).

[0105] Retrieve unstructured / quantitative knowledge from the Smart City Evaluation Dataset (e.g., "Statistical Indicators: Urban photovoltaic penetration rate 35% in 2020; original text snippet of 'Sewage treatment compliance rate must be ≥ 95%' in Policies & Reports").

[0106] Step 4: The large language model generates the final response The system passes the Query+Context (tuned query + retrieved domain knowledge) to the LargeLanguage Model.

[0107] The large language model combines "general language capabilities" and "domain context" to generate precise responses (for example, "The environmental sustainability of a smart city can be evaluated from three dimensions: 1. Energy structure (e.g., photovoltaic penetration rate of 35%...); 2. Pollution control (e.g., sewage treatment compliance rate of 96%, in line with the requirements of the "... Policy"...); 3. Ecological synergy (e.g., coverage of the park smart management system...)").

[0108] Step 5: Respond to the user The response generated by the Large Language Model is ultimately fed back to the user through the AI ​​Query System.

[0109] (4) Knowledge graph analysis. The constructed knowledge graph is analyzed using the hierarchical clustering algorithm and community detection technology in GraphRAG, and the community structure is automatically clustered. These communities are divided based on the similarity or closeness of relationships between entities. Each community identified by the algorithm automatically generates a corresponding abstract to explain the community. When a user submits a question-and-answer query, the generated community abstract is prioritized as background knowledge and provided to the selected large language model, enabling a better comprehensive understanding of the multiple text materials in the vector database and analytical responses.

[0110] Through the above steps, we can leverage the learning and reasoning capabilities of large language models to quickly understand knowledge information across multiple text datasets and generate more valuable comprehensive understanding and analytical answers to user questions, thereby optimizing the quality of general large models' responses to smart city domain knowledge.

[0111] Calculation example: (1) Data collection. 39 urban operation evaluation index systems, 15 urban governance policy documents, and 18 theoretical documents in my country were selected as learning materials for the local knowledge base.

[0112] (2) Build a vector database. Based on the benchmark results of the Embedding model on the open source model sharing platform Hugging Face, we selected gte-qwen2-7b-instruct as the Embedding model. We converted the organized text materials into vectors and stored them in the vector database.

[0113] (3) Constructing a knowledge graph. Based on the evaluation results of large language models on the open source model sharing platform Hugging Face, including language comprehension, common sense comprehension, and instruction-following capabilities, the llama3.1:70b model was selected to construct the knowledge graph. Factor extraction was performed using a triple (entity-relationship-attribute) approach, forming a knowledge graph covering the key entities, relationships, and attributes for smart city operation evaluation.

[0114] (4) Knowledge graph analysis. The constructed knowledge graph in the field of smart city evaluation contains 25,698 node data, 29,249 relationship data, and identifies 10,086 communities, which serve as knowledge materials for comprehensive understanding and analysis by the large language model.

[0115] 2. Generate a smart city evaluation index system through knowledge query Implementation steps: (1) Model role setting. The user sets the selected large language model as an expert in the field of "smart city planning and governance" through natural language dialogue, clarifying the domain knowledge that the model understands and excels in. The large language model will retrieve relevant community summaries and knowledge information from the graph database based on its set role and the user's questions.

[0116] (2) Prompt word design. Users provide prompt words through natural language dialogue around the key goals, evaluation dimensions, and measurement indicators of smart city evaluation, and continuously optimize and adjust them until the results generated by the large language model meet user needs.

[0117] (3) Knowledge query generation. The user proposes a query requirement for constructing smart city evaluation indicators. The selected large language model acts as a converter for question-answering queries, mapping the user's natural language input questions into "semantic queries" on the knowledge graph. Furthermore, GraphRAG prioritizes the communities most relevant to constructing evaluation indicators and retrieves specific entity and relationship information from within the communities. This structured graph data information (entities, relationships, and community context) is then used as input for "knowledge enhancement" and passed to the large language model for analysis and synthesis of answers, which are then fed back to the user until they meet the user's needs.

[0118] Calculation example: First, we extract the key goals of smart city development and obtain the enumerated answers of the big language model on the eight key goals of smart city development. Furthermore, we let the big language model combine the eight key goals of smart city development, summarize the key evaluation dimensions, extract the measurement indicators under each evaluation dimension, and attach the definition, calculation method and basis of each measurement indicator. Figure 5 shown.

[0119] 3. Modification of the evaluation index system Implementation steps: To ensure that the indicator system generated by the large language model has practical application value, we added a manual correction step and established correction rules based on the availability and applicability of the indicators (see Table 1) to retain, merge, adjust or delete the indicator system generated by artificial intelligence.

[0120] Table 1 Evaluation index system revision rules

[0121] Calculation example: The measurement indicator "Intelligent Monitoring and Management Rate of Municipal Pipelines" is calculated as (Length of Municipal Pipelines with Intelligent Monitoring / Total Length of Municipal Pipelines) × 100%. Regarding availability, the length of municipal pipelines with intelligent monitoring and the total length of municipal pipelines are available from relevant government departments, resulting in an "Availability" rating of A. Regarding applicability, intelligent monitoring of municipal pipelines is a key component of smart city development, resulting in an "A" rating for applicability. Therefore, this indicator is revised to "Aa Full Retention."

[0122] 4. Likert scale method to verify reliability and validity Implementation steps: We use expert scoring and verification to ensure the scientific validity of the final smart city indicator system. If the final indicator system fails verification, we will report the issue to the knowledge query system and optimize processes one and two by adjusting knowledge query prompts, increasing the size of the data set, and other methods until the output meets verification.

[0123] The specific steps of expert verification are as follows: (1) Design a questionnaire and Likert scale to clarify the basic criteria for scoring the indicator system generated by the large language model. Distribute questionnaires to experts in the field of smart cities and conduct quantitative scoring using the Likert scale.

[0124] (2) Based on the scores of all experts, Cronbach's α and item-level content validity index (I-CVI) were calculated to test the internal consistency of the indicator system and the content validity of the items. Cronbach's α>0.7 indicates that the internal consistency of the indicator system is good; I-CVI ≥0.78 (when the number of experts is ≥6) indicates that the content validity of the items is good.

[0125] Calculation example: (1) Design a questionnaire and Likert scale, and invite 10 experts in the field of smart cities to conduct a Likert 5-level quantitative scoring of the smart city evaluation index system according to the scoring criteria in Table 2.

[0126] Table 2 Expert scoring criteria

[0127] (2) Based on the scores of all experts, Cronbach's α and item-level content validity index (I-CVI) were calculated using SPSS 27.0 software. As shown in Table 3, the Cronbach's α values ​​were all greater than 0.7, indicating that the internal consistency of the evaluation system was good; the I-CVI values ​​were all greater than 0.78, indicating that the item content validity of the indicator system was good.

[0128] Table 3 Evaluation system verification results

[0129] The method for creating an indicator system provided in the embodiment of the present application can be executed by an indicator system creation device. In the embodiment of the present application, the method for creating an indicator system performed by the indicator system creation device is taken as an example to illustrate the indicator system creation device provided in the embodiment of the present application.

[0130] like Figure 6 As shown, it shows a schematic diagram of the structure of the device for creating the indicator system provided by the embodiment of the present application. Figure 6 The index system creation device 90 includes: A first conversion module 901 is configured to control the large language model to convert a query request for creating a target indicator system into a semantic query request for a knowledge graph if a query request for creating a target indicator system is detected, wherein the target indicator system is an indicator system for evaluating a target object; A first query module 902 is configured to control the graph retrieval enhancement model to query a target community hit by the query request in a target knowledge graph; the target knowledge graph is a knowledge graph created based on triples extracted from target knowledge data, wherein the triples include: entities, attributes of entities, and relationships between entities; the target knowledge data is knowledge data containing target indicators, and the target indicators are indicators used to evaluate the target objects; The first generating module 903 is configured to control the large language model to generate the target indicator system according to the triples in the found target community and the summary of the found target community.

[0131] The indicator system creation device 90 in the embodiments of the present application can be an electronic device or a component of an electronic device, such as an integrated circuit or chip. The electronic device can be a terminal or other device other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA). It can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine (ATM), or a self-service machine, etc., and the embodiments of the present application do not specifically limit this.

[0132] The index system creation device 90 in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0133] The index system creation device 90 provided in the embodiment of the present application can achieve Figures 1 to 5 To avoid repetition, the various processes implemented in the method embodiment are not described here.

[0134] In some optional embodiments, such as Figure 7 As shown, an embodiment of the present application also provides an electronic device 1300, including a processor 1301 and a memory 1302, wherein the memory 1302 stores a program or instruction that can be run on the processor 1301, and when the program or instruction is executed by the processor 1301, the various steps of the embodiment of the method for creating the above-mentioned indicator system are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0135] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0136] Figure 8 A schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.

[0137] The electronic device 170 includes, but is not limited to, components such as a radio frequency unit 1701, a network module 1702, an audio output unit 1703, an input unit 1704, a sensor 1705, a display unit 1706, a user input unit 1707, an interface unit 1708, a memory 1709, and a processor 17010. Those skilled in the art will appreciate that the electronic device 170 may also include a power supply (such as a battery) to power the various components. The power supply may be logically connected to the processor 17010 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be repeated here.

[0138] The processor 17010 is configured to: If a query request for creating a target indicator system is detected, controlling the large language model to convert the query request into a semantic query request for the knowledge graph, wherein the target indicator system is an indicator system for evaluating the target object; The control graph retrieval enhancement model searches for a target community hit by the query request in a target knowledge graph; the target knowledge graph is a knowledge graph created based on triples extracted from target knowledge data, the triples including: entities, attributes of entities, and relationships between entities; the target knowledge data is knowledge data containing target indicators, and the target indicators are indicators used to evaluate the target object; The large language model is controlled to generate the target indicator system according to the triples in the found target community and the summary of the found target community.

[0139] It should be understood that in this embodiment of the present application, the input unit 1704 may include a graphics processing unit (GPU) 17041 and a microphone 17042. The GPU 17041 processes image data of still images or videos captured by an image capture device (e.g., a camera) in video capture mode or image capture mode. The display unit 1706 may include a display panel 17061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1707 includes a touch panel 17071 or at least one of other input devices 17072. The touch panel 17071, also known as a touch screen, may include a touch detection device and a touch controller. Other input devices 17072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on / off keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail here.

[0140] Memory 1709 can be used to store software programs and various data. Memory 1709 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store an operating system, applications or instructions required for at least one function (such as sound playback or image playback), and the like. Furthermore, memory 1709 may include volatile memory or non-volatile memory, or both. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM RAM (DRRAM). Memory 1709 in the embodiments of the present application includes, but is not limited to, these and any other suitable types of memory.

[0141] Processor 17010 may include one or more processing units. Optionally, processor 17010 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 17010.

[0142] Any of the above-mentioned product embodiments can implement the various processes of the above-mentioned indicator system creation method embodiment through its own processor operation, and can achieve the same technical effect. To avoid repetition, they will not be described one by one.

[0143] The embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by the processor, each process of the embodiment of the method for creating the above-mentioned indicator system is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. Among them, the processor is the processor in the electronic device or electronic system described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a disk or an optical disk, etc.

[0144] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the embodiment of the method for creating the above-mentioned indicator system, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0145] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0146] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the embodiment of the method for creating an indicator system as described above, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0147] In the embodiments provided in the examples of the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0148] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.

[0149] In addition, each functional unit in each implementation of the embodiment of the present application may be integrated into a processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The above-mentioned integrated units may be implemented in the form of hardware or software functional units.

[0150] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0151] The above description is only an implementation method of the embodiment of the present application, and does not limit the patent scope of the embodiment of the present application. The above specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can make equivalent structural or equivalent process changes using the description and drawings of the embodiment of the present application, or directly or indirectly apply them in other related technical fields. Without departing from the scope of protection of the purpose of this application and the claims, many forms can be made, which are also included in the patent protection scope of the embodiment of the present application.

Claims

1. A method for creating an indicator system, characterized in that: The method comprises: If a query request for creating a target indicator system is detected, controlling the large language model to convert the query request into a semantic query request for the knowledge graph, wherein the target indicator system is an indicator system for evaluating the target object; The control graph retrieval enhancement model searches for a target community hit by the query request in a target knowledge graph; the target knowledge graph is a knowledge graph created based on triples extracted from target knowledge data, the triples including: entities, attributes of entities, and relationships between entities; the target knowledge data is knowledge data containing target indicators, and the target indicators are indicators used to evaluate the target object; The large language model is controlled to generate the target indicator system according to the triples in the found target community and the summary of the found target community.

2. The method according to claim 1, characterized in that After controlling the large language model to generate the target indicator system according to the triples in the found target community, the method further includes: If an evaluation attribute input by the user for the i-th indicator is received, the i-th indicator is corrected according to the evaluation attribute input by the user for the i-th indicator and the preset correction rule of the i-th indicator, where the i-th indicator is any indicator in the target indicator system.

3. The method according to claim 2, characterized in that After receiving the evaluation attribute input by the user for the i-th indicator, and correcting the i-th indicator according to the evaluation attribute input by the user for the i-th indicator and the preset correction rule for the i-th indicator, the method further includes: Obtain the Cronbach's alpha coefficient and item-level content validity index of the target indicator system; Verify the reliability and validity of the target indicator system based on the Cronbach's alpha coefficient and the item-level content validity index; If the verification is passed, the target indicator system is output.

4. The method according to claim 3, characterized in that If a query request for creating a target indicator system is detected, before controlling the large language model to convert the query request into a semantic query request for the knowledge graph, the method further includes: Acquire target knowledge data; Controlling the text embedding model to convert the target knowledge data into a vector; If a user prompt word for extracting the target indicator is received, controlling the large language model to extract a triplet related to the user prompt word from the vector; Controlling the graph retrieval enhancement model to create the target knowledge graph according to the extracted triples related to the user prompt word; The graph retrieval enhancement model is controlled to perform community analysis on the knowledge graph to cluster at least one community from the target knowledge graph and generate a summary of each community in the at least one community.

5. The method according to claim 4, characterized in that The method further comprises: If the verification fails and a new user prompt word is received from the user, the method returns to the step of controlling the large language model to extract triples related to the user prompt word from the vector.

6. The method according to claim 1, characterized in that The method further comprises: Control the large language model to generate the definition, calculation method and extraction basis of each indicator in the target indicator system.

7. A device for creating an indicator system, characterized in that: The device comprises: A first conversion module is configured to, upon detecting a query request requesting the creation of a target indicator system, control the large language model to convert the query request into a semantic query request for the knowledge graph, wherein the target indicator system is an indicator system for evaluating a target object; A first query module is configured to control the graph retrieval enhancement model to query a target community hit by the query request in a target knowledge graph; the target knowledge graph is a knowledge graph created based on triples extracted from target knowledge data, the triples including: entities, attributes of entities, and relationships between entities; the target knowledge data is knowledge data containing target indicators, and the target indicators are indicators used to evaluate the target objects; The first generating module is used to control the large language model to generate the target indicator system according to the triples in the found target community and the summary of the found target community.

8. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method for creating an indicator system as described in any one of claims 1 to 6 are implemented.

9. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method for creating an indicator system according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program includes a program or an instruction, and when the program or the instruction is executed by a processor, the steps of the method for creating an indicator system according to any one of claims 1 to 6 are implemented.

Images