A software full life cycle performance measurement method, device, equipment and medium

Abstract

The application discloses a software full life cycle performance measurement method, device, equipment and medium. The method comprises the following steps: acquiring a natural language query sentence of software performance input by a user; performing multi-path parallel retrieval according to the natural language query sentence based on a software full life cycle multi-layer knowledge graph to obtain at least one retrieval result; performing fusion processing on the at least one retrieval result to obtain a target keyword; generating fusion corpus according to the target keyword and target domain corpus retrieved in the software full life cycle knowledge base, and generating a target meta prompt word according to the target keyword; inputting the natural language query sentence, the fusion corpus and the target meta prompt word into a fine-tuned large language model to obtain a target structured query sentence; and outputting a software performance query result corresponding to the natural language query sentence according to the target structured query sentence. The embodiment of the application can improve the efficiency and accuracy of software development performance analysis.

Description

Technical Field

[0001] This invention relates to the field of software development performance analysis technology, and in particular to a method, apparatus, device and medium for measuring the performance of software throughout its entire lifecycle. Background Technology

[0002] With the widespread adoption of cloud-native architecture, microservice architecture, and DevOps development models, software development activities are increasingly permeating multiple stages, including requirements management, code development, continuous integration, testing and verification, release and deployment, operational monitoring, and security governance. Throughout the software lifecycle, a large amount of data related to software performance is generated, including code performance data, testing performance data, operational performance data, and security performance data. How to uniformly measure and intelligently analyze software performance based on this data has become an important research direction in the field of software development management.

[0003] Currently, existing technologies typically require manually writing structured query statements, script code, or indicator calculation rules based on specific analytical needs, and then executing the corresponding query operations through database query tools, monitoring and analysis platforms, or data analysis systems to obtain the corresponding software performance results.

[0004] However, existing technologies result in low efficiency and low accuracy in software development performance analysis, making it difficult to meet the application needs of complex software performance analysis scenarios. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and medium for measuring the performance of software throughout its entire lifecycle. The embodiments of this invention can improve the efficiency and accuracy of software development performance analysis.

[0006] In a first aspect, embodiments of the present invention provide a method for measuring the performance of software throughout its entire lifecycle, the method comprising:

[0007] Obtain a natural language query statement for software performance input by the user; software performance includes at least one of code performance, test performance, runtime performance, and security performance.

[0008] Based on a multi-layered knowledge graph throughout the software lifecycle, multi-path parallel retrieval is performed according to natural language query statements to obtain at least one retrieval result.

[0009] At least one search result is merged to obtain the target keyword entries;

[0010] Based on the target keyword entries and the target domain corpus retrieved from the software's full lifecycle knowledge base, a fused corpus is generated, and target meta-prompt words are generated based on the target keyword entries;

[0011] The natural language query, the fused corpus, and the target meta-prompts are input into the fine-tuned large language model to obtain the target structured query.

[0012] Based on the target structured query statement, output the software performance query results corresponding to the natural language query statement.

[0013] Secondly, embodiments of the present invention also provide a software lifecycle performance measurement device, the device comprising:

[0014] The natural language query statement acquisition module is used to acquire the natural language query statement of software performance input by the user; the software performance includes at least one of code performance, test performance, runtime performance and security performance;

[0015] The search result generation module is used to perform multi-path parallel retrieval based on a multi-layer knowledge graph throughout the software lifecycle and natural language query statements to obtain at least one search result.

[0016] The target keyword generation module is used to fuse at least one search result to obtain target keyword entries;

[0017] The corpus and meta-prompt word generation module is used to generate fused corpus based on target keyword entries and target domain corpus retrieved from the software's full lifecycle knowledge base, and to generate target meta-prompt words based on target keyword entries;

[0018] The target structured query statement generation module is used to input natural language query statements, fused corpora and target meta-prompts into the fine-tuned large language model to obtain target structured query statements;

[0019] The software performance query result output module is used to output the software performance query results corresponding to the natural language query statement based on the target structured query statement.

[0020] Thirdly, embodiments of the present invention also provide a software lifecycle performance measurement device, which includes:

[0021] At least one processor; and

[0022] A memory that is communicatively connected to at least one processor; wherein,

[0023] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to execute the software lifecycle performance measurement method according to any embodiment of the present invention.

[0024] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions for causing a processor to execute a software lifecycle performance measurement method according to any embodiment of the present invention.

[0025] The technical solution of this invention obtains a natural language query statement on software performance input by the user, and performs multi-path parallel retrieval of the natural language query statement based on a multi-layer knowledge graph of the entire software lifecycle. This allows for the acquisition of retrieval information related to the query requirements from knowledge related to the entire software lifecycle. By fusing multiple retrieval results to generate target keyword entries, and combining them with target domain corpus from the software lifecycle knowledge base to generate fused corpus and target meta-prompts, contextual information related to the software performance domain can be provided for subsequent query processing. The natural language query statement, fused corpus, and target meta-prompts are input into a fine-tuned large language model to generate a target structured query statement, thereby achieving automatic conversion of natural language query requirements into structured query statements. Finally, the corresponding software performance query results are output based on the target structured query statement. This solves the technical problem of low efficiency and low accuracy in software development performance analysis caused by existing technologies, thereby improving the efficiency and accuracy of software development performance analysis.

[0026] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating a software lifecycle performance measurement method provided in this embodiment of the invention;

[0029] Figure 2 A flowchart illustrating a software lifecycle performance measurement method provided in this embodiment of the invention;

[0030] Figure 3 A schematic diagram of a software lifecycle performance measurement device provided in an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of a software lifecycle performance measurement device provided in an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] The acquisition, storage, and application of natural language query statements, multi-layer knowledge graphs for the entire software lifecycle, and knowledge bases for the entire software lifecycle involved in the technical solutions of this invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0035] Figure 1 This is a flowchart illustrating a software lifecycle performance measurement method provided by an embodiment of the present invention. This embodiment is applicable to situations involving software lifecycle performance measurement. The method can be executed by a software lifecycle performance measurement device, which can be implemented in hardware and / or software.

[0036] See Figure 1 The software lifecycle performance measurement methods shown include:

[0037] S101. Obtain a natural language query statement for software performance input by the user; software performance includes at least one of code performance, test performance, runtime performance, and security performance.

[0038] Software performance refers to a metric that characterizes the overall performance of a software system across its development, testing, deployment, operation, and security stages. It reflects the system's quality level, operational status, and development efficiency throughout its entire lifecycle. Software performance serves as a target for user queries and analysis, used to determine the software's operational or development status that is of interest to the user.

[0039] Natural language queries refer to queries entered by users using natural language expressions. These queries reflect the user's need for software performance information. For example, a user's input of "changes in interface error rate since the latest version release" or "comparison of P99 latency changes in the API gateway between the two most recent releases" would be a natural language query for software performance.

[0040] Among them, code performance can refer to a software performance that characterizes the quality level, development efficiency, and maintainability of the software code itself; test performance can refer to a software performance that reflects the execution effect and quality assurance capability of software testing activities; runtime performance can refer to a software performance that characterizes the performance of the software system during actual operation; and security performance can refer to a software performance that measures the software's security protection capabilities.

[0041] S102. Based on the multi-layer knowledge graph of the entire software lifecycle, perform multi-path parallel retrieval according to natural language query statements to obtain at least one retrieval result.

[0042] Among them, the multi-layer knowledge graph of the software lifecycle can refer to a structured knowledge organization model that describes various entities and their relationships in the entire lifecycle stages of software development, testing, release, operation and maintenance.

[0043] Multi-path parallel retrieval refers to a method of information retrieval from a knowledge graph simultaneously through multiple different retrieval paths for the same query requirement. Multi-path parallel retrieval is used to simultaneously obtain semantic, relational, and temporal information, avoiding information loss caused by relying on a single retrieval method.

[0044] The search results refer to the set of information related to the query requirements output by the knowledge graph after retrieval processing. One search path corresponds to one search result.

[0045] S103. Merge at least one search result to obtain the target keyword entry.

[0046] Fusion processing refers to the process of combining multiple search results from multiple paths. By comprehensively considering factors such as semantic matching, relational reliability, and temporal matching among different search results, a unified evaluation is performed on the multiple search results, and the information most relevant to the query requirements is selected. Fusion processing is used to improve the relevance and accuracy of the information used in subsequent keyword generation.

[0047] Among them, target keyword entries can refer to a set of information that, after fusion processing, can represent the core semantics of the user's query needs.

[0048] S104. Generate fused corpus based on the target keyword entries and the target domain corpus retrieved from the software lifecycle knowledge base, and generate target meta-prompt words based on the target keyword entries.

[0049] The software lifecycle knowledge base refers to a collection of data resources that stores domain knowledge from the stages of software development, testing, release, operation, and maintenance. It may include knowledge content such as indicator definition documents, query templates, business rules, technical specifications, domain terminology, and historical analysis cases.

[0050] The target domain corpus refers to domain knowledge content retrieved from the software's entire lifecycle knowledge base that is relevant to the current query requirement. The target domain corpus has a high correlation with the target keywords. It is used to supplement the domain knowledge involved in the query requirement, improving the subsequent ability of the large language model to understand specialized semantics.

[0051] The fused corpus refers to a comprehensive information set formed by integrating target keyword entries with target domain corpus. This fused corpus contains both the query requirement information reflected in the target keyword entries and the software domain knowledge information carried in the target domain corpus. The fused corpus is used to provide large language models with background knowledge and contextual information relevant to the query task.

[0052] Target meta-cue words refer to controlling prompts used to guide large language models in performing specific tasks, following specific reasoning logic, and generating target output results. Meta-cue words are meta-level instructions used to guide the output behavior of large language models; they are not directly embedded in user queries but rather parameterized instructions that control the model to perform operations in specific dimensions. Target meta-cue words can include cue word intent frames, output constraints, evidence fragment embeddings, prohibition declarations, and meta-cue word sequence combinations and verifications.

[0053] S105. Input the natural language query statement, the fused corpus, and the target meta-prompts into the fine-tuned large language model to obtain the target structured query statement.

[0054] In this context, a large language model refers to an artificial intelligence model trained on a large-scale corpus, possessing semantic understanding, knowledge reasoning, and text generation capabilities. Large language models analyze the semantic and contextual relationships within the input information to generate output results that meet the target requirements. Language models are used to convert natural language query requests into executable structured query statements. For example, when inputting "Query the changes in the delay of order service P99 in the last seven days," a large language model can generate the corresponding structured query expression.

[0055] Among them, the target structured query statement can refer to the structured query expression obtained after being generated by a large language model and passing syntax validation, semantic validation and logical validation, such as the PromQL query statement.

[0056] S106. Based on the target structured query statement, output the software performance query results corresponding to the natural language query statement.

[0057] In this context, software performance query results refer to the software performance analysis results obtained after executing a query operation based on a target structured query statement. The software performance query results are the final output of the entire software performance measurement process, used to provide users with performance information corresponding to their query requests. For example, when a user queries "the trend of order service error rate changes in the past month," the final output error rate trend data, statistical reports, or trend curves can all serve as software performance query results.

[0058] As can be seen, in this embodiment, by acquiring the user-input natural language query statement on software performance and performing multi-path parallel retrieval of the natural language query statement based on a multi-layer knowledge graph of the software lifecycle, retrieval information related to the query requirements can be obtained from knowledge related to the software lifecycle. By fusing multiple search results to generate target keyword entries, and combining them with the target domain corpus in the software lifecycle knowledge base to generate fused corpus and target meta-prompts, contextual information related to the software performance domain can be provided for subsequent query processing. The natural language query statement, fused corpus, and target meta-prompts are input into a fine-tuned large language model to generate a target structured query statement, thereby achieving automatic conversion of natural language query requirements into structured query statements. Finally, the corresponding software performance query results are output based on the target structured query statement. This solves the technical problem of low efficiency and low accuracy in software development performance analysis caused by existing technologies, thereby improving the efficiency and accuracy of software development performance analysis.

[0059] In an optional embodiment, Figure 2The flowchart of a software lifecycle performance measurement method provided in this embodiment of the invention refines the process of "using a multi-path parallel retrieval based on a multi-layer knowledge graph of the software lifecycle to obtain at least one retrieval result" into "converting the natural language query into a natural language query vector; performing semantic retrieval in the knowledge graph based on the natural language query vector to obtain candidate entities; extracting entity names from the natural language query and determining entity nodes based on the entity names; performing graph traversal reasoning from the multi-layer knowledge graph of the software lifecycle based on the entity nodes to obtain a set of relational paths associated with the entity nodes; extracting time constraints from the natural language query and parsing time constraints from the knowledge graph based on the time constraints to obtain a time window; and using the candidate entity, the set of relational paths, and the target time window as at least one retrieval result," thereby improving the operation of software lifecycle performance measurement.

[0060] It should be noted that for parts not described in detail in the embodiments of the present invention, please refer to the descriptions in other embodiments.

[0061] See Figure 2 The software lifecycle performance measurement methods shown include:

[0062] S201. Obtain a natural language query statement for software performance input by the user; software performance includes at least one of code performance, test performance, runtime performance, and security performance.

[0063] S202. Convert the natural language query statement into a natural language query vector.

[0064] Natural language query vectors can refer to the data representation of natural language query statements after numerical representation. Natural language query vectors are used to describe the semantic information contained in natural language query statements.

[0065] S203. Based on the natural language query vector, perform semantic retrieval in the multi-layer knowledge graph of the software's entire lifecycle to obtain candidate entities.

[0066] Candidate entities refer to entity objects that are relevant to the natural language query vector and are selected from the knowledge graph after semantic retrieval. Candidate entities represent entity objects that may be related to the user's query requirements. When a user queries order service performance information, candidate entities may include "order service," "transaction service," and "P99 delay," among others.

[0067] S204. Extract the entity names from the natural language query statement and determine the entity nodes based on the entity names.

[0068] Entity names can refer to name information used to identify a specific entity object. Entity names are used to identify specific services, metrics, versions, or event objects from natural language queries and to further locate the corresponding nodes in the knowledge graph. For example, when the natural language query is "Compare the P99 latency changes of the API gateway in the two most recent releases", the entity name could include "API gateway" and the metric "P99 latency".

[0069] In this context, entity nodes refer to graph structure nodes in a knowledge graph used to represent entity objects. As the basic building blocks of a knowledge graph, entity nodes are connected to other nodes through relational edges, thus forming a complete knowledge network structure. Entity nodes serve as the starting point for graph traversal reasoning to discover other knowledge information related to the target entity.

[0070] S205. Based on the entity nodes, perform graph traversal reasoning from the knowledge graph to obtain the set of relational paths associated with the entity nodes.

[0071] Graph traversal reasoning refers to the process of accessing and deriving the graph structure based on nodes and relational edges in a knowledge graph. Starting with entity nodes, it progressively visits related nodes along the relational edges in the knowledge graph to discover potential relationships and knowledge paths between entities. Graph traversal reasoning is used to obtain topological relationship information related to the queried entity, providing associated knowledge support for subsequent retrieval result fusion.

[0072] The relation path set refers to a collection of multiple entity relation paths obtained through graph traversal reasoning. Each relation path typically consists of several entity nodes and relation edges connecting these nodes, while the relation path set contains multiple different relation paths. The relation path set is used to represent upstream and downstream dependencies, business relationships, or temporal relationships related to the query object.

[0073] S206. Extract the time constraints from the natural language query statement, and based on the time constraints, perform time constraint parsing from the knowledge graph to obtain the time window.

[0074] Time constraints refer to the time range or time limit information specified by the user during the query process. Time constraints guide the time constraint parsing process and determine the time window involved in subsequent queries. For example, "the last seven days," "the last two publications," and "this quarter" are all time constraints.

[0075] Time constraint resolution refers to the process of identifying, understanding, and transforming time constraints. By analyzing time expressions in natural language, relative time descriptions are converted into specific time ranges that can be calculated and executed.

[0076] S207. The candidate entity, the set of relational paths, and the target time window are taken as at least one search result.

[0077] S208. Merge at least one search result to obtain the target keyword entry.

[0078] S209. Generate fused corpus based on the target keyword entries and the target domain corpus retrieved from the software lifecycle knowledge base, and generate target meta-prompt words based on the target keyword entries.

[0079] S210. Input the natural language query statement, the fused corpus, and the target meta-prompts into the fine-tuned large language model to obtain the target structured query statement.

[0080] S211. Based on the target structured query statement, output the software performance query results corresponding to the natural language query statement.

[0081] As can be seen, in this embodiment, based on a multi-layered knowledge graph throughout the software lifecycle, multi-path parallel retrieval is performed according to natural language queries to obtain at least one retrieval result. By combining semantic retrieval, graph traversal reasoning, and time constraint parsing, entity information, relationship information, and time information related to the natural language query can be obtained, thereby helping to improve the completeness of the retrieval results and providing a data foundation for the subsequent generation of software performance query results.

[0082] In some embodiments, a natural language query, a fused corpus, and target meta-prompts are input into a fine-tuned large language model to obtain a target structured query, including:

[0083] Perform intent recognition on natural language queries to obtain the query intent type.

[0084] The natural language query statement, query intent type, fused corpus, and target meta-prompt words are input into the fine-tuned large language model to obtain the initial structured query statement;

[0085] The initial structured query statement is parsed using a syntax tree to obtain the syntax parsing result;

[0086] Based on the set of standard entity names in the multi-layered knowledge graph of the software lifecycle, the initial structured query statement is subjected to entity consistency verification to obtain semantic alignment results;

[0087] Based on the results of syntax parsing and semantic alignment, the initial structured query statement is subjected to aggregation logic verification to obtain the target structured query statement.

[0088] Intent recognition refers to the process of semantically analyzing natural language queries input by users to determine their actual query purpose. By analyzing the semantic information in the query, it can be determined what type of software performance information the user wants to obtain. For example, a query for "What is the error rate of order services in the last month?" might correspond to statistical analysis; a query for "Changes in error rates between the last two versions?" might correspond to comparative analysis.

[0089] The query intent type refers to the category information used to characterize the user's query purpose, determined after intent identification. Query intent types can include: entity queries (such as "current status of the API gateway"), relationship queries (such as "which downstream services the order service depends on"), attribute queries (such as "average response time of the order service"), statistical queries (such as "calculate the average daily throughput of the transaction service"), comparison queries (such as "compare the P99 latency of the API gateway in the two most recent releases"), time-series queries (such as "the trend of error rate changes last week"), and mixed queries (compound queries that include multiple of the above patterns).

[0090] The initial structured query statement can refer to the preliminary query expression result generated by the large language model based on the natural language query statement, query intent type, fused corpus, and target meta prompts.

[0091] Syntax tree parsing refers to the hierarchical analysis of a structured query statement according to predetermined syntax rules. Syntax tree parsing is used to verify the syntactic correctness of structured query statements and provides an analytical basis for subsequent logical verification.

[0092] The syntax parsing result refers to the syntactic structure analysis information obtained after parsing the syntax tree. The syntax parsing result serves as an important basis for subsequent aggregation logic verification, helping to determine whether the query logic is reasonable.

[0093] The standard entity name set refers to a pre-established and stored set of standardized entity names within a multi-layered knowledge graph throughout the software's lifecycle. This set is used to perform entity consistency checks, preventing query errors caused by name discrepancies.

[0094] Entity consistency verification refers to the process of comparing entity information referenced in a structured query with standard entity information in the knowledge graph. Entity consistency verification improves the consistency between structured queries and the knowledge graph, reducing the possibility of incorrect entity references.

[0095] The semantic alignment result refers to the entity matching result and semantic mapping result obtained after entity consistency verification. The semantic alignment result can include information on successfully matched entities, entity mapping relationships, and information on unmatched entities. The semantic alignment result is used to assist in subsequent aggregation logic verification and query execution.

[0096] Aggregate logic validation refers to the process of verifying the rationality of statistical calculation logic in a structured query statement. Aggregate logic validation is used to improve the logical integrity and execution reliability of the target structured query statement.

[0097] As can be seen, in this embodiment, by inputting the natural language query statement, query intent type, fused corpus, and target meta-prompt words into the fine-tuned large language model, and combining it with syntax parsing, entity consistency verification, and aggregation logic verification, a target structured query statement can be generated. This helps to improve the matching degree between the structured query statement and the user's query needs, and provides support for the generation of software performance query results.

[0098] In some embodiments, at least one search result is fused to obtain target keyword entries, including:

[0099] Determine the semantic similarity, graph path reliability, and time accuracy corresponding to the search results, and determine the comprehensive score corresponding to the search results based on the semantic similarity, graph path reliability, and time accuracy;

[0100] Based on the overall score, the target search result is determined from at least one search result;

[0101] Generate target keyword entries based on the target search results.

[0102] Semantic similarity refers to the degree of closeness between two semantic objects at the meaning level, and is used to measure the semantic matching degree between query content and knowledge objects. Semantic similarity is used to evaluate the matching degree between search results and user query needs, and serves as an important basis for the fusion and processing of search results.

[0103] Graph path credibility refers to the reliability of a relationship path in a knowledge graph in reflecting the actual relationships between entities. Graph path credibility is used to evaluate the reliability of graph traversal reasoning results and serves as an important reference for filtering search results.

[0104] Time precision refers to the degree of matching between the time information in the search results and the time window of the user's query. Time precision is used to evaluate the adaptability of search results in the time dimension, thereby improving the matching between the final search results and the query requirements.

[0105] The overall score refers to the result score obtained after comprehensively calculating multiple evaluation indicators. The overall score reflects the overall matching degree between the search results and the user's query needs, and serves as an important basis for determining the target search results.

[0106] The target search result can refer to the search result obtained by filtering multiple search results based on the overall score. For example, the search result corresponding to the highest overall score can be used as the target search result.

[0107] As can be seen, in this embodiment, by combining semantic similarity, graph path reliability, and time accuracy to comprehensively score the search results, and determining the target search results based on the scoring results, the search results can be filtered, thereby helping to improve the relevance between the target keywords and the user's query needs.

[0108] In some embodiments, a fused corpus is generated based on target keyword entries and target domain corpus retrieved from a software lifecycle knowledge base, including:

[0109] Obtain the term entity information and term intent category from the target keyword entries, and then vectorize the term entity information and term intent category to obtain the term query vector;

[0110] Based on the term query vector, the target domain corpus is retrieved from the software lifecycle knowledge base;

[0111] The target domain corpus and target keyword entries are fused to generate a fused corpus.

[0112] In this context, a term query vector refers to the data representation obtained by vectorizing the term entity information and term intent category in a target keyword term. By converting the target keyword term into vector form, the corpus information in the knowledge base can be vector-matched with the query requirements, thereby achieving semantic retrieval.

[0113] As can be seen, in this embodiment, by retrieving the corresponding target domain corpus based on the target keyword, and fusing the target domain corpus and the target keyword, domain knowledge can be associated and integrated with the retrieval results, thereby helping to improve the matching degree between the fused corpus and the user's query needs.

[0114] In some embodiments, a multi-layered knowledge graph covering the entire software lifecycle is constructed in the following ways:

[0115] Obtain raw R&D data from at least one data source corresponding to the R&D system;

[0116] Based on the original R&D data, extract the service entity, indicator entity, version entity, time entity, and event entity corresponding to the original R&D data;

[0117] Based on the relationships between service entities, indicator entities, version entities, time entities, and event entities, construct relationship edges in the knowledge graph;

[0118] Based on the relation edges, generate the initial knowledge graph corresponding to the R&D system;

[0119] The initial knowledge graph is divided and a summary is generated to create a multi-layered knowledge graph for the entire software lifecycle.

[0120] Among them, a service entity can refer to a software service object in a software system that can independently complete a specific business function; an indicator entity can refer to a metric object that describes the operating status, development process status, or business status of a software system; a version entity can refer to an entity object that represents a version release or software iteration result of a software system; a time entity can refer to an entity object that represents time information; and an event entity can refer to an activity object that has business or operational significance during the entire software lifecycle.

[0121] Here, a relation edge can refer to a data structure in a knowledge graph used to connect two entity nodes and represent the relationship between entities.

[0122] The initial knowledge graph can refer to a knowledge graph that is constructed based on entities and their relation edges, but has not yet undergone community division and summary generation.

[0123] As can be seen, in this embodiment, by extracting service entities, indicator entities, version entities, time entities, and event entities from the original R&D data, and constructing the relationships between entities to generate a knowledge graph, the association between information throughout the software lifecycle can be established; by dividing the initial knowledge graph and generating summaries, the hierarchical organization of knowledge content can be achieved.

[0124] In some embodiments, the initial knowledge graph is divided and a summary is generated to produce a multi-layered knowledge graph for the entire software lifecycle, including:

[0125] The initial knowledge graph is divided into communities to obtain at least one knowledge community.

[0126] Generate community summary information for at least one knowledge community;

[0127] Based on the initial knowledge graph and community summary information corresponding to at least one knowledge community, a multi-layered knowledge graph for the entire software lifecycle is generated.

[0128] Community partitioning refers to the process of grouping entities within a knowledge graph according to the degree of association between them. By analyzing the topological features of the knowledge graph, community partitioning identifies clustering relationships between entities, thus forming multiple knowledge regions with clearly defined thematic characteristics. Community partitioning is used to structure and organize the initial knowledge graph, enabling subsequent knowledge retrieval to prioritize knowledge regions relevant to the query requirements, thereby reducing the search scope and improving retrieval efficiency.

[0129] Among them, a knowledge community can refer to a collection of knowledge formed after community division, consisting of multiple related entities and their relationships.

[0130] Community summary information refers to structured descriptive information formed by extracting and summarizing the core knowledge content of a knowledge community. Community summary information may include community topic information, core entity information, main relationship information, key indicator information, knowledge scope description information, and community identification information. Community summary information is used to assist in quickly identifying target knowledge communities during subsequent retrieval processes and provides a reference for query routing and knowledge location.

[0131] As can be seen, in this embodiment, by dividing the initial knowledge graph into communities and generating corresponding community summary information, the hierarchical organization and summary expression of knowledge content can be realized; by generating a multi-layer knowledge graph for the entire software lifecycle, it can help improve the efficiency of locating relevant knowledge in the knowledge graph and the targeting of retrieval.

[0132] Figure 3 This invention provides a schematic diagram of a software lifecycle performance measurement device. This invention is applicable to situations involving software lifecycle performance measurement. The device can execute methods for measuring software lifecycle performance and can be implemented in hardware and / or software.

[0133] See Figure 3 The software lifecycle performance measurement device shown includes: a natural language query statement acquisition module 301, a search result generation module 302, a target keyword entry generation module 303, a corpus and meta-prompt word generation module 304, a target structured query statement generation module 305, and a software performance query result output module 306.

[0134] Natural language query statement acquisition module 301 is used to acquire a natural language query statement of software performance input by the user; software performance includes at least one of code performance, test performance, runtime performance and security performance;

[0135] The retrieval result generation module 302 is used to perform multi-path parallel retrieval based on a multi-layer knowledge graph throughout the software lifecycle and natural language query statements to obtain at least one retrieval result.

[0136] The target keyword generation module 303 is used to fuse at least one search result to obtain a target keyword.

[0137] The corpus and meta-prompt word generation module 304 is used to generate fused corpus based on the target keyword entries and the target domain corpus retrieved from the software's full lifecycle knowledge base, and to generate target meta-prompt words based on the target keyword entries;

[0138] The target structured query statement generation module 305 is used to input natural language query statements, fused corpora and target meta prompts into the fine-tuned large language model to obtain target structured query statements;

[0139] The software performance query result output module 306 is used to output the software performance query result corresponding to the natural language query statement based on the target structured query statement.

[0140] The technical solution of this invention obtains a natural language query statement on software performance input by the user, and performs multi-path parallel retrieval of the natural language query statement based on a multi-layer knowledge graph of the entire software lifecycle. This allows for the acquisition of retrieval information related to the query requirements from knowledge related to the entire software lifecycle. By fusing multiple retrieval results to generate target keyword entries, and combining them with target domain corpus from the software lifecycle knowledge base to generate fused corpus and target meta-prompts, contextual information related to the software performance domain can be provided for subsequent query processing. The natural language query statement, fused corpus, and target meta-prompts are input into a fine-tuned large language model to generate a target structured query statement, thereby achieving automatic conversion of natural language query requirements into structured query statements. Finally, the corresponding software performance query results are output based on the target structured query statement. This solves the technical problem of low efficiency and low accuracy in software development performance analysis caused by existing technologies, thereby improving the efficiency and accuracy of software development performance analysis.

[0141] In some embodiments, regarding obtaining at least one search result by performing multi-path parallel retrieval based on a multi-layer knowledge graph throughout the software lifecycle and according to a natural language query, the search result generation module 302 is specifically used for:

[0142] Convert natural language query statements into natural language query vectors;

[0143] Based on natural language query vectors, semantic retrieval is performed in a multi-layered knowledge graph throughout the software lifecycle to obtain candidate entities;

[0144] Extract entity names from natural language query statements and determine entity nodes based on entity names;

[0145] Based on entity nodes, perform graph traversal reasoning from the knowledge graph to obtain a set of relational paths associated with entity nodes;

[0146] Extract time constraints from natural language query statements, and based on these time constraints, perform time constraint parsing from the knowledge graph to obtain the time window;

[0147] Candidate entities, relational path sets, and target time windows are considered as at least one search result.

[0148] In some embodiments, in obtaining a target structured query statement by inputting a natural language query statement, a fused corpus, and target meta-prompts into a fine-tuned large language model, the target structured query statement generation module 305 is specifically used for:

[0149] Perform intent recognition on natural language queries to obtain the query intent type.

[0150] The natural language query statement, query intent type, fused corpus, and target meta-prompt words are input into the fine-tuned large language model to obtain the initial structured query statement;

[0151] The initial structured query statement is parsed using a syntax tree to obtain the syntax parsing result;

[0152] Based on the set of standard entity names in the multi-layered knowledge graph of the software lifecycle, the initial structured query statement is subjected to entity consistency verification to obtain semantic alignment results;

[0153] Based on the results of syntax parsing and semantic alignment, the initial structured query statement is subjected to aggregation logic verification to obtain the target structured query statement.

[0154] In some embodiments, in fusing at least one search result to obtain a target keyword entry, the target keyword entry generation module 303 is specifically used for:

[0155] Determine the semantic similarity, graph path reliability, and time accuracy corresponding to the search results, and determine the comprehensive score corresponding to the search results based on the semantic similarity, graph path reliability, and time accuracy;

[0156] Based on the overall score, the target search result is determined from at least one search result;

[0157] Generate target keyword entries based on the target search results.

[0158] In some embodiments, in generating a fused corpus based on target domain corpus retrieved from the software lifecycle knowledge base and target keyword entries, the corpus and meta-prompt word generation module 304 is specifically used for:

[0159] Obtain the term entity information and term intent category from the target keyword entries, and then vectorize the term entity information and term intent category to obtain the term query vector;

[0160] Based on the term query vector, the target domain corpus is retrieved from the software lifecycle knowledge base;

[0161] The target domain corpus and target keyword entries are fused to generate a fused corpus.

[0162] In some embodiments, a multi-layered knowledge graph covering the entire software lifecycle is constructed in the following ways:

[0163] Obtain raw R&D data from at least one data source corresponding to the R&D system;

[0164] Based on the original R&D data, extract the service entity, indicator entity, version entity, time entity, and event entity corresponding to the original R&D data;

[0165] Based on the relationships between service entities, indicator entities, version entities, time entities, and event entities, construct relationship edges in the knowledge graph;

[0166] Based on the relation edges, generate the initial knowledge graph corresponding to the R&D system;

[0167] The initial knowledge graph is divided and a summary is generated to create a multi-layered knowledge graph for the entire software lifecycle.

[0168] In some embodiments, the initial knowledge graph is divided and a summary is generated to produce a multi-layered knowledge graph for the entire software lifecycle, including:

[0169] The initial knowledge graph is divided into communities to obtain at least one knowledge community.

[0170] Generate community summary information for at least one knowledge community;

[0171] Based on the initial knowledge graph and community summary information corresponding to at least one knowledge community, a multi-layered knowledge graph for the entire software lifecycle is generated.

[0172] The software lifecycle performance measurement device provided in this embodiment of the invention can execute the software lifecycle performance measurement method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the software lifecycle performance measurement method.

[0173] Figure 4 This is a schematic diagram of a software lifecycle performance measurement device provided in an embodiment of the present invention.

[0174] like Figure 4 As shown, the software lifecycle performance measurement device 400 includes at least one processor 401 and a memory, such as a read-only memory (ROM) 402 and a random access memory (RAM) 403, communicatively connected to the at least one processor 401. The memory stores computer programs executable by the at least one processor. The processor 401 can perform various appropriate actions and processes based on the computer program stored in the ROM 402 or loaded from storage unit 408 into the RAM 403. The RAM 403 can also store various programs and data required for the operation of the software lifecycle performance measurement device 400. The processor 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0175] Multiple components in the software lifecycle performance measurement device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard or mouse; an output unit 407, such as various types of displays or speakers; a storage unit 408, such as a hard disk or optical disk; and a communication unit 409, such as a network interface card (NIC), a modem, or a wireless transceiver. The communication unit 409 allows the software lifecycle performance measurement device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0176] Processor 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 401 performs the various methods and processes described above, such as performance measurement methods throughout the software lifecycle.

[0177] In some embodiments, the software lifecycle performance measurement method can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed onto the software lifecycle performance measurement device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by processor 401, one or more steps of the software lifecycle performance measurement method described above can be performed. Alternatively, in other embodiments, processor 401 can be configured to perform the software lifecycle performance measurement method by any other suitable means (e.g., by means of firmware).

[0178] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0179] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0180] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0181] To provide user interaction, the systems and techniques described herein can be implemented on an operational testing device. This software lifecycle performance measurement device includes: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the software lifecycle performance measurement device. Other types of devices can also be used to provide user interaction; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0182] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0183] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0184] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0185] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for measuring the performance of software throughout its entire lifecycle, characterized in that, The method includes: Obtain a natural language query statement on software performance input by the user; the software performance includes at least one of code performance, test performance, runtime performance, and security performance. Based on a multi-layered knowledge graph throughout the software lifecycle, multi-path parallel retrieval is performed according to the natural language query statement to obtain at least one retrieval result. At least one of the search results is fused to obtain the target keyword entries; Based on the target keyword entries and the target domain corpus retrieved from the software lifecycle knowledge base, a fused corpus is generated, and target meta-prompt words are generated based on the target keyword entries; The natural language query statement, the fused corpus, and the target meta-prompt words are input into the fine-tuned large language model to obtain the target structured query statement; Based on the target structured query statement, output the software performance query results corresponding to the natural language query statement.

2. The method according to claim 1, characterized in that, The method, based on a multi-layered knowledge graph throughout the software lifecycle, performs multi-path parallel retrieval according to the natural language query statement to obtain at least one retrieval result, including: Convert the natural language query statement into a natural language query vector; Based on the natural language query vector, semantic retrieval is performed in the multi-layer knowledge graph of the software's entire lifecycle to obtain candidate entities; Extract the entity names from the natural language query statement, and determine the entity nodes based on the entity names; Based on the entity nodes, perform graph traversal reasoning from the knowledge graph to obtain the set of relational paths associated with the entity nodes; Extract the time constraints from the natural language query statement, and based on the time constraints, perform time constraint parsing from the knowledge graph to obtain the time window; The candidate entity, the set of relationship paths, and the target time window are used as at least one of the search results.

3. The method according to claim 1, characterized in that, The step of inputting the natural language query statement, the fused corpus, and the target meta-prompt words into a fine-tuned large language model to obtain the target structured query statement includes: The query intent type is obtained by performing intent recognition on the natural language query statement. The natural language query statement, the query intent type, the fused corpus, and the target meta-prompt words are input into the fine-tuned large language model to obtain the initial structured query statement; The initial structured query statement is parsed using a syntax tree to obtain the syntax parsing result; Based on the set of standard entity names in the multi-layer knowledge graph of the software's entire lifecycle, the initial structured query statement is subjected to entity consistency verification to obtain semantic alignment results; Based on the syntax parsing results and the semantic alignment results, the initial structured query statement is subjected to aggregation logic verification to obtain the target structured query statement.

4. The method according to claim 1, characterized in that, The process of fusing at least one of the search results to obtain the target keyword entry includes: Determine the semantic similarity, graph path reliability, and time accuracy corresponding to the search results, and determine the comprehensive score corresponding to the search results based on the semantic similarity, graph path reliability, and time accuracy; Based on the comprehensive score, the target search result is determined from at least one search result; Based on the target search results, the target keyword entry is generated.

5. The method according to claim 1, characterized in that, The step of generating fused corpus based on the target keyword entries and the target domain corpus retrieved from the software's full lifecycle knowledge base includes: Obtain the term entity information and term intent category from the target keyword entry, and then vectorize the term entity information and term intent category to obtain the term query vector; Based on the term query vector, the target domain corpus is retrieved from the software lifecycle knowledge base; The target domain corpus and the target keyword entries are fused to generate the fused corpus.

6. The method according to claim 1, characterized in that, The multi-layered knowledge graph covering the entire software lifecycle is constructed in the following ways: Obtain raw R&D data from at least one data source corresponding to the R&D system; Based on the original R&D data, extract the service entity, indicator entity, version entity, time entity, and event entity corresponding to the original R&D data; Based on the relationships between the service entity, the indicator entity, the version entity, the time entity, and the event entity, construct relationship edges in the knowledge graph; Based on the relationship edges, an initial knowledge graph corresponding to the R&D system is generated; The initial knowledge graph is divided and a summary is generated to produce a multi-layer knowledge graph for the entire software lifecycle.

7. The method according to claim 6, characterized in that, The process of dividing and summarizing the initial knowledge graph to generate a multi-layered knowledge graph for the entire software lifecycle includes: The initial knowledge graph is divided into communities to obtain at least one knowledge community; Based on at least one of the knowledge communities, generate community summary information corresponding to at least one of the knowledge communities; Based on the initial knowledge graph and community summary information corresponding to at least one of the knowledge communities, a multi-layer knowledge graph for the entire software lifecycle is generated.

8. A performance measurement device for the entire software lifecycle, characterized in that, include: The natural language query statement acquisition module is used to acquire a natural language query statement of software performance input by the user; the software performance includes at least one of code performance, test performance, runtime performance and security performance. The retrieval result generation module is used to perform multi-path parallel retrieval based on the natural language query statement, according to the multi-layer knowledge graph of the software's entire life cycle, to obtain at least one retrieval result. The target keyword generation module is used to fuse at least one of the search results to obtain target keyword entries; The corpus and meta-prompt word generation module is used to generate fused corpus based on the target keyword entries and the target domain corpus retrieved from the software's full lifecycle knowledge base, and to generate target meta-prompt words based on the target keyword entries; The target structured query statement generation module is used to input the natural language query statement, the fused corpus, and the target meta-prompt words into the fine-tuned large language model to obtain the target structured query statement; The software performance query result output module is used to output the software performance query result corresponding to the natural language query statement based on the target structured query statement.

9. A performance measurement device for the entire software lifecycle, characterized in that, The performance measurement devices for the entire software lifecycle include: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the software lifecycle performance measurement method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the software lifecycle performance measurement method according to any one of claims 1-7.

Images