Customized software function point identification and diagnosis system based on big data technology

Abstract

The invention relates to the technical field of customized software identification, and particularly discloses a customized software function point identification and diagnosis system based on a big data technology, which comprises a total data acquisition module, an intelligent data preprocessing module, a function feature extraction module and a dynamic weight adaptive identification module, an abnormal mode diagnosis module and a decision support and optimization suggestion module; through an online element learning algorithm in the dynamic weight adaptive recognition module, the system can quickly update weight parameters based on a very small number of fault samples, large-scale off-line training is not needed, and high recognition performance can be kept in a new project or new version scene. The few-sample rapid adaptation capability effectively solves the generalization bottleneck of a traditional model in the face of frequent iteration and scene diversity, and the model deployment and optimization period is remarkably shortened.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of custom software identification, and particularly relates to a custom software function point identification and diagnosis system based on big data technology. BACKGROUND

[0002] In today's era of rapid digital development, custom software is widely used in various industries, and accurate identification and timely diagnosis of its function points are crucial for ensuring stable software operation, improving user experience, and meeting business needs. However, existing software function point identification and diagnosis technologies have many shortcomings in practical application, making it difficult to meet the increasingly complex and changing software environment and business scenario needs.

[0003] In terms of feature extraction, existing technologies usually use traditional linear or tree analysis methods, only considering the node's own attributes, while ignoring cross-module call relationships. In complex microservice architectures, the interaction and dependency between software function points are complex, and traditional methods cannot capture this deep module dependency and interaction mode. For example, a malfunction of a function point may propagate through cross-module calls, triggering a chain reaction, and traditional methods cannot identify this potential coupling risk, resulting in inaccurate characterization of function points and affecting the reliability of function point identification.

[0004] Simple clustering algorithms can identify some abnormal points, but cannot distinguish between isolated fault points and real risk clusters. In asynchronous call scenarios, the call timing between function points is complex, and simple clustering algorithms cannot capture this timing relationship, resulting in inaccurate positioning of risk points. For example, isolated fault points caused by asynchronous calls may be misjudged as part of a risk cluster, while real risk clusters may be ignored, affecting the credibility and explainability of fault warning.

[0005] To address this, the present application proposes a custom software function point identification and diagnosis system based on big data technology to solve the above problems. SUMMARY

[0006] The purpose of the present application is to provide a custom software function point identification and diagnosis system based on big data technology to solve the problems raised in the background.

[0007] To achieve the above purpose, the present application provides the following technical solutions:

[0008] A custom software function point identification and diagnosis system based on big data technology, comprising:

[0009] A full-data acquisition module for real-time extraction of raw data from multi-source system logs, code repositories, user interaction records, and demand management platforms to obtain software usage behavior data sets;

[0010] An intelligent data preprocessing module is configured to clean, deduplicate, format normalize, and deeply semantically label the data set to obtain a multi-modal data set;

[0011] A function feature extraction module is configured to extract semantic, frequency, and correlation features based on the multi-modal data set and by constructing a function call graph and applying a graph neural network to extract the features based on a graph database to obtain a function feature vector;

[0012] A dynamic weight adaptive identification module is configured to assign adaptive weights to the function feature vector in combination with project domain labels and runtime feedback by an online meta-learning algorithm to obtain a function point importance score;

[0013] An abnormal pattern diagnosis module is configured to perform Bayesian inference and density peak clustering analysis on the function point importance score and a historical fault library to locate a risk function point and generate a risk function point list;

[0014] A decision support and optimization suggestion module is configured to perform gray box testing on the list in a simulation sandbox environment, evaluate repair costs and regression risks, and output a priority repair scheme.

[0015] Preferably, the dynamic weight adaptive identification module assigns adaptive weights by an online meta-learning algorithm.

[0016] The online meta-learning algorithm is based on a few-shot update mechanism to realize rapid adaptation of weight parameters to new project scenarios, and a formula of the online meta-learning algorithm is as follows:

[0017] w k+1 =w k -α▽wL task (fw k (C))

[0018] wherein C is a function feature vector that integrates semantic features, call frequencies, cross-module correlations and other multi-dimensional information and is used to represent a call behavior of a function point;

[0019] wk is an importance weight vector of each feature dimension at the kth iteration;

[0020] α is a meta-learning rate (step size) that controls the amplitude of a single update and can be set to 0.001-0.01 in a scenario to balance convergence speed and stability;

[0021] Ltask(·) is a task loss function that is composed of two parts:

[0022] a historical fault prediction error that enables the model to reproduce a known risk point distribution;

[0023] Domain label consistency error: ensure the model output matches the project domain label (e.g. "finance", "e-commerce");

[0024] fw(C): scoring function with weight distribution, input feature vector, output function point importance score.

[0025] When accessing a new project, only a small amount of labeled fault samples are needed to quickly obtain new weights wk+1 through online updating, avoiding large-scale training from scratch.

[0026] Preferably, the anomaly pattern diagnosis module captures asynchronous call timing through a temporal Bayesian network and identifies clustered abnormal points using density peak clustering.

[0027] The formula of the temporal Bayesian network is:

[0028]

[0029] Wherein, Ri: event "function point i is a risk point";

[0030] Di: importance score of the function point (derived from the output of formula one);

[0031] P(Ri): prior risk probability, calculated from historical fault library (e.g. average failure rate of a certain type of function);

[0032] P(Di|Ri): probability distribution of score Di under the condition that the risk is true;

[0033] Calculate the risk tendency of each function point under the current score, for preliminary screening of high-risk candidates;

[0034] The formula of the density peak clustering is:

[0035]

[0036] Wherein, dij: distance between function points i and j in the score space (absolute difference or cosine distance in the scenario);

[0037] dc: cutoff distance threshold, generally 5% of the maximum possible value of the score;

[0038] χ(x): indicator function, 1 if x<0, otherwise 0;

[0039] pi: local density of the ith point, the greater the value, the greater the nearby high-density aggregation;

[0040] δi: the minimum distance from the ith point to the next high-density cluster center, the greater the value, the farther away from the next high-density cluster center;

[0041] Pi and delta i are used together to identify "density peaks" - that is, the most prominent risk cluster centers in the score space; can effectively distinguish isolated fault points caused by asynchronous calls and real risk clusters.

[0042] Preferably, the full data acquisition module further comprises a pluggable acquisition adapter for dynamically loading acquisition schemes of different system architectures.

[0043] Preferably, the intelligent data preprocessing module adopts a deep mapping network to automatically interface log fields with demand models and supports semantic unification of multilingual logs.

[0044] Preferably, the function feature extraction module constructs a function call graph containing cross-process and cross-service call edges and simultaneously fuses node and edge features based on a graph neural network.

[0045] Preferably, the decision support and optimization suggestion module follows a repair-verification closed-loop strategy, feeds back test results in a simulation sandbox in real time and dynamically adjusts subsequent repair priorities.

[0046] Compared with the prior art, the present application has the following beneficial effects:

[0047] (1) The present application can extract more representative multi-modal features on the basis of rich function context by means of the full data acquisition module for real-time convergence of system logs, code repositories, user interactions and demand management multi-dimensional information and deep semantic annotation in the intelligent preprocessing stage. This feature fusion can significantly improve the comprehensiveness and accuracy of function point recognition compared with single source methods, thereby more reliably reflecting the real running state and business value.

[0048] (2) The present application uses a graph database to construct a function call graph and introduces a graph neural network. This scheme first considers both node attributes and cross-module call relationships in the field of function point diagnosis. Compared with traditional linear or tree-shaped analysis, the graph neural network can capture deep module dependencies and interaction patterns, providing more accurate feature characterization and identification of potential functional coupling risks in complex microservice architectures, laying a solid foundation for subsequent weight allocation and risk diagnosis.

[0049] (3) The application can quickly update the weight parameters based on a small amount of fault samples through the online meta-learning algorithm in the dynamic weight adaptive identification module, without large-scale offline training, so as to maintain high identification performance in a new project or new version scenario. This small-sample rapid adaptation capability effectively solves the generalization bottleneck of the traditional model when facing frequent iteration and scene diversity, significantly shortening the model deployment and optimization cycle; the abnormal mode diagnosis module combines Bayesian posterior inference and density peak clustering, which can not only quantify the risk tendency of a single point using a probability model, but also automatically identify risk clusters based on spatial density. Compared with single clustering or threshold screening, this 'double filtering' strategy can more reliably eliminate incidental noise and accurately locate the real high-risk function point group, thereby improving the reliability and explainability of fault early warning. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 A customized software function point identification and diagnosis system based on big data technology is shown in the accompanying drawings. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the application.

[0052] Embodiment one:

[0053] Referring to Figure 1 A customized software function point identification and diagnosis system based on big data technology is shown in the accompanying drawings.

[0054] The full-amount data acquisition module is used to extract original data from multi-source system logs, code repositories, user interaction records and demand management platforms in real time to obtain a software usage behavior data set;

[0055] The full-amount data acquisition module further includes a pluggable acquisition adapter for dynamically loading acquisition schemes of different system architectures;

[0056] The intelligent data preprocessing module is used to clean, deduplicate, format normalize and deeply semantically annotate the data set to obtain a multi-modal data set;

[0057] The intelligent data preprocessing module uses a deep mapping network to automatically interface log fields and demand models, and supports semantic unification of multi-lingual logs;

[0058] The function feature extraction module is configured to extract semantic, frequency and correlation features based on the multi-modal data set and by constructing a function call graph and applying a graph neural network through a graph database to obtain a function feature vector. The function call graph constructed by the function feature extraction module contains cross-process and cross-service call edges and simultaneously fuses node and edge features based on the graph neural network.

[0059] The dynamic weight adaptive identification module is configured to assign adaptive weights to the function feature vector in combination with project domain labels and runtime feedback through an online meta-learning algorithm to obtain a function point importance score.

[0060] The dynamic weight adaptive identification module assigns adaptive weights through the online meta-learning algorithm.

[0061] The online meta-learning algorithm is based on a few-shot update mechanism to realize rapid adaptation of weight parameters to new project scenarios. The formula of the online meta-learning algorithm is as follows:

[0062]

[0063] C: The function feature vector fuses multi-dimensional information such as semantic features, call frequencies and cross-module correlations and is used to represent the call behavior of a certain function point.

[0064] wk: The importance weight vector of each feature dimension at the kth iteration.

[0065] α: Meta-learning rate (step size) that controls the update amplitude at a time. It can be set to 0.001-0.01 in the scene to balance the convergence speed and stability.

[0066] Ltask(·): Task loss function composed of two parts:

[0067] Historical fault prediction error: enables the model to reproduce the distribution of known risk points.

[0068] Domain label consistency error: ensures that the model output matches the project domain label (such as "finance" and "e-commerce").

[0069] fw(C): Weighted scoring function with input feature vector and output function point importance score.

[0070] When a new project is accessed, a new weight wk+1 can be quickly obtained through online update with only a small amount of labeled fault samples, avoiding large-scale training from scratch.

[0071] The abnormal pattern diagnosis module is configured to perform Bayesian inference and density peak clustering analysis on the function point importance score and the historical fault library to locate risk function points and generate a risk function point list.

[0072] The abnormal pattern diagnosis module captures asynchronous call timing through a temporal Bayesian network, and identifies clustered abnormal points using density peak clustering;

[0073] The formula of the temporal Bayesian network is as follows:

[0074]

[0075] wherein, Ri: event "function point i is a risk point";

[0076] Di: importance score of the function point (derived from the output of formula one);

[0077] P(Ri): prior risk probability, statistically derived from historical failure library (for example, average failure rate of a certain type of function);

[0078] P(Di|Ri): probability distribution of occurrence of score Di under the condition that the risk is true;

[0079] The risk tendency of each function point under the current score is calculated, which is used for preliminary screening of high-risk candidates;

[0080] The formula of the density peak clustering is as follows:

[0081]

[0082] wherein, dij: distance between function points i and j in the score space (absolute difference or cosine distance in the scenario);

[0083] dc: threshold value of the cut-off distance, generally 5% of the maximum possible value of the score;

[0084] χ(x): indicator function, 1 if x<0, otherwise 0;

[0085] pi: local density of the ith point, the greater the value, the higher the density of the nearby points;

[0086] δi: the minimum distance from the ith point to the next high-density cluster center, the greater the value, the farther away from the next high-density cluster center;

[0087] pi and δi are used together to identify "density peaks", i.e., the most significant risk cluster centers in the score space; they can effectively distinguish isolated failure points caused by asynchronous calls from real risk clusters

[0088] The decision support and optimization suggestion module is used for gray box testing of the list in a simulation sandbox environment, evaluation of repair cost and regression risk, and output of a priority repair scheme;

[0089] The decision support and optimization suggestion module follows a repair-verification closed-loop strategy, providing real-time feedback on test results in the simulation sandbox and dynamically adjusting the priority of subsequent repairs.

[0090] As shown above, by using a full-data acquisition module to aggregate multi-dimensional information from system logs, code repositories, user interactions, and requirement management in real time, and performing deep semantic annotation during the intelligent preprocessing stage, this solution can extract more representative multi-modal features based on a richer functional context. This feature fusion significantly improves the comprehensiveness and accuracy of functional point identification compared to single-source methods, thus more reliably reflecting the actual operating status and business value.

[0091] Example 2:

[0092] E-commerce platform function point diagnosis:

[0093] Environment and data acquisition platform scale: 500,000 peak concurrent users per day, 120 microservice modules.

[0094] Data source:

[0095] System logs: Collected 5,000,000 logs from the last 30 days (including request timestamps, module IDs, and interface execution time).

[0096] Code repository: 12,000 Git commit records, extracting module change statistics for each commit.

[0097] User interaction: 200,000 front-end click tracking points to record the frequency of function usage.

[0098] Requirement Management: 3000 Jira tickets, including function tags and priorities. Step 1: Full Data Collection (corresponding to claim 1.1) Pluggable adapters pull data in real time from logs (Fluentd), Git (Webhook), front-end event tracking (self-developed SDK), and Jira (API);

[0099] The "Software Usage Behavior Dataset" was obtained, with a total of approximately 5.412M entries.

[0100] Step 2: Intelligent data preprocessing, cleaning, and deduplication: Duplicate requests were removed from the logs, accounting for 2% of the total, leaving 4.9MB.

[0101] Semantic annotation: Based on a deep mapping network, log fields are mapped to function tags, and a total of 85 function points are identified;

[0102] Output: A multimodal dataset containing 4.9 million structured records and 3000 requirement texts. Step 3: Functional feature extraction and construction of a call graph: Nodes = 85 functional points, Edges = 420 call relationships;

[0103] Graph neural network:

[0104] Input features: node dimension 3 (semantic, frequency, module dependency);

[0105] GNN layers: 2, hidden units 64, training Epoch = 50;

[0106] Output: functional feature vector, dimension 85 x 64.

[0107] Step 4: Dynamic weight adaptive identification (corresponding to claim 1.4 & from claim 5) Initialize weight: w0 = 164;

[0108] Meta-learning rate: a = 0.005;

[0109] Few-shot update: use the last 20 fault tickets to calculate Iterate k = 10 times, output weight vector w10;

[0110] Score: D = fw10(C), normalized to [0, 100].

[0111] Step 5: Abnormal pattern diagnosis Bayesian posterior:

[0112] Prior failure rate P(Ri): based on half-year historical failure statistics (mean 3%).

[0113] Calculate posterior P(Ri|Di), and filter out 10 functional points with posterior greater than 0.5.

[0114] Density peak clustering:

[0115] Distance threshold dc = 5 (5% of the maximum value 100 in the score space);

[0116] Calculate each point ρi, δi, identify 3 cluster centers, and finally locate the high-risk cluster - "order payment", "coupon cancellation", "inventory locking" three major functional points.

[0117] Step 6: Decision support and optimization suggestions (corresponding to claim 1.6) Simulation sandbox: gray test on the above 3 functional points;

[0118] Repair cost: according to the average daily fee of ￥3000 / person, the evaluation is shown in Table 1:

[0119] Table 1

[0120]

[0121] Priority plan:

[0122] First repair "coupon cancellation" (lowest cost, smallest risk);

[0123] The second revision is "Order Payment";

[0124] Finally, fix "inventory lock".

[0125] As shown above, by constructing a function call graph using a graph database and introducing a graph neural network, this solution is the first to simultaneously consider node attributes and cross-module call relationships in the field of function point diagnosis. Compared to traditional linear or tree-like analysis, graph neural networks can capture deep-seated module dependencies and interaction patterns, providing more accurate feature characterization and identification of potential functional coupling risks in complex microservice architectures, laying a solid foundation for subsequent weight allocation and risk diagnosis.

[0126] Example 3:

[0127] Diagnostic analysis of financial trading system functions:

[0128] Environment and data acquisition system scale: peak transaction concurrency of 10,000 transactions / minute, 75 microservices.

[0129] Data source:

[0130] System log entries: 3,600,000 (1 month);

[0131] 8000 Git commit records;

[0132] There are 600,000 transaction records on the client terminal;

[0133] There are 1200 work orders.

[0134] Step 1: Collect all data from Grafana, Git (GitLab API), client SDK, and Jira, resulting in a dataset of approximately 4.2 million records.

[0135] Step 2: After intelligent data preprocessing and cleaning, the logs are 3.5M records, with approximately 3% deduplication.

[0136] The dataset contains 65 semantically labeled functionalities.

[0137] Step 3: Extracting functional features from the function call graph: 65 nodes, 220 edges;

[0138] GNN configuration: 3 layers × 128 hidden units, Epoch = 40;

[0139] Output feature vector, 65×128.

[0140] Step 4: Dynamic weight adaptive identification w0=1128, α=0.003, using the most recent 30 fault samples to iterate 8 times to obtain w8;

[0141] Score D∈[0,1], normalized to [0,100].

[0142] Step 5: Abnormal pattern diagnosis Bayesian posterior:

[0143] Prior failure rate 2.5%;

[0144] Posterior screening out 12 high-risk points.

[0145] Density peak clustering: dc=4;

[0146] Four risk cluster centers are identified, corresponding to "transaction matching", "risk control verification", "acknowledgment notification", and "settlement clearing" function points. Step 6: Decision support and optimization suggestion simulation sandbox gray test results are shown in Table 2 as follows:

[0147] Table 2

[0148] Function point Estimated man-day Cost (Yuan) Regression risk (%) Risk control check 4 12000 2.0 Transaction matching 6 18000 3.2 Notice of return 2 6000 1.5 Settlement clearing 5 15000 2.8

[0149] Repair order:

[0150] Acknowledgment notification→2) Risk control verification→3) Settlement clearing→4) Transaction matching.

[0151] The technical effect comparison is shown in Table 3 as follows:

[0152] Table 3

[0153]

[0154] As can be seen from the above, through the online meta-learning algorithm in the dynamic weight adaptive identification module, the system can quickly update the weight parameters based on a small amount of fault samples, without the need for large-scale offline training, so as to maintain high identification performance in new project or new version scenarios. This small-sample rapid adaptation capability effectively solves the generalization bottleneck of traditional models when facing frequent iterations and scene diversity, significantly shortening the model deployment and optimization cycle;

[0155] The abnormal pattern diagnosis module combines Bayesian posterior inference and density peak clustering, which not only quantifies the risk tendency of a single point using a probability model, but also automatically identifies risk clusters based on spatial density. This "double filtering" strategy can more reliably eliminate incidental noise and accurately locate the real high-risk function point group compared to single clustering or threshold screening, improving the credibility and explainability of fault early warning.

[0156] Although embodiments of the present application have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A customized software function point identification diagnosis system based on big data technology, characterized in that, The application relates to a software risk diagnosis and optimization system, which comprises the following modules: a full-data acquisition module for extracting original data from multi-source system logs, code repositories, user interaction records and demand management platforms to obtain a software usage behavior dataset; an intelligent data preprocessing module for cleaning, deduplicating, format normalizing and deep semantic annotating the dataset to obtain a multi-modal dataset; a functional feature extraction module for constructing a function call graph based on the multi-modal dataset and applying a graph neural network to extract semantic, frequency and correlation features to obtain a functional feature vector; a dynamic weight adaptive identification module for assigning adaptive weights to the functional feature vector in combination with project domain labels and runtime feedback to obtain a functional point importance score; an abnormal pattern diagnosis module for analyzing the functional point importance score and a historical fault library to locate risk functional points and generate a risk functional point list; a decision support and optimization suggestion module for performing gray box testing on the list in a simulation sandbox environment, evaluating repair costs and regression risks and outputting a priority repair scheme.

2. The customized software function point identification and diagnosis system based on big data technology according to claim 1, characterized in that, The dynamic weight adaptive identification module assigns adaptive weights through an online meta-learning algorithm; The online meta-learning algorithm is based on a few-shot updating mechanism to realize rapid adaptation of weight parameters to new project scenarios, and the formula of the online meta-learning algorithm is as follows: wherein C represents a functional feature vector that integrates semantic features, call frequency and cross-module correlation multi-dimensional information and is used to represent the call behavior of a functional point; wk represents an importance weight vector of each feature dimension at the kth iteration; alpha represents a meta-learning rate that controls the updating amplitude; Ltask(·) represents a task loss function that is composed of two parts: a historical fault prediction error that enables the model to reproduce the distribution of known risk points and a domain label consistency error that ensures that the model output matches the project domain label; and fw(C) represents a scoring function with weight distribution, which inputs a feature vector and outputs a functional point importance score. The abnormal pattern diagnosis module captures asynchronous call timing through a time series Bayesian network and identifies clustered abnormal points by using a density peak clustering method; The formula of the time series Bayesian network is as follows: wherein Ri represents an event "functional point i is a risk point"; Di represents the importance score of the functional point; P(Ri) represents a prior risk probability that is statistically obtained from a historical fault library; and P(Di|Ri) represents a probability distribution of the score Di under the condition that the risk is true. The formula of the density peak clustering method is as follows: wherein dij represents the distance between functional points i and j in the score space; dc represents a threshold distance value, which is generally 5% of the maximum possible value of the score; chi(x) represents an indicator function, which is 1 when x is less than 0, and 0 otherwise; pi represents the local density of the ith point, and a larger value indicates a higher density of the nearby points; and delta i represents the minimum distance from the ith point to the center of the next high-density cluster, and a larger value indicates that the ith point is farther away from the center of the next high-density cluster; pi and delta i are used together to identify "density peaks", that is, the most significant risk cluster centers in the score space, and can effectively distinguish isolated fault points caused by asynchronous calls from real risk clusters. ​ 3.The customized software function point identification and diagnosis system based on big data technology according to claim 1, wherein, ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ 4.The customized software function point identification and diagnosis system based on big data technology according to claim 1, wherein, The full-amount data acquisition module further comprises a pluggable acquisition adapter for dynamically loading acquisition schemes of different system architectures. 5.The customized software function point identification and diagnosis system based on big data technology according to claim 1, wherein, The intelligent data preprocessing module automatically connects log fields and demand models by using a deep mapping network, and supports semantic unification of multi-lingual logs. 6.The customized software function point identification and diagnosis system based on big data technology according to claim 1, wherein, The function feature extraction module constructs a function call graph containing cross-process and cross-service call edges, and simultaneously fuses node and edge features based on a graph neural network. 7.The customized software function point identification and diagnosis system based on big data technology according to claim 1, wherein, The decision support and optimization suggestion module dynamically adjusts the subsequent repair priority in accordance with a repair-verification closed-loop strategy, and feeds back test results in a simulation sandbox in real time.

Images