An ai-driven automatic identification and cleaning system for software bloat technical debt

Abstract

The application relates to the technical field of data processing, in particular to an AI-driven software useless function technical debt automatic identification and cleaning system, which comprises a data acquisition module, a data screening module, a first cleaning decision generation module, a second cleaning decision generation module and a cleaning execution module. Code running data is acquired, multi-dimensional data analysis is carried out, an intelligent identification model is constructed, cleaning decisions are generated, the decisions are optimized through multi-layer security verification, and automatic code cleaning is executed. The identification process is continuously optimized through a feedback mechanism, accurate identification and safe cleaning can be realized, and the technical debt management efficiency and system stability are improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an AI-driven system for automatically identifying and cleaning up software deadweights. Background Technology

[0002] In the long-term iteration and evolution of large-scale software systems, invalid or redundant code, known as "technical debt," accumulates due to feature decommissioning and requirement changes. This useless code not only increases system complexity and maintenance costs but also harbors security risks. Currently, identifying useless functions mainly relies on manual review by developers or static scanning tools based on simple rules (such as zero calls). The former is inefficient and highly dependent on personal experience; the latter, due to its failure to fully consider static dependencies, historical call seasonality (such as annual report functions), and relevance to core business modules, has a high risk of misjudgment and cannot make reliable decisions in complex production environments.

[0003] Chinese patent application CN106254279B discloses an automatic migration system and management method for virtual network elements based on SDN (Software-Defined Networking). The system utilizes software-defined networking to provide network connectivity and includes virtual switches, an SDN controller, and virtual network elements. Upon receiving a virtual network element removal event reported by the source host's virtual switch, the SDN controller adjusts the virtual switch's configuration rules to maintain network connectivity during the migration process. It also cleans up the configuration rules of the moved-out virtual network elements from the source host's virtual switch and issues the configuration rules for the moved-in virtual network elements to the target host's virtual switch. Based on software-defined networking, the system eliminates barriers between different virtual switches and employs a simple system management method. The SDN controller automatically cleans up useless configurations left over from the migration process and automatically maintains the connection status between virtual network elements, ensuring uninterrupted network connectivity during the migration process and constructing a highly efficient control platform. However, this solution still has the problems of being unable to adapt to the seasonality of code, failing to consider the correlation of code usage, relying mainly on manual cleanup, which leads to the accumulation of useless data, resulting in low accuracy and efficiency in the automatic identification and cleanup of useless software functional technical debt. Summary of the Invention

[0004] To address this, the present invention provides an AI (Artificial Intelligence) driven automatic identification and cleanup system for useless software functional technical debt, which overcomes the problems of existing technologies that cannot adapt to the seasonality of code, do not consider the correlation of code usage, and mainly rely on manual cleanup, leading to the accumulation of useless data and resulting in low accuracy and efficiency of automatic identification and cleanup of useless software functional technical debt.

[0005] To achieve the above objectives, the present invention provides an AI-driven system for automatically identifying and clearing software deadweight technology debt, comprising:

[0006] The data acquisition module is used to collect data from code execution.

[0007] The data filtering module is used to perform initial screening of code execution data based on the zero-traffic initial screening method to obtain the initial screening code execution data. It is also used to filter the initial screening code execution data based on the static dependency filtering method to obtain the filtered code execution data. Furthermore, it is used to generate a list of functions to be reviewed based on the function to be reviewed list generation method and the filtered code execution data.

[0008] The first cleanup decision generation module is used to generate function search results based on the list of functions to be reviewed. The function search results include useless function search results and code search results. The first cleanup decision is generated based on the code search results.

[0009] The second cleanup decision generation module is used to judge the authenticity of useless functions based on the useless function retrieval results, and generate a second cleanup decision based on the judgment results. It is also used to seasonally optimize the judgment process of the authenticity of useless functions, and to judge the code correlation, and to adjust the correlation of the seasonal optimization process based on the judgment results of the code correlation.

[0010] The cleanup execution module is used to perform code cleanup based on the first cleanup decision and the second cleanup decision.

[0011] Furthermore, the zero-flow initial screening method includes:

[0012] Step S11: Set a continuous historical time window and a traffic threshold;

[0013] Step S12: Within the historical time window, preliminary candidate functions are marked based on the number of calls and traffic thresholds of each function in the code execution data, wherein:

[0014] Functions with fewer calls than the traffic threshold are marked as initial candidate functions.

[0015] Functions whose call count exceeds the traffic threshold will not be flagged.

[0016] Functions whose call count equals the traffic threshold are not marked;

[0017] Step S13: Collect the functions marked as preliminary candidate functions to obtain the code execution data after preliminary screening.

[0018] Furthermore, the static dependency filtering method includes:

[0019] Step S21: Perform static scanning on the codebase to obtain the static scanning results, and construct a global code call relationship graph based on the static scanning results;

[0020] Step S22: Identify the nodes of the preliminary candidate functions in the code execution data after preliminary screening in the global code call relationship graph;

[0021] Step S23: Analyze the dependency relationship of the node. If there are other active call links besides the system startup initialization path, the preliminary candidate function is classified as a useless function. If there are no other active call links besides the system startup initialization path, the preliminary candidate function is classified as dead code.

[0022] Step S24: The retained preliminary candidate functions are output as the filtered code execution data.

[0023] Furthermore, the method for generating the list of functions to be reviewed, executed in the data filtering module, includes:

[0024] Step S31: Read the filtered code execution data to obtain all candidate function identifiers;

[0025] Step S32: Based on the candidate function identifier, automatically extract its corresponding code file path, class name, method name, and category;

[0026] Step S33: Output the candidate function identifier and the corresponding code file path, class name, method name and category as a list of functions to be reviewed.

[0027] Furthermore, the first cleanup decision generation module generates a function retrieval result based on the list of functions to be reviewed. The function retrieval result includes useless function retrieval results and code retrieval results. The module then generates a first cleanup decision based on the code retrieval results, specifically including:

[0028] Step R1: Collect all candidate function identifiers classified as dead code in the list of functions pending review, along with the corresponding code file paths, class names, and method names, as a dead code list. Collect all candidate function identifiers classified as useless functions in the list of functions pending review, along with the corresponding code file paths, class names, and method names, as a useless function list.

[0029] Step R2: Use the list of dead code as the code retrieval result and the list of useless functions as the function retrieval result;

[0030] The cleanup decision module outputs the code to be deleted from the code retrieval results as the first cleanup decision.

[0031] Furthermore, the second cleanup decision generation module constructs a useless function retrieval confidence model based on the useless function retrieval confidence judgment model construction method, wherein the useless function retrieval confidence model construction method includes:

[0032] Step M11: Collect historical code cleanup data as training samples. The historical code cleanup data includes historically acquired multidimensional feature vectors and the useless function confidence corresponding to the historically acquired multidimensional feature vectors.

[0033] Step M12: Use the training samples to train the XGBoost classifier to obtain the useless function retrieval confidence model.

[0034] Furthermore, the second cleanup decision generation module determines the authenticity of useless functions based on the useless function retrieval results, and generates a second cleanup decision based on the determination result, and also includes:

[0035] Step M21: Extract the static and dynamic features of the useless functions from the useless function retrieval results, and use the static and dynamic features of the useless functions as the multidimensional feature vector of the useless functions.

[0036] Step M22: Input the multidimensional feature vector into the useless function retrieval confidence model;

[0037] Step M23: Obtain the useless function confidence score P output by the useless function retrieval confidence model, compare the useless function confidence score P with the preset useless function confidence score P0, determine the authenticity of the useless function based on the comparison result, and generate the second cleanup decision based on the determination result, wherein:

[0038] When P≥P0, the second cleanup decision generation module determines that the authenticity of the useless function is real and outputs the code for deleting the search results of the useless function as the second cleanup decision.

[0039] When P < P0, the second cleanup decision generation module determines that the authenticity of the useless function is not true, and outputs the code that retains the retrieval result of the useless function as the second cleanup decision.

[0040] Furthermore, the second cleanup decision generation module seasonally optimizes the process for determining the authenticity of useless functions, including:

[0041] The seasonal judgment model is constructed according to the seasonal judgment model construction method, which includes:

[0042] Step T11: Collect the historical useless function retrieval results and the corresponding seasonal judgment results, and use them as the training dataset for the seasonal judgment model.

[0043] Step T12: Select the random forest classifier as the base model for the seasonality judgment model;

[0044] Step T13: Use the seasonal judgment model training dataset to train the random forest classifier to obtain the seasonal judgment model;

[0045] The second cleanup decision generation module inputs the useless function retrieval results into the seasonal judgment model, obtains the seasonal judgment results output by the seasonal judgment model, the seasonal judgment results include those that are seasonal and those that are not, and performs seasonal optimization on the judgment process of the authenticity of useless functions based on the seasonal judgment results, wherein:

[0046] When the seasonality judgment result is seasonal, the second cleanup decision generation module performs seasonal optimization on the judgment process of the authenticity of useless functions. The seasonal optimization includes: seasonally optimizing the preset useless function confidence P0 through the seasonal optimization coefficient α to obtain the optimized preset useless function confidence P0y. P0y is set to α×P0, α=1.15. The value of the preset useless function confidence P0 is replaced with the value of the optimized preset useless function confidence P0y. The useless function confidence P is then compared with the preset useless function confidence P0 again.

[0047] When the seasonality judgment result is that it is not seasonal, the second cleanup decision generation module does not perform seasonal optimization on the judgment process of the authenticity of useless functions.

[0048] Furthermore, the code correlation judgment described in the second cleanup decision generation module includes:

[0049] Step C11: For each useless function in the useless function retrieval results, based on the global code call relationship graph constructed by the static dependency filtering method, use the breadth-first search algorithm to calculate the shortest call path length between the useless function and the predefined core business function set.

[0050] Step C12: Compare the calculated shortest call path length with a preset hop count threshold. When the shortest call path length is less than or equal to the preset hop count threshold, it is determined that the useless function has code association. When the shortest call path length is greater than the preset hop count threshold, it is determined that the useless function does not have code association.

[0051] The process of adjusting seasonal optimization based on code correlation judgment results, as described in the second cleanup decision generation module, includes:

[0052] When it is determined that the function has code correlation, the second cleanup decision generation module adjusts the correlation of the seasonal optimization process, cancels the seasonal optimization, and adjusts the preset useless function confidence P0 according to the correlation adjustment coefficient β to obtain the adjusted preset useless function confidence P0t. P0t is set to β×P0, β=1.20. The value of the preset useless function confidence P0 is replaced with the value of the adjusted preset useless function confidence P0t, and the useless function confidence P is compared with the preset useless function confidence P0 again.

[0053] When it is determined that the function does not have code relevance, the second cleanup decision generation module will not make any relevance adjustments to the seasonal optimization process.

[0054] Furthermore, the code cleanup performed in the cleanup execution module based on the first cleanup decision and the second cleanup decision includes:

[0055] The first and second cleanup decisions are sent to the display terminal, and the first and second cleanup decisions are manually verified on the display terminal to obtain the verified target cleanup code, and the verified target cleanup code is cleaned.

[0056] Compared with existing technologies, the beneficial effects of this invention are as follows: the system replaces the traditional method of relying mainly on manual screening through automated pipeline processing of the data acquisition module and the data filtering module, solving the problems of low efficiency and insufficient coverage of manual cleanup from the source. Furthermore, the system introduces a seasonal optimization mechanism in the second cleanup decision generation module, which can intelligently identify functions with seasonal usage characteristics, preventing misjudgment as useless functions during non-use seasons. The system also uses the code correlation judgment function in the second cleanup decision generation module to analyze the correlation between useless functions and core business functions based on a global call relationship graph. The system's automatic decision-making on dead code by the first cleanup decision generation module and intelligent judgment of useless functions by the second cleanup decision generation module significantly reduce the need for manual intervention, thereby improving the efficiency of useless function identification and cleanup. Finally, the system automates the cleanup process while ensuring security through the manual verification function of the cleanup execution module, improving the accuracy and efficiency of identifying software useless function technical debts. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of the AI-driven automatic identification and cleanup system for useless software functionalities in this embodiment.

[0058] Figure 2 This is a schematic diagram of the zero-flow initial screening method in this embodiment;

[0059] Figure 3This is a flowchart illustrating the static dependency filtering method in this embodiment;

[0060] Figure 4 This is a flowchart illustrating the method for constructing a confidence model for useless function retrieval in this embodiment. Detailed Implementation

[0061] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0062] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0063] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0064] Please see Figure 1 As shown, this is a schematic diagram of the structure of the AI-driven automatic identification and cleanup system for useless software functionalities in this embodiment. The system includes:

[0065] The data acquisition module is used to collect data from code execution.

[0066] The data filtering module is used to perform initial screening of code execution data according to the zero-flow initial screening method to obtain the initial screening code execution data. It is also used to filter the initial screening code execution data according to the static dependency filtering method to obtain the filtered code execution data. Furthermore, it is used to generate a list of functions to be reviewed based on the function to be reviewed list generation method and the filtered code execution data. The data filtering module is connected to the data acquisition module.

[0067] The first cleanup decision generation module is used to generate a first cleanup decision based on the function retrieval results according to the list of functions to be reviewed. The function retrieval results include useless function retrieval results and code retrieval results. The first cleanup decision generation module is connected to the data filtering module.

[0068] The second cleanup decision generation module is used to judge the authenticity of useless functions based on the useless function retrieval results, and generate a second cleanup decision based on the judgment results. It is also used to seasonally optimize the judgment process of the authenticity of useless functions, and to judge the code correlation, and adjust the correlation of the seasonal optimization process based on the judgment result of the code correlation. The second cleanup decision generation module is connected to the first cleanup decision generation module.

[0069] The cleanup execution module is used to perform code cleanup based on the first cleanup decision and the second cleanup decision, and the cleanup execution module is connected to the second cleanup decision generation module.

[0070] Specifically, the AI-driven automatic identification and cleanup system for useless software functionalities is applied to the technical debt governance terminal of the software system. By collecting code execution data, performing multi-dimensional data analysis, constructing an intelligent identification model to generate cleanup decisions, optimizing decisions through multi-layered security verification, and executing automated code cleanup, the system also continuously optimizes the identification process through a feedback mechanism. This achieves accurate identification and secure cleanup, improving the efficiency and stability of technical debt governance. The system replaces the traditional method relying primarily on manual screening with automated pipeline processing of data acquisition and data filtering modules, fundamentally solving the problems of low efficiency and insufficient coverage of manual cleanup. Furthermore, the system introduces a second cleanup decision generation module... The system employs a seasonal optimization mechanism that intelligently identifies functions with seasonal usage characteristics, preventing them from being misjudged as useless during off-seasons. Furthermore, the system utilizes a code correlation judgment function in the second cleanup decision generation module to analyze the correlation between useless functions and core business functions based on a global call relationship graph. The system also significantly reduces the need for manual intervention through automatic decision-making on dead code in the first cleanup decision generation module and intelligent judgment of useless functions in the second cleanup decision generation module, thereby improving the efficiency of useless function identification and cleanup. Finally, the system automates the cleanup process while ensuring security through the manual verification function of the cleanup execution module, thus improving the accuracy and efficiency of identifying software useless function technical debt.

[0071] Specifically, the code execution data includes the number of calls and the code repository. The code repository refers to the central repository that stores and manages all source code of the software project. The data acquisition module collects code repository information through the Git command-line tool and the JGit library. The number of calls refers to the number of times a certain function is called. The data acquisition module collects the number of function calls at the JVM level through bytecode enhancement using Java Agent technology.

[0072] Specifically, the method for generating the list of functions to be reviewed executed in the data filtering module includes:

[0073] Step S31: Read the filtered code execution data to obtain all candidate function identifiers;

[0074] Step S32: Based on the candidate function identifier, automatically extract its corresponding code file path, class name, method name, and category;

[0075] Step S33: Output the candidate function identifier and the corresponding code file path, class name, method name and category as a list of functions to be reviewed.

[0076] Specifically, the candidate function identifier refers to a string that uniquely identifies the initial candidate function, including the code file path, class name, method name, and classification. The code file path refers to the unique location identifier of the source code file in the version control system or project directory structure. The class name refers to the unique name used to identify a class or interface in an object-oriented programming language. The method name refers to the identifier name of the method or function defined in the class. The classification refers to the useless functions and dead code that are classified into the initial candidate functions in the static dependency method.

[0077] Specifically, the first cleanup decision generation module generates a function retrieval result based on the list of functions to be reviewed. The function retrieval result includes useless function retrieval results and code retrieval results. The module then generates a first cleanup decision based on the code retrieval results, specifically including:

[0078] Step R1: Collect all candidate function identifiers classified as dead code in the list of functions pending review, along with the corresponding code file paths, class names, and method names, as a dead code list. Collect all candidate function identifiers classified as useless functions in the list of functions pending review, along with the corresponding code file paths, class names, and method names, as a useless function list.

[0079] Step R2: Use the list of dead code as the code retrieval result and the list of useless functions as the function retrieval result;

[0080] The cleanup decision module outputs the code to be deleted from the code retrieval results as the first cleanup decision.

[0081] Specifically, dead code refers to code segments that exist in the codebase but cannot be triggered for execution in any actual runtime environment.

[0082] Specifically, the second cleanup decision generation module determines the authenticity of useless functions based on the useless function retrieval results, and generates a second cleanup decision based on the determination result, and also includes:

[0083] Step M21: Extract the static and dynamic features of the useless functions from the useless function retrieval results, and use the static and dynamic features of the useless functions as the multidimensional feature vector of the useless functions.

[0084] Step M22: Input the multidimensional feature vector into the useless function retrieval confidence model;

[0085] Step M23: Obtain the useless function confidence score P output by the useless function retrieval confidence model, compare the useless function confidence score P with the preset useless function confidence score P0, determine the authenticity of the useless function based on the comparison result, and generate the second cleanup decision based on the determination result, wherein:

[0086] When P≥P0, the second cleanup decision generation module determines that the authenticity of the useless function is real and outputs the code for deleting the search results of the useless function as the second cleanup decision.

[0087] When P < P0, the second cleanup decision generation module determines that the authenticity of the useless function is not true, and outputs the code that retains the retrieval result of the useless function as the second cleanup decision.

[0088] Specifically, this embodiment extracts static features of useless functions using the static code analysis tool Checkstyle. These static features include the number of lines of code, cyclomatic complexity, number of parameters, inheritance depth, and comment rate. It also extracts dynamic features of useless functions using an application performance monitoring system. These dynamic features include the number of calls in the past N days / months, the diversity of call source IPs / services, average execution time, and the most recent call time. The useless function confidence score refers to the probability value output by the useless function retrieval confidence model, indicating that the function is useless, and its value range is [0, 1]. The preset useless function confidence score is a preset value used to judge the status of useless functions; in this embodiment, P0 = 0.65. The useless function authenticity refers to whether the useless function is truly useless, as judged based on the useless function confidence score and the preset useless function confidence score. The useless function authenticity includes both true and false.

[0089] Specifically, the second cleanup decision generation module seasonally optimizes the process for determining the authenticity of useless functions, including:

[0090] The seasonal judgment model is constructed according to the seasonal judgment model construction method, which includes:

[0091] Step T11: Collect the historical useless function retrieval results and the corresponding seasonal judgment results, and use them as the training dataset for the seasonal judgment model.

[0092] Step T12: Select the random forest classifier as the base model for the seasonality judgment model;

[0093] Step T13: Use the seasonal judgment model training dataset to train the random forest classifier to obtain the seasonal judgment model.

[0094] Specifically, this embodiment does not limit the specific implementation method of training the random forest classifier using the seasonal judgment model training dataset. Those skilled in the art can set it according to the actual situation, such as dividing the seasonal judgment model training dataset into 50% judgment training set, 20% judgment test set and 30% judgment validation set, training, testing and validating the random forest classifier, and outputting the random forest classifier with an accuracy of 90% after validation as the seasonal judgment model.

[0095] Specifically, the second cleanup decision generation module inputs the useless function retrieval results into the seasonal judgment model, obtains the seasonal judgment results output by the seasonal judgment model, the seasonal judgment results include those that are seasonal and those that are not, and performs seasonal optimization on the judgment process of the authenticity of useless functions based on the seasonal judgment results, wherein:

[0096] When the seasonality judgment result is seasonal, the second cleanup decision generation module performs seasonal optimization on the judgment process of the authenticity of useless functions. The seasonal optimization includes: seasonally optimizing the preset useless function confidence P0 through the seasonal optimization coefficient α to obtain the optimized preset useless function confidence P0y. P0y is set to α×P0, α=1.15. The value of the preset useless function confidence P0 is replaced with the value of the optimized preset useless function confidence P0y. The useless function confidence P is then compared with the preset useless function confidence P0 again.

[0097] When the seasonality judgment result is that it is not seasonal, the second cleanup decision generation module does not perform seasonal optimization on the judgment process of the authenticity of useless functions.

[0098] Specifically, setting α=1.15 is based on the fact that when the seasonality of a function is judged to be seasonal, the probability of it being a useless function is lower. At this time, by increasing the preset confidence of useless functions to 1.15 times, the difficulty of judging the authenticity of useless functions is increased, thus avoiding the deletion of seasonal functions.

[0099] Specifically, the code correlation judgment in the second cleanup decision generation module includes:

[0100] Step C11: For each useless function in the useless function retrieval results, based on the global code call relationship graph constructed by the static dependency filtering method, use the breadth-first search algorithm to calculate the shortest call path length between the useless function and the predefined core business function set.

[0101] Step C12: Compare the calculated shortest call path length with a preset hop count threshold. When the shortest call path length is less than or equal to the preset hop count threshold, it is determined that the useless function has code association. When the shortest call path length is greater than the preset hop count threshold, it is determined that the useless function does not have code association.

[0102] The process of adjusting seasonal optimization based on code correlation judgment results, as described in the second cleanup decision generation module, includes:

[0103] When it is determined that the function has code correlation, the second cleanup decision generation module adjusts the correlation of the seasonal optimization process, cancels the seasonal optimization, and adjusts the preset useless function confidence P0 according to the correlation adjustment coefficient β to obtain the adjusted preset useless function confidence P0t. P0t is set to β×P0, β=1.20. The value of the preset useless function confidence P0 is replaced with the value of the adjusted preset useless function confidence P0t, and the useless function confidence P is compared with the preset useless function confidence P0 again.

[0104] When it is determined that the function does not have code relevance, the second cleanup decision generation module will not make any relevance adjustments to the seasonal optimization process.

[0105] Specifically, the breadth-first search algorithm is a queue-based graph traversal algorithm used to systematically explore the connections between nodes in a code call graph. In this embodiment, the traversal depth of the breadth-first search algorithm is limited to 10 levels. The core business function set refers to the set of functional modules in the system that implement key business processes and core value. In this embodiment, the core business function set is obtained by marking functions that directly participate in revenue generation or significantly affect revenue scale, key functions whose unavailability will lead to customer complaints or churn, essential functions that meet mandatory legal and regulatory requirements, and basic functions that affect the continuous operation of core business processes as core functions. The shortest call path length refers to the length of a function from a node in the code call graph. The minimum number of calls required for a point to reach another functional node. The preset hop count threshold refers to the maximum allowable call distance threshold set in the code association judgment. The preset hop count threshold is set to 3, which is based on the "three-hop principle" in software engineering. This principle indicates that most meaningful code associations are completed within three calls. Those skilled in the art can adjust this threshold within the range of 2-5 hops according to the specific system architecture characteristics. The setting β=0.2 is to reduce the probability of a function being a useless function when it has code association. At this time, by increasing the preset confidence of useless functions to 1.2 times, the difficulty of judging the authenticity of useless functions is increased, thus avoiding the deletion of important functions with code association.

[0106] Specifically, the code cleanup performed in the cleanup execution module based on the first cleanup decision and the second cleanup decision includes:

[0107] The first and second cleanup decisions are sent to the display terminal, and the first and second cleanup decisions are manually verified on the display terminal to obtain the verified target cleanup code, and the verified target cleanup code is cleaned.

[0108] Specifically, this embodiment does not limit the specific implementation method of sending the first and second cleanup decisions to the display terminal. Those skilled in the art can set it according to the actual situation, such as sending the first and second cleanup decisions to the display terminal through wireless signal transmission. This embodiment does not limit the specific implementation method of manually verifying the first and second cleanup decisions on the display terminal. Those skilled in the art can set it according to the actual situation, such as generating a code deletion preview view for the first and second cleanup decisions, highlighting the code segment to be deleted and its position in the call relationship diagram, and having staff judge whether the code can be deleted and select the code that can be deleted as the target cleanup code after verification. This embodiment does not limit the specific implementation method of cleaning the target cleanup code after verification. Those skilled in the art can set it according to the actual situation, such as generating a code deletion script and executing the code deletion script to accurately remove the target cleanup code after verification.

[0109] Please see Figure 2 As shown, this is a schematic diagram of the zero-flow primary screening method in this embodiment. The zero-flow primary screening method includes:

[0110] Step S11: Set a continuous historical time window and a traffic threshold;

[0111] Step S12: Within the historical time window, preliminary candidate functions are marked based on the number of calls and traffic thresholds of each function in the code execution data, wherein:

[0112] Functions with fewer calls than the traffic threshold are marked as initial candidate functions.

[0113] Functions whose call count exceeds the traffic threshold will not be flagged.

[0114] Functions whose call count equals the traffic threshold are not marked;

[0115] Step S13: Collect the functions marked as preliminary candidate functions to obtain the code execution data after preliminary screening.

[0116] Specifically, the number of calls refers to the number of times a certain function is called; the historical time window refers to a continuous observation period set for function traffic analysis, such as the past 6 months; the traffic threshold refers to the minimum number of calls required to determine whether a function is considered "zero traffic", such as 5 times. When the number of calls to a certain function is lower than this threshold, it is marked as a preliminary candidate function.

[0117] Please see Figure 3 As shown, this is a flowchart illustrating the static dependency filtering method in this embodiment. The static dependency filtering method includes:

[0118] Step S21: Perform static scanning on the codebase to obtain the static scanning results, and construct a global code call relationship graph based on the static scanning results;

[0119] Step S22: Identify the nodes of the preliminary candidate functions in the code execution data after preliminary screening in the global code call relationship graph;

[0120] Step S23: Analyze the dependency relationship of the node. If there are other active call links besides the system startup initialization path, the preliminary candidate function is classified as a useless function. If there are no other active call links besides the system startup initialization path, the preliminary candidate function is classified as dead code.

[0121] Step S24: The retained preliminary candidate functions are output as the filtered code execution data.

[0122] Specifically, this embodiment uses code parsing tools to perform lexical and syntactic analysis on source code files. For Java language projects, the Eclipse JDT (Eclipse Java Development Tools) compiler toolkit is used for parsing. This embodiment constructs the static scanning results into a directed graph structure, where: the graph nodes include: class nodes, interface nodes, method nodes, and field nodes; the graph edges include: inheritance edges, implementation edges, call edges, and reference edges; node attributes include: fully qualified name, modifier, and source code location; edge attributes include: dependency type, call frequency, and source code location; the directed graph structure uses an adjacency list storage method, supporting fast traversal and query operations; the global code call relationship graph refers to a directed graph model reflecting the call and dependency relationships between all classes, methods, and functions, constructed through static analysis of source files or bytecode of the code library; the active call chain refers to the explicit code execution path driven by business logic, excluding the automatic loading and initialization process of the system framework; and the dependent relationship refers to the set of calls and dependencies of other code elements to the current analyzed node in the code call relationship graph.

[0123] Please see Figure 4 As shown, this is a flowchart illustrating the method for constructing a confidence model for useless function retrieval in this embodiment. The method for constructing a confidence model for useless function retrieval includes:

[0124] Step M11: Collect historical code cleanup data as training samples. The historical code cleanup data includes historically acquired multidimensional feature vectors and the useless function confidence corresponding to the historically acquired multidimensional feature vectors.

[0125] Step M12: Use the training samples to train the XGBoost (eXtreme Gradient Boosting) classifier to obtain the useless function retrieval confidence model.

[0126] Specifically, the historical code cleanup data refers to a collection of code cleanup operation records that have been manually reviewed and confirmed in the past. The multidimensional feature vector is a numerical representation used to quantify a functional characteristic. The multidimensional feature vector includes static features and dynamic features. The static features originate from the code itself, such as loop complexity and number of lines of code. The dynamic features originate from system operation behavior, such as call frequency. The XGBoost classifier is a usable gradient boosting decision tree machine learning model. This embodiment does not limit its specific training parameters. Those skilled in the art can set them according to the actual situation. For example, if a tree model is selected as the basic learner, its learning rate is set to 0.1, the maximum depth of the tree is 6, and its accuracy is not less than 95%.

[0127] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. An AI-driven system for automatically identifying and clearing software deadweight technology debt, characterized in that, include: The data acquisition module is used to collect data from code execution. The data filtering module is used to perform initial screening of code execution data based on the zero-traffic initial screening method to obtain the initial screening code execution data. It is also used to filter the initial screening code execution data based on the static dependency filtering method to obtain the filtered code execution data. Furthermore, it is used to generate a list of functions to be reviewed based on the function to be reviewed list generation method and the filtered code execution data. The first cleanup decision generation module is used to generate function search results based on the list of functions to be reviewed. The function search results include useless function search results and code search results. The first cleanup decision is generated based on the code search results. The second cleanup decision generation module is used to judge the authenticity of useless functions based on the useless function retrieval results, and generate a second cleanup decision based on the judgment results. It is also used to seasonally optimize the judgment process of the authenticity of useless functions, and to judge the code correlation, and to adjust the correlation of the seasonal optimization process based on the judgment results of the code correlation. The cleanup execution module is used to perform code cleanup based on the first cleanup decision and the second cleanup decision; The zero-flow primary screening method includes: Step S11: Set a continuous historical time window and a traffic threshold; Step S12: Within the historical time window, preliminary candidate functions are marked based on the number of calls and traffic thresholds of each function in the code execution data, wherein: Functions with fewer calls than the traffic threshold are marked as initial candidate functions. Functions whose call count exceeds the traffic threshold will not be flagged. Functions whose call count equals the traffic threshold are not marked; Step S13: Collect the functions marked as preliminary candidate functions to obtain the code execution data after preliminary screening; The static dependency filtering method includes: Step S21: Perform static scanning on the codebase to obtain the static scanning results, and construct a global code call relationship graph based on the static scanning results; Step S22: Identify the nodes of the preliminary candidate functions in the code execution data after preliminary screening in the global code call relationship graph; Step S23: Analyze the dependency relationship of the node. If there are other active call links besides the system startup initialization path, the preliminary candidate function is classified as a useless function. If there are no other active call links besides the system startup initialization path, the preliminary candidate function is classified as dead code. Step S24: Output the retained preliminary candidate functions as the filtered code execution data; The method for generating the list of functions to be reviewed, executed in the data filtering module, includes: Step S31: Read the filtered code execution data to obtain all candidate function identifiers; Step S32: Based on the candidate function identifier, automatically extract its corresponding code file path, class name, method name, and category; Step S33: Output the candidate function identifier and the corresponding code file path, class name, method name and category as a list of functions to be reviewed; The first cleanup decision generation module generates a function retrieval result based on the list of functions to be reviewed. The function retrieval result includes useless function retrieval results and code retrieval results. The module then generates a first cleanup decision based on the code retrieval results, specifically including: Step R1: Collect all candidate function identifiers classified as dead code in the list of functions pending review, along with the corresponding code file paths, class names, and method names, as a dead code list. Collect all candidate function identifiers classified as useless functions in the list of functions pending review, along with the corresponding code file paths, class names, and method names, as a useless function list. Step R2: Use the list of dead code as the code retrieval result and the list of useless functions as the function retrieval result; The first cleanup decision generation module outputs the code to be deleted from the code retrieval results as the first cleanup decision; The second cleanup decision generation module constructs a useless function retrieval confidence model based on the useless function retrieval confidence judgment model construction method, wherein the useless function retrieval confidence model construction method includes: Step M11: Collect historical code cleanup data as training samples. The historical code cleanup data includes historically acquired multidimensional feature vectors and the useless function confidence corresponding to the historically acquired multidimensional feature vectors. Step M12: Use the training samples to train the XGBoost classifier to obtain the useless function retrieval confidence model; The second cleanup decision generation module determines the authenticity of useless functions based on the useless function retrieval results and generates a second cleanup decision based on the determination result. It also includes: Step M21: Extract the static and dynamic features of the useless functions from the useless function retrieval results, and use the static and dynamic features of the useless functions as the multidimensional feature vector of the useless functions. Step M22: Input the multidimensional feature vector into the useless function retrieval confidence model; Step M23: Obtain the useless function confidence score P output by the useless function retrieval confidence model, compare the useless function confidence score P with the preset useless function confidence score P0, determine the authenticity of the useless function based on the comparison result, and generate the second cleanup decision based on the determination result, wherein: When P≥P0, the second cleanup decision generation module determines that the authenticity of the useless function is real and outputs the code for deleting the search results of the useless function as the second cleanup decision. When P < P0, the second cleanup decision generation module determines that the authenticity of the useless function is not true, and outputs the code that retains the search results of the useless function as the second cleanup decision. The second cleanup decision generation module performs seasonal optimization on the process of judging the authenticity of useless functions, including: The seasonal judgment model is constructed according to the seasonal judgment model construction method, which includes: Step T11: Collect the historical useless function retrieval results and the corresponding seasonal judgment results, and use them as the training dataset for the seasonal judgment model. Step T12: Select the random forest classifier as the base model for the seasonality judgment model; Step T13: Use the seasonal judgment model training dataset to train the random forest classifier to obtain the seasonal judgment model; The second cleanup decision generation module inputs the useless function retrieval results into the seasonal judgment model, obtains the seasonal judgment results output by the seasonal judgment model, the seasonal judgment results include those that are seasonal and those that are not, and performs seasonal optimization on the judgment process of the authenticity of useless functions based on the seasonal judgment results, wherein: When the seasonality judgment result is seasonal, the second cleanup decision generation module performs seasonal optimization on the judgment process of the authenticity of useless functions. The seasonal optimization includes: seasonally optimizing the preset useless function confidence P0 through the seasonal optimization coefficient α to obtain the optimized preset useless function confidence P0y. P0y is set to α×P0, α=1.

15. The value of the preset useless function confidence P0 is replaced with the value of the optimized preset useless function confidence P0y. The useless function confidence P is then compared with the preset useless function confidence P0 again. When the seasonality judgment result is that it is not seasonal, the second cleanup decision generation module does not perform seasonal optimization on the judgment process of the authenticity of useless functions. The code correlation judgment described in the second cleanup decision generation module includes: Step C11: For each useless function in the useless function retrieval results, based on the global code call relationship graph constructed by the static dependency filtering method, use the breadth-first search algorithm to calculate the shortest call path length between the useless function and the predefined core business function set. Step C12: Compare the calculated shortest call path length with a preset hop count threshold. When the shortest call path length is less than or equal to the preset hop count threshold, it is determined that the useless function has code association. When the shortest call path length is greater than the preset hop count threshold, it is determined that the useless function does not have code association. The process of adjusting seasonal optimization based on code correlation judgment results, as described in the second cleanup decision generation module, includes: When it is determined that the function has code correlation, the second cleanup decision generation module adjusts the correlation of the seasonal optimization process, cancels the seasonal optimization, and adjusts the preset useless function confidence P0 according to the correlation adjustment coefficient β to obtain the adjusted preset useless function confidence P0t. P0t is set to β×P0, β=1.

20. The value of the preset useless function confidence P0 is replaced with the value of the adjusted preset useless function confidence P0t, and the useless function confidence P is compared with the preset useless function confidence P0 again. When it is determined that the function does not have code relevance, the second cleanup decision generation module will not make any relevance adjustments to the seasonal optimization process.

2. The AI-driven automatic identification and cleanup system for useless software functionalities as described in claim 1, characterized in that, The code cleanup performed in the cleanup execution module based on the first cleanup decision and the second cleanup decision includes: The first and second cleanup decisions are sent to the display terminal, and the first and second cleanup decisions are manually verified on the display terminal to obtain the verified target cleanup code, and the verified target cleanup code is cleaned.

Images