A machine learning-based code defect detection method, device and medium

Abstract

The application discloses a code defect detection method and device based on machine learning and a medium, relates to the technical field of code detection, and comprises the following steps: collecting protection statements and dangerous operation statements in a current submission change function, determining verification protection statements, and generating source code protection relations; generating production construction products according to production construction parameters, replacing a condition with an opposite condition, and generating reverse verification construction products; performing identity verification on dangerous operations of the two construction products, generating minimum verification inputs after the verification passes, and respectively running to form protection execution difference data; and inputting the source code protection relations, the production construction product protection relations, the protection execution difference data and the dangerous operation types into a protection effectiveness identification model to output code defect detection information. The application realizes verification of the real effectiveness of protection statements in construction products, and reduces the hidden defect omission rate.

Description

Technical Field

[0001] This invention relates to the field of code detection technology, and in particular to a method, device and medium for code defect detection based on machine learning. Background Technology

[0002] Code defect detection is a crucial quality assurance technique in software development and continuous integration. Conventional methods typically combine static rule scanning, commit difference analysis, unit test execution, and machine learning models to identify defects in source code, such as null access, boundary violations, permission checks, resource release issues, and exception handling. As project build processes become more complex, the objects of detection are gradually expanding from single code snippets to function call relationships, control relationships, and variable constraint relationships to improve the accuracy of defect localization.

[0003] During the production build process, conditional compilation, fixed build parameters, code generation, bytecode enhancement, or obfuscation may alter the actual control of source code protection statements over dangerous operations. Conventional methods primarily rely on protection relationships at the source code level or test execution phenomena for judgment. This makes it difficult to verify whether the same dangerous operation in the final build artifact truly changes its execution state with changes in protection conditions, and it is also difficult to rule out misjudgments such as incomparable build artifacts or unusable verification inputs. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a machine learning-based code defect detection method to solve the problems of existing technologies, such as difficulty in verifying the actual effectiveness of source code protection statements in the final build product and difficulty in accurately identifying hidden defects such as protection failure after build.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] Firstly, this invention provides a machine learning-based code defect detection method, comprising: collecting protection statements and dangerous operation statements within the modified function in the current commit; determining verification protection statements and generating source code protection relationships based on the branch control relationship and variable restriction relationship between the protection statements and the dangerous operation statements; generating production build artifacts according to production build parameters; locating the execution instructions and conditional jump instructions corresponding to the dangerous operations in the production build artifacts; and generating reverse verification build artifacts by replacing the pass conditions of the verification protection statements with opposite conditions without changing the dangerous operation statements, function entry points, and business call paths; performing identity verification on the dangerous operations in the production build artifacts and the reverse verification build artifacts; generating minimum verification input based on the verification protection statements after the verification passes; running the two build artifacts respectively and recording the execution status of the same dangerous operation to form protection execution difference data; inputting the source code protection relationship, production build artifact protection relationship, protection execution difference data, and dangerous operation type into a protection effectiveness identification model and outputting protection failure candidate flags; performing binary gating and outputting code defect detection information.

[0008] As a preferred embodiment of the machine learning-based code defect detection method of the present invention, the step of collecting protected statements and dangerous operation statements within the modified functions in the current commit includes: reading the code difference file of the current commit, performing syntax parsing on the function containing the modified code fragment through the compiler front-end, and generating a syntax node table, branch control table, variable definition table, variable usage table, and function entry parameter table; retrieving protected statements based on the syntax node table, and retrieving dangerous operation statements based on the syntax node table; the code difference file includes the modified file path, modified line number, code fragment before modification, and code fragment after modification.

[0009] As a preferred embodiment of the machine learning-based code defect detection method of the present invention, the step of determining the verification protection statement includes: determining the dangerous operation usage variables and the corresponding entry assignment paths of the dangerous operation usage variables in the dangerous operation statement according to the variable definition table and the variable usage table; for each protection statement, determining whether the protection statement is located before the execution of the dangerous operation statement, whether the dangerous operation statement is within the control scope of the protection statement, and whether the variable restricted by the protection statement is consistent with the dangerous operation usage variable or can be passed to the dangerous operation usage variable through the entry assignment path; determining the protection statement that simultaneously satisfies the conditions of execution before, control scope coverage, and variable restriction association as candidate protection statements; calculating the protection proximity value of the candidate protection statements according to the variable restriction association value, branch control coverage value, execution order validity value, and statement proximity value; determining the candidate protection statement with the largest protection proximity value as the verification protection statement; when two or more candidate protection statements have the same protection proximity value, selecting the protection statement that is closest to the dangerous operation statement and located before the execution of the dangerous operation statement as the verification protection statement.

[0010] As a preferred embodiment of the machine learning-based code defect detection method of the present invention, the generation of source code protection relationships includes: when there are no candidate protection statements with a protection proximity value greater than 0, marking dangerous operation statements as unverified protection statements and generating a source code unprotected relationship; when dangerous operation statements are within the control scope of verified protection statements and the variables used in dangerous operation are restricted by verified protection statements, generating a valid source code protection relationship; when dangerous operation statements are not within the control scope of verified protection statements, or the variables used in dangerous operation are not restricted by verified protection statements, generating a suspected invalid source code protection relationship.

[0011] As a preferred embodiment of the machine learning-based code defect detection method of the present invention, the step of locating the execution instructions and conditional jump instructions corresponding to dangerous operations in the production build artifact includes: determining the production build parameter identifier based on the continuous integration configuration file associated with the current commit; executing the production build according to the production build parameter identifier to generate the production build artifact and the build mapping file; locating the execution instructions corresponding to the dangerous operation statements in the production build artifact based on the function signature and source code line number of the dangerous operation statement; using the execution instructions as the verification object, searching forward along the control flow of the production build artifact for conditional jump instructions that can control whether the execution instructions are executed, and generating a production build artifact protection relationship based on the control relationship between the conditional jump instructions and the execution instructions; the production build parameter identifier includes the build command name, the target runtime environment name, the compilation configuration name, and the build parameter set; the build mapping file includes the correspondence between the source code line number, the function signature, the call instruction position, and the conditional jump instruction position.

[0012] As a preferred embodiment of the machine learning-based code defect detection method of the present invention, the generation of production build artifact protection relationships includes: generating a production effective protection relationship when a conditional jump instruction exists and the branch of the conditional jump instruction contains the execution instruction corresponding to the dangerous operation; generating a production protection outward movement relationship when a conditional jump instruction exists and the execution instruction corresponding to the dangerous operation is outside the control range of the conditional jump instruction; generating a production protection disappearance relationship when a conditional jump instruction does not exist and the execution instruction corresponding to the dangerous operation can be directly executed in the reachable path of the function entry point; generating a production fixed release relationship when a conditional jump instruction exists and the condition variable is fixed by the production build parameter to a value that allows the execution of the dangerous operation; and generating a production fixed blocking relationship when a conditional jump instruction exists and the condition variable is fixed by the production build parameter to a value that prohibits the execution of the dangerous operation.

[0013] As a preferred embodiment of the machine learning-based code defect detection method of the present invention, the generation of reverse verification build product includes: generating reverse values ​​of protection conditions based on verification protection statements; generating a reverse verification code copy based on the reverse values ​​of protection conditions; using the same production build parameter identifier as the production build product, isolating and building the reverse verification code copy to generate a reverse verification build product; the reverse values ​​of protection conditions include: non-empty pass condition corresponding to empty value pass condition, length less than condition corresponding to length not less than condition, permission pass condition corresponding to permission fail condition, resource initialized condition corresponding to resource uninitialized condition, transaction started condition corresponding to transaction not started condition, and exception blocking condition corresponding to exception not blocked condition; the reverse verification code copy only replaces the pass condition of the verification protection statement, without changing the dangerous operation statement, function entry point, business call path, and dangerous operation usage variables.

[0014] As a preferred embodiment of the machine learning-based code defect detection method of the present invention, the step of forming protection execution difference data includes: generating a minimum verification input based on the verification protection statement, and determining whether the minimum verification input can be passed to the variable used by the dangerous operation based on the entry assignment path; when the minimum verification input can be passed to the variable used by the dangerous operation, setting dangerous operation execution record points in the production build product and the reverse verification build product respectively; sending the same minimum verification input to the function entry points corresponding to the production build product and the reverse verification build product respectively, and recording the execution status of the same dangerous operation in the two build products; when the same dangerous operation in the production build product has not entered the execution state but the same dangerous operation in the reverse verification build product has entered the execution state, generating protection valid data; when the same dangerous operation in both the production build product and the reverse verification build product has entered the execution state, generating protection failure confirmation data; when the same dangerous operation in the production build product has entered the execution state but the same dangerous operation in the reverse verification build product has not entered the execution state, generating protection direction abnormal data; when the same dangerous operation in neither the production build product nor the reverse verification build product has entered the execution state, generating other condition blocking data.

[0015] As a preferred embodiment of the machine learning-based code defect detection method described in this invention, wherein,

[0016] In a second aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the machine learning-based code defect detection method described in the first aspect of the present invention.

[0017] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any step of the machine learning-based code defect detection method described in the first aspect of the present invention.

[0018] The beneficial effects of this invention are as follows: by generating production build products and reverse verification build products, the actual effectiveness of source code protection statements in the final build product is verified; by comparing the execution status of the same dangerous operation in the two build products, the direct verification of whether the protection statement truly controls the dangerous operation is achieved; by using the protection effectiveness identification model and defect output confirmation value for binary gating judgment, the stable identification of hidden defects of protection failure type after construction is achieved, and false alarms are reduced. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a machine learning-based code defect detection method.

[0021] Figure 2 A schematic diagram for verifying the protection statement.

[0022] Figure 3 This is a schematic diagram for verifying the identity of dual-construction products.

[0023] Figure 4 A schematic diagram for determining execution differences and defects in output. Detailed Implementation

[0024] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0025] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0026] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0027] Reference Figures 1-4 This is one embodiment of the present invention, which provides a code defect detection method based on machine learning, including the following steps:

[0028] S1. Collect the protected statements and dangerous operation statements within the modified functions in the current commit. Based on the branch control relationship and variable restriction relationship of the protected statements on the dangerous operation statements, determine the verified protected statements and generate source code protection relationships.

[0029] After receiving the code of the current commit, read the code difference file of the current commit.

[0030] The code difference file includes the changed file path, the changed line number, the code snippet before the change, and the code snippet after the change.

[0031] Furthermore, the compiler front-end performs syntax parsing on the function containing the modified code snippet, generating a syntax node table, branch control table, variable definition table, variable usage table, and function entry parameter table.

[0032] The syntax node table records the positions of conditional statements, call statements, assignment statements, return statements, and exception handling statements within a function; the branch control table records the scope of statements controlled by conditional statements; the variable definition table records the declaration position, assignment position, and source expression of variables; the variable usage table records the usage position of variables in call statements, array access statements, collection access statements, resource release statements, and database operation statements; and the function entry parameter table records the name, type, and first assignment position of the function entry parameter within the function body.

[0033] Further, the protected statement is retrieved from the syntax node table.

[0034] The protection statements include NOT NULL checks, length checks, permission checks, resource status checks, transaction status checks, and exception prevention statements.

[0035] It should be noted that: NOT NULL checks are determined by identifying whether a variable is not equal to a null value, checking the existence of an object, and excluding empty sets; Length checks are determined by identifying the comparison expression between the array length, set length, string length, and access index; Permission checks are determined by identifying the comparison expression between user role, access token, tenant number, operation permission flag, and allowed conditions; Resource status checks are determined by identifying the initialization, open, or closed status of connection objects, file handles, lock objects, and stream objects; Transaction status checks are determined by identifying the transaction start, commit, and rollback status; Exception blocking statements are identified by identifying statements that throw exceptions, return error codes, refuse execution, and terminate the current branch.

[0036] Furthermore, dangerous operation statements are retrieved from the syntax node table.

[0037] Dangerous operations include database deletion, database writing, file writing, external command execution, permission-sensitive interface calls, set index access, resource release, and transaction commit.

[0038] It should be noted that the dangerous operation type is determined by the statement name, the type of the calling object, and the parameter type; the variables that actually participate in the call, access, release, or commit in the dangerous operation statement are read through the variable usage table to form the dangerous operation usage variables.

[0039] For set index access, dangerous operations use variables including the accessed set variable and the access index variable; for permission-sensitive interface calls, dangerous operations use variables including user identity variable, resource identifier variable, and operation type variable; for resource release, dangerous operations use variables including the resource variable that was closed or unlocked; for transaction commit, dangerous operations use variables including transaction object variable and write object variable.

[0040] Furthermore, an entry assignment path is established based on the variable definition table and the variable usage table.

[0041] Specifically, starting from the location where the variable is used in the dangerous operation, search backwards for the most recent assignment statement. If the assignment statement originates from the function's entry parameter, then the sequence of assignment statements between the function's entry parameter and the variable used in the dangerous operation is recorded as the direct entry assignment path.

[0042] If the assignment statement originates from an intermediate object field, and the intermediate object is constructed or passed in by a function entry parameter, then the sequence of assignment statements between the function entry parameter, the intermediate object field, and the variable used by the dangerous operation is recorded as an indirect entry assignment path.

[0043] If a dangerous operation uses a variable that is generated only by a fixed value inside the function, and the function's input parameter cannot change the variable used in the dangerous operation, then it is recorded as an assignment path without an input.

[0044] Furthermore, candidate protection statements are determined based on protection statements, dangerous operation statements, and variables used in dangerous operations.

[0045] For each protection statement, three conditions are determined: precedence, control scope coverage, and variable restriction association.

[0046] The prerequisites include whether the protection statement is executed before the execution of the dangerous operation statement.

[0047] The control scope coverage conditions include: whether the dangerous operation statement is within the control scope of the protection statement.

[0048] Variable restriction association conditions include: whether the variable restricted by the protection statement is consistent with the variable used in the dangerous operation, or whether the variable restricted by the protection statement can be passed to the variable used in the dangerous operation through the entry assignment path.

[0049] Protection statements that meet the three conditions of execution first, control scope coverage, and variable restriction association are recorded as candidate protection statements.

[0050] Furthermore, in order to determine a unique verification protection statement from the candidate protection statements, the protection proximity value of the candidate protection statement is calculated, expressed as:

[0051] ;

[0052] ;

[0053] in, This indicates the protection proximity value of the candidate protection statement. This indicates that the variable restricts the associated values. Indicates the branch control coverage value. Indicates the order of execution of valid values. This indicates the closest value to the statement. This indicates the number of executable statements between protected statements and dangerous operation statements.

[0054] It should be noted that, The number of variable transfer steps between the protection statement's restriction variable and the variable used in the dangerous operation is determined by the shortest number of variable transfer steps. If a transfer path exists, it is calculated by dividing 1 by 1 and summing it with the shortest number of variable transfer steps. If no transfer path exists, it is taken as 0. It is determined by the ratio of the number of paths leading to the dangerous operation statement after the execution of the branch allowed by the candidate protected statement to the number of paths from the function entry point to the dangerous operation statement; The protection statement is determined by the ratio of the number of paths where the candidate protected statement is executed before the dangerous operation statement to the number of paths from the function entry point to the dangerous operation statement.

[0055] The protection statement with the largest protection proximity value among the candidate protection statements is selected as the verification protection statement.

[0056] When two or more candidate protection statements have the same protection proximity value, the protection statement that is closest to the dangerous operation statement and is located before the execution of the dangerous operation statement is selected as the verification protection statement.

[0057] When there are no candidate protection statements with a protection proximity value greater than 0, dangerous operation statements are marked as unverified protection statements, and source code is generated without protection.

[0058] Furthermore, after determining the verification protection statement, the reverse value of the protection condition is generated.

[0059] Among them, the non-empty pass condition corresponds to the null value pass condition, the length less than condition corresponds to the length not less than condition, the permission pass condition corresponds to the permission fail condition, the resource initialized condition corresponds to the resource not initialized condition, the transaction started condition corresponds to the transaction not started condition, and the exception blocking condition corresponds to the exception not blocking condition.

[0060] Furthermore, source code protection relationships are generated based on the control status of the protection statements for dangerous operation statements.

[0061] When a dangerous operation statement is within the scope of a verification protection statement, and the variables used by the dangerous operation are restricted by the verification protection statement, a valid source code protection relationship is generated.

[0062] When a dangerous operation statement is not within the scope of the verification protection statement, or when the variables used in a dangerous operation are not restricted by the verification protection statement, a suspected invalid protection relationship is generated in the source code.

[0063] When the source code protection relationship is no source code protection relationship, output a source code no protection message. The source code no protection message includes dangerous operation statements, dangerous operation types, and supplementary protection statement prompts, and stop the reverse verification and construction process for dangerous operation statements.

[0064] S2. Generate production build artifacts according to production build parameters. Locate the execution instructions and conditional jump instructions corresponding to dangerous operations in the production build artifacts. Without changing the dangerous operation statements, function entry points, and business call paths, only replace the pass conditions of the verification protection statements with the opposite conditions to generate reverse verification build artifacts.

[0065] The production build parameter identifier is determined by the continuous integration configuration file associated with the current commit.

[0066] The continuous integration configuration file includes the build command name, the target runtime environment name, the compilation configuration name, and the set of build parameters.

[0067] Read the build command name, target runtime environment name, and compilation configuration name used in the continuous integration configuration file to form the production build parameter identifier.

[0068] The build executor calls the project build tool according to the build command name, loads the production environment configuration file according to the target runtime environment name, loads the compilation configuration file according to the compilation configuration name, writes the compilation parameters, conditional compilation parameters, code generation parameters, and obfuscation parameters according to the build parameter set, executes the production build in the isolated build directory, and obtains the production build artifacts.

[0069] Furthermore, the build executor outputs a build mapping file simultaneously when generating production build artifacts.

[0070] The mapping file includes the correspondence between source code line numbers, function signatures, call instruction locations, and conditional jump instruction locations.

[0071] Based on the dangerous operation statement, read the function signature and source code line number to which the dangerous operation statement belongs from the build mapping file, and locate the execution instruction corresponding to the dangerous operation statement in the production build artifact.

[0072] The execution instructions include database delete call instructions, database write call instructions, file write call instructions, external command call instructions, permission-sensitive interface call instructions, set index access instructions, resource release call instructions, and transaction commit call instructions.

[0073] Furthermore, in the production build product, the execution instructions corresponding to the dangerous operation are used as the verification objects, and the conditional jump instructions that can control whether the execution instructions of the dangerous operation are executed are searched forward along the control flow of the production build product.

[0074] If a conditional jump instruction exists, and the branch of the conditional jump instruction contains instructions for executing dangerous operations, then a production-effective protection relationship is generated.

[0075] If a conditional jump instruction exists, but the dangerous operation execution instruction is outside the control scope of the conditional jump instruction, then a production protection shift relationship is generated.

[0076] If the conditional jump instruction does not exist, and the dangerous operation execution instruction can be executed directly in the reachable path of the function entry, then the production protection disappearance relationship is generated.

[0077] If a conditional jump instruction exists, but the condition variable of the conditional jump instruction is fixed by the production build parameters to a value that allows dangerous operations to be executed, then a production fixed release relationship is generated.

[0078] If a conditional jump instruction exists, but the condition variable is fixed by the production build parameter to a value that prohibits the execution of dangerous operations, then a production fixed blocking relationship is generated.

[0079] Among them, the effective protection relationship of production, the relocation relationship of production protection, the disappearance relationship of production protection, the fixed release relationship of production, and the fixed prevention relationship of production together constitute the protection relationship of the products constructed by production.

[0080] Furthermore, a reverse verification code copy is generated by taking values ​​in reverse based on the protection conditions.

[0081] The reverse verification code copy only replaces the pass conditions of the verification protection statement, without changing the dangerous operation statement, function entry point, business call path, or variables used in the dangerous operation.

[0082] Furthermore, the build executor uses the same production build parameter identifier to build the reverse verification code copy in isolation, and obtains the reverse verification build artifact.

[0083] In this process, the reverse verification build product and the production build product use the same build command name, the same target runtime environment name, the same compilation configuration name, and the same set of build parameters, so that the difference between the two build products is limited to the difference in the pass conditions of the verification protection statement.

[0084] Furthermore, in the reverse verification build product, the execution instructions corresponding to the dangerous operation statements in the reverse verification build product are located according to the dangerous operation location method in the production build product.

[0085] The dangerous operation call name, target object type, number of parameters, and parameter source location are read from the production build artifact and the reverse verification build artifact respectively to form dangerous operation identity verification data.

[0086] Furthermore, when the dangerous operation call names, target object types, number of parameters, and parameter source locations are the same, the dangerous operation identity is determined by a value of 1, indicating that the dangerous operations in the two build artifacts are the same operation to be verified.

[0087] When any of the following is inconsistent: dangerous operation call name, target object type, number of parameters, or parameter source location, the dangerous operation identity is set to 0, the dual build product running verification is stopped, a build product incomparability prompt is output, and no defect blocking is output.

[0088] Furthermore, the equivalent protection conditions are examined in the production and construction products.

[0089] Among them, the equivalent protection condition refers to the condition jump instruction that is not replaced by the reverse value of the protection condition, but can still control the execution of the same dangerous operation instruction and restrict the use of variables for the same dangerous operation.

[0090] When equivalent protection conditions exist in the production and construction product, the equivalent protection confirmation flag is set to 1; when equivalent protection conditions do not exist in the production and construction product, the equivalent protection confirmation flag is set to 0.

[0091] S3. Perform identity verification on the hazardous operations in the production build product and the reverse verification build product. After the verification is passed, generate the minimum verification input according to the verification protection statement, run the two build products respectively and record the execution status of the same hazardous operation to form protection execution difference data.

[0092] When the critical operation identity pass value is 0, the verification will stop, the build product will be output as an incomparable warning, and no defect blocking will be output.

[0093] When the identity of dangerous operations passes with a value of 1, the minimum verification input generation process is initiated.

[0094] Furthermore, the minimum verification input is generated by taking values ​​in reverse according to the protection conditions.

[0095] When the verification protection statement is a non-null check, the function entry parameter that can be passed to the variable used in the dangerous operation is set to a null value.

[0096] When the verification protection statement is a length check, the function entry parameter that can be passed to the accessed set, array, or string is set to an input whose length is equal to the access index value.

[0097] When the verification protection statement is used for permission judgment, the function entry parameter that can be passed to the identity verification variable is set to "permission denied".

[0098] When the verification protection statement is used to determine the resource status, the function entry parameter that can be passed to the resource object variable is set to an uninitialized resource.

[0099] When the verification protection statement is used to determine the transaction status, the function entry parameter that can be passed to the transaction object variable is set to "transaction not started".

[0100] When the verification protection statement is an exception blocking statement, the function entry parameter is set to the input that triggers the corresponding exception type.

[0101] Furthermore, the minimum verification input state is determined based on the entry assignment path.

[0102] The minimum validation input is deemed acceptable when it can be passed directly to the variable used in the dangerous operation through the function's entry parameter, or through the upper-level interface parameter, intermediate object field, or callback parameter.

[0103] When the minimum verification input cannot be passed to the variable used in the dangerous operation, it is determined that the minimum verification input cannot be sent, and a message indicating that the verification input cannot be sent is output. No defect blocking is output.

[0104] Furthermore, when the minimum verification input can be sent, dangerous operation execution record points are set in both the production build product and the reverse verification build product.

[0105] The dangerous operation execution record point is set at the position of the executable instruction preceding the dangerous operation execution instruction, and is used to record whether the dangerous operation has entered the execution state.

[0106] It should be noted that dangerous operation execution record points are implemented using runtime log markers, debug event records, or execution count markers.

[0107] Furthermore, the same minimum verification input is sent to the function entry point corresponding to the production build artifact, the production build artifact is run, and the dangerous operation execution record point is read.

[0108] The same minimum verification input is fed into the function entry point corresponding to the reverse verification build product, the reverse verification build product is run, and the dangerous operation execution record point is read.

[0109] The two runs use the same function entry point, the same input content, the same construction parameter identifier, and the same runtime environment configuration, thus limiting the source of the difference in execution status to the difference in the pass conditions of the verification protection statement.

[0110] Furthermore, protection execution difference data is generated based on the execution status of the same hazardous operation in the two constructed products.

[0111] When the same hazardous operation in the production build product has not entered the execution state, and the same hazardous operation in the reverse verification build product has entered the execution state, it is determined that the verification protection statement can control the hazardous operation, outputs a protection effective prompt, and does not output defect blocking.

[0112] When the same hazardous operation in both the production build product and the reverse verification build product enters the execution state, it is determined that the hazardous operation has not changed its execution state with the verification protection statement through condition changes, and protection failure confirmation data is generated.

[0113] When the same hazardous operation in the production build product enters the execution state, and the same hazardous operation in the reverse verification build product does not enter the execution state, it is determined that the production build product still executes the hazardous operation when the protection conditions are not met, and abnormal data in the protection direction is generated.

[0114] When the same dangerous operation in both the production build product and the reverse verification build product has not entered the execution state, it is determined that the dangerous operation is blocked by other execution conditions, and an "other condition blocking" prompt is output, but no defect blocking is output.

[0115] S4. Input the source code protection relationship, production build artifact protection relationship, protection execution difference data and dangerous operation type into the protection effectiveness identification model, output protection failure candidate flags, perform binary gating and output code defect detection information.

[0116] Furthermore, the source code protection relationship, production build artifact protection relationship, hazardous operation identity pass value, equivalent protection confirmation data, minimum verification input status, protection execution difference data, and hazardous operation type are used as input data for the protection effectiveness identification model.

[0117] It should be noted that the protection effectiveness identification model adopts a binary decision tree classification structure. The model input fields include source code protection relationships, production build artifact protection relationships, hazardous operation identity pass values, equivalent protection confirmation flags, minimum verification input submission status, protection execution difference data, and hazardous operation types. During training, historical verification samples are first organized into a training table. Each historical verification sample includes historical source code protection relationships, historical production build artifact protection relationships, historical hazardous operation identity pass values, historical equivalent protection confirmation flags, historical minimum verification input submission status, historical protection execution difference data, historical hazardous operation types, and developer review categories. When the developer review confirmation verification protection statement fails to control the hazardous operation in the production build artifact and restores protection control through a repair submission, the developer review category is marked as protection failure. The developer review confirmation verification protection statement is valid, equivalent protection is valid, and the minimum verification input is valid. When small verification inputs cannot be fed in or the constructed products are incomparable, the developer review category is marked as non-protection failure. During training, the purity improvement of the sample categories on both sides after field partitioning is calculated one by one from the input fields. The field with the largest purity improvement is selected as the current node partitioning field, and lower-level nodes are generated according to the field values ​​until the developer review categories of the training samples in the leaf node are consistent, or all input fields have been partitioned. When there are still both protected failure and non-protection failure samples in the leaf node, the leaf node is recorded as the category with the larger number of samples. If the number is the same, it is recorded as non-protection failure. During online detection, the current input field is fed into the trained binary decision tree classification structure. Starting from the root node, it enters the corresponding lower-level nodes level by level according to the field values ​​until the leaf node is reached. When the leaf node category is protected failure, the protected failure candidate label 1 is output. When the leaf node category is non-protection failure, the protected failure candidate label 0 is output.

[0118] Furthermore, the current input data is fed into the protection effectiveness identification model.

[0119] When the output category of the protection effectiveness identification model is protection failure, the protection failure candidate flag is set to 1; when the output category of the protection effectiveness identification model is non-protection failure, the protection failure candidate flag is set to 0.

[0120] Furthermore, the defect output confirmation value is calculated, expressed as:

[0121] ;

[0122] in, This is the defect output confirmation value, which can be either 0 or 1. For confirmation of the identity of hazardous operations, A flag can be sent to the minimum validation input. To protect the execution exception confirmation flag, To protect the failure candidate marker, This is a confirmation mark for equivalent protection.

[0123] It should be noted that, The value is set to 1 when the "identity of dangerous operations" pass value is 1, and to 0 when the "identity of dangerous operations" pass value is 0. The value is 1 when the minimum verification input is in the state of being able to be sent, and 0 when the minimum verification input is in the state of being unable to be sent. The value is 1 when protection failure confirmation data or protection direction abnormal data is generated, and 0 when no protection failure confirmation data or protection direction abnormal data is generated. The value is 1 when the protection effectiveness identification model outputs protection failure, and 0 when the protection effectiveness identification model outputs non-protection failure. The value is 1 when the equivalent protection confirmation data indicates that the equivalent protection conditions are valid, and 0 when the equivalent protection confirmation data indicates that the equivalent protection conditions are invalid or do not exist.

[0124] It should also be noted that the defect output confirmation value adopts a binary multiplicative gating form to determine whether protection failure is established, rather than the degree of risk. The identity of dangerous operation, the availability of minimum verification input, protection anomaly confirmation, protection effectiveness identification model candidate judgment, and equivalent protection ineffectiveness confirmation are all necessary conditions for defect output. If any condition is not met, defect blocking will not be output, thereby avoiding false alarms caused by relying solely on source code relationships, operational phenomena, or model judgment. First, it is confirmed that the two constructed products are comparing the same dangerous operation. Then, it is confirmed that the verification input can reach the variables used in the dangerous operation. Then, it is confirmed that the dangerous operation is still executed abnormally after the protection conditions change. Then, the model trained by historical verification samples is reviewed. Finally, other effective protection conditions are excluded. Through collaborative judgment, it is possible to reliably identify hidden defects where there are protection statements in the source code but the protection statements in the produced constructed products do not actually control dangerous operations, and reduce false judgments caused by incomparable constructed products, unreachable inputs, or equivalent protection effectiveness.

[0125] Furthermore, when the identification flag for dangerous operations is 0, a message indicating that the constructed products are incomparable is output, but no defect blocking is output.

[0126] When the minimum verification input can be sent is marked as 0, the output prompts that the verification input cannot be sent, and no defect blocking is output.

[0127] When the equivalent protection confirmation flag is 1, an equivalent protection valid prompt is output, but defect blocking is not output.

[0128] When the protection execution abnormality confirmation flag is 0, the output prompts that the protection execution difference has not met the defect output conditions, and no defect blocking is output.

[0129] Furthermore, when the defect output confirmation value is 1 and the protection failure confirmation data exists, the protection failure class code defect detection information is output.

[0130] When the defect output confirmation value is 1 and the protection direction abnormal data exists, output the protection direction abnormal class code defect detection information.

[0131] The code defect detection information includes source code protection statements, dangerous operation locations, production build artifact execution evidence, reverse verification of build artifact execution evidence, protection failure types, equivalent protection judgment information, and suggested repair directions.

[0132] Furthermore, the recommended repair direction is determined based on the protection relationship of the production-constructed products and the difference data in protection implementation.

[0133] The recommended remediation direction for the production protection relocation relationship is to restore the dangerous operation statement to the control range of the verification protection statement.

[0134] The suggested remediation approach for the disappearance of production protection is to write the verification protection statement into the function where the dangerous operation is located.

[0135] The recommended remediation direction for the fixed release relationship in production is to cancel the fixed release of verification protection statements by production build parameters.

[0136] The recommended remediation direction for abnormal data in the protection direction is to restore the consistency of the conditional direction of the verification protection statement in the production build artifact and the reverse verification build artifact.

[0137] This embodiment also provides a computer device applicable to the machine learning-based code defect detection method, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the machine learning-based code defect detection method proposed in the above embodiment.

[0138] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0139] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the machine learning-based code defect detection method proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0140] In summary, this invention verifies the actual effectiveness of source code protection statements in the final build product by generating production build products and reverse verifying build products; it directly verifies whether protection statements truly control dangerous operations by comparing the execution status of the same dangerous operation in the two build products; and it achieves stable identification of hidden defects of protection failure type after construction and reduces false alarms by using a protection effectiveness identification model and defect output confirmation value for binary gating judgment.

[0141] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A machine learning-based code defect detection method, characterized in that, include: Collect protected statements and dangerous operation statements within the modified functions in the current commit. Based on the branch control relationship and variable restriction relationship between the protected statements and the dangerous operation statements, determine the verified protected statements and generate source code protection relationships. Generate production build artifacts according to production build parameters, locate the execution instructions and condition jump instructions corresponding to dangerous operations in the production build artifacts, and generate reverse verification build artifacts by replacing the pass conditions of the verification protection statement with the opposite conditions without changing the dangerous operation statement, function entry and business call path. Perform identity verification on hazardous operations in the production build product and the reverse verification build product. After the verification is passed, generate the minimum verification input according to the verification protection statement, run the two build products respectively and record the execution status of the same hazardous operation to form protection execution difference data. After inputting source code protection relationships, production build artifact protection relationships, protection execution difference data, and dangerous operation types into the protection effectiveness identification model, the model outputs protection failure candidate flags, performs binary gating, and outputs code defect detection information.

2. The machine learning-based code defect detection method as described in claim 1, characterized in that, The collection of protected and dangerous operation statements within the modified function in the current submission includes: Read the code difference file of the current submission, and use the compiler front-end to perform syntax parsing on the function containing the changed code snippet, generating a syntax node table, branch control table, variable definition table, variable usage table, and function entry parameter table; Retrieve protected statements and dangerous operation statements based on the syntax node table. The code difference file includes the changed file path, the changed line number, the code snippet before the change, and the code snippet after the change.

3. The machine learning-based code defect detection method as described in claim 2, characterized in that, The defined verification protection statement includes: Based on the variable definition table and the variable usage table, determine the dangerous operation variables used in the dangerous operation statements and the corresponding entry assignment paths for the dangerous operation variables. For each protection statement, determine whether the protection statement is executed before the execution of the dangerous operation statement, whether the dangerous operation statement is within the control scope of the protection statement, and whether the variable restricted by the protection statement is consistent with the variable used in the dangerous operation or can be passed to the variable used in the dangerous operation through the entry assignment path; Protective statements that simultaneously satisfy the requirements of execution first, control scope coverage, and variable restriction association are identified as candidate protective statements. The protection proximity value of candidate protected statements is calculated based on variable restriction association value, branch control coverage value, execution order validity value, and statement proximity value. The candidate protection statement with the largest protection proximity value is selected as the verification protection statement; When two or more candidate protection statements have the same protection proximity value, the protection statement that is closest to the dangerous operation statement and is located before the execution of the dangerous operation statement is selected as the verification protection statement.

4. The machine learning-based code defect detection method as described in claim 3, characterized in that, The generated source code protection relationship includes: When there are no candidate protection statements with a protection proximity value greater than 0, the dangerous operation statement is marked as an unverified protection statement, and an unprotected relationship is generated in the source code. When a dangerous operation statement is within the control scope of a verification protection statement, and the variables used by the dangerous operation are restricted by the verification protection statement, a valid source code protection relationship is generated. When a dangerous operation statement is not within the control scope of a verification protection statement, or when the variables used in a dangerous operation are not restricted by a verification protection statement, a suspected invalid protection relationship is generated in the source code.

5. The machine learning-based code defect detection method as described in claim 1, characterized in that, The execution instructions and conditional jump instructions for locating hazardous operations in the production build product include: The production build parameter identifier is determined based on the current continuous integration configuration file associated with the commit. Execute the production build according to the production build parameter identifier, and generate the production build artifacts and build mapping files; Based on the function signature and source code line number of the dangerous operation statement, locate the execution instruction corresponding to the dangerous operation statement in the production build artifact; Using the execution instruction as the verification object, the system searches forward along the control flow of the production build product for conditional jump instructions that can control whether the execution instruction is executed, and generates the production build product protection relationship based on the control relationship between the conditional jump instructions and the execution instructions. The production build parameter identifier includes the build command name, target runtime environment name, compilation configuration name, and build parameter set; The construction mapping file includes the correspondence between source code line numbers, function signatures, call instruction locations, and conditional jump instruction locations.

6. The machine learning-based code defect detection method as described in claim 5, characterized in that, The protection relationships for the generated production build artifacts include: When a conditional jump instruction exists, and the branch of the conditional jump instruction contains the execution instruction corresponding to the dangerous operation, a valid production protection relationship is generated. When a conditional jump instruction exists, and the execution instruction corresponding to the dangerous operation is outside the control range of the conditional jump instruction, a production protection shift relationship is generated. When a conditional jump instruction does not exist, and the execution instruction corresponding to the dangerous operation can be executed directly in the reachable path of the function entry point, a production protection disappearance relationship is generated; When a conditional jump instruction exists, and the condition variable is fixed by the production build parameters to a value that allows dangerous operations to be executed, a production fixed release relationship is generated. When a conditional jump instruction exists, and the condition variable is fixed by the production build parameter to a value that prohibits the execution of dangerous operations, a production fixed blocking relationship is generated.

7. The machine learning-based code defect detection method as described in claim 1, characterized in that, The generated reverse verification construct includes: The protection condition is generated by reverse-evaluating the protection statement; Generate a reverse verification code copy based on the reverse value of the protection conditions; Using the same production build parameter identifiers as the production build artifacts, the reverse verification code copy is built in isolation to generate the reverse verification build artifacts. The reverse value of the protection conditions includes: non-empty pass condition corresponds to empty value pass condition, length less than condition corresponds to length not less than condition, permission pass condition corresponds to permission not pass condition, resource initialized condition corresponds to resource not initialized condition, transaction started condition corresponds to transaction not started condition, and exception blocking condition corresponds to exception not blocking condition. The reverse verification code copy only replaces the pass conditions of the verification protection statement, without changing the dangerous operation statement, function entry point, business call path, or variables used in the dangerous operation.

8. The machine learning-based code defect detection method as described in claim 1, characterized in that, The data that forms the protection execution difference includes: Generate the minimum verification input based on the verification protection statement, and determine whether the minimum verification input can be passed to the variable used in the dangerous operation based on the entry assignment path; When the minimum verification input can be passed to the variable used in the hazardous operation, set the hazardous operation execution record point in both the production build artifact and the reverse verification build artifact; The same minimum verification input is sent to the function entry points corresponding to the production build product and the reverse verification build product, respectively, and the execution status of the same dangerous operation in the two build products is recorded. When the same hazardous operation in the production build product has not entered the execution state, but the reverse verification shows that the same hazardous operation in the build product has entered the execution state, valid protection data is generated. When the same hazardous operation in both the production build product and the reverse verification build product enters the execution state, protection failure confirmation data is generated. When the same hazardous operation in the production build product enters the execution state and the same hazardous operation in the reverse verification build product does not enter the execution state, protection direction abnormal data is generated. When the same dangerous operation in both the production build artifact and the reverse verification build artifact has not entered the execution state, generate additional conditional blocking data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the machine learning-based code defect detection method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the machine learning-based code defect detection method according to any one of claims 1 to 8.

Images