Method and device for identifying defect statement in software code

Abstract

The invention discloses a method for identifying a defect statement in a software code. The method comprises the following steps: acquiring a to-be-processed software code; the to-be-processed software codes are recognized through a first large language model to obtain a first set, and the first large language model is used for checking the to-be-processed software codes according to the at least one code checking item to obtain a second set; based on auxiliary information and an association statement related to the first suspected defect statement in the to-be-processed software code, whether the first suspected defect statement has a defect is determined, the second set comprises the first suspected defect statement, and the auxiliary information comprises at least one statement and a condition required to be met when each statement does not have the defect; and if the first suspected defect statement does not have the defect, obtaining a first set which does not comprise the first suspected defect statement according to the second set. By utilizing the scheme, the first large language model can be combined with the auxiliary information and the at least one code inspection item to accurately determine the suspected defect statement in the software code to be processed.

Description

Technical Field

[0001] The present application relates to the field of software testing, and in particular to a method and device for identifying defective statements in software codes. Background Art

[0002] In today's environment driven by big data and algorithms, carefully processed data has become the key to decision-making, algorithm development, and personalized recommendations. Currently, data can be processed by manually written software code so that the processed data can be passed to downstream applications.

[0003] After writing the software code, the software code can be tested to determine whether the software code includes defective statements. Currently, the software code can be checked manually or by using a static code scanning tool to determine whether the software code includes defective statements. The "defective statements" mentioned here refer to statements with defects.

[0004] However, using manual inspection or static code scanning tools to inspect software codes cannot accurately identify defective statements in the software codes. Therefore, a solution is urgently needed to solve the above problem. Summary of the invention

[0005] In order to solve or at least partially solve the above technical problems, the present application provides a method and device for identifying defective statements in software code.

[0006] In a first aspect, the present application provides a method for identifying defective statements in software code, the method comprising:

[0007] Obtain the software code to be processed;

[0008] The software code to be processed is identified by using a first large language model to obtain a first set, wherein the first set includes at least one suspected defect sentence in the software code to be processed, wherein the first large language model is used to:

[0009] Checking the software code to be processed according to at least one code check item to obtain a second set, wherein the second set includes at least one suspected defect statement in the software code to be processed, and the first set is a subset of the second set;

[0010] Based on the auxiliary information and the associated statements related to the first suspected defect statement in the software code to be processed, determining whether the first suspected defect statement has a defect, wherein the second set includes the first suspected defect statement, the auxiliary information includes at least one statement and the conditions that each of the statements must satisfy for not having a defect, and if the first suspected defect statement hits the first statement in the auxiliary information, and the associated statements meet the conditions that the first statement must satisfy for not having a defect, then determining that the first suspected defect statement does not have a defect;

[0011] If the first suspected defect sentence does not have a defect, the first set is obtained according to the second set, and the first set does not include the first suspected defect sentence.

[0012] Optionally, the code inspection items include:

[0013] The check logic corresponding to the code check item, or,

[0014] Keywords corresponding to the code check items, and check logic corresponding to the keywords.

[0015] Optionally, the using the first large language model to identify the software code to be processed to obtain a first set includes:

[0016] A first prompt word is input into the first large language model, where the first prompt word is used to prompt the first large language model to identify defective statements in the software code to be processed, and the first prompt word includes first indication information, where the first indication information instructs the first large language model to output a thinking process of identifying defective statements in the software code to be processed, and instructs the first large language model not to add assumptions in the process of identifying defective statements in the software code to be processed.

[0017] Optionally, the first set includes a second suspected defect statement, and the method further includes:

[0018] Analyzing the second suspected defect sentence using the second largest language model to determine that the second suspected defect sentence has a defect when the value of the target field is a target value, and the value of the target field comes from a target data table;

[0019] The second largest language model is used to generate a first test code, where the first test code is used to test whether the value of the target field output by the target data table is equal to the target value.

[0020] Optionally, the method further includes:

[0021] Run the first test code;

[0022] If the execution result of the first test code indicates that the value of the target field output by the target data table is equal to the target value, then it is determined that the second suspected defect statement has a defect; or, if the execution result of the first test code indicates that the value of the target field output by the target data table is not equal to the target value, then it is determined that the second suspected defect statement does not have a defect.

[0023] Optionally, the method further includes:

[0024] Determine the matching degree between the first test code and the second suspected defect sentence by using a third language model;

[0025] When the matching degree is less than or equal to a preset threshold, the first test code is modified by using the third language model to obtain a second test code whose matching degree with the second suspected defect sentence is greater than the preset threshold.

[0026] Optionally, the method further includes:

[0027] Run the second test code;

[0028] If the execution result of the second test code indicates that the value of the target field output by the target data table is equal to the target value, then it is determined that the second suspected defect statement has a defect; or, if the execution result of the second test code indicates that the value of the target field output by the target data table is not equal to the target value, then it is determined that the second suspected defect statement does not have a defect.

[0029] Optionally, the method further includes:

[0030] If the execution result of the target test code indicates that the value of the target field output by the target data table is not equal to the target value, the target test code is periodically executed to periodically determine whether the second suspected defect statement has a defect, wherein the target test code is the first test code or the second test code.

[0031] Optionally, the method further includes:

[0032] If it is determined that the second suspected defect statement has a defect based on the execution result of the target test code, the second suspected defect statement and the location information of the second suspected defect statement in the software code to be processed are output, wherein the target test code is the first test code or the second test code.

[0033] Optionally, the method further includes:

[0034] Obtaining a manual inspection result on whether the second suspected defective sentence has a defect;

[0035] If the manual inspection result indicates that the second suspected defect sentence does not have a defect, the second suspected defect sentence and a condition that makes the second suspected defect sentence not have a defect are added to the auxiliary information.

[0036] In a second aspect, the present application provides a device for identifying defective statements in software code, the device comprising:

[0037] A first acquisition unit, used to acquire the software code to be processed;

[0038] an identification unit, configured to identify the software code to be processed by using a first large language model to obtain a first set, wherein the first set includes at least one suspected defect sentence in the software code to be processed, wherein the first large language model is used to:

[0039] Checking the software code to be processed according to at least one code check item to obtain a second set, wherein the second set includes at least one suspected defect statement in the software code to be processed, and the first set is a subset of the second set;

[0040] Based on the auxiliary information and the associated statements related to the first suspected defect statement in the software code to be processed, determining whether the first suspected defect statement has a defect, wherein the second set includes the first suspected defect statement, the auxiliary information includes at least one statement and the conditions that each of the statements must satisfy for not having a defect, and if the first suspected defect statement hits the first statement in the auxiliary information, and the associated statements meet the conditions that the first statement must satisfy for not having a defect, then determining that the first suspected defect statement does not have a defect;

[0041] If the first suspected defect sentence does not have a defect, the first set is obtained according to the second set, and the first set does not include the first suspected defect sentence.

[0042] Optionally, the code inspection items include:

[0043] The check logic corresponding to the code check item, or,

[0044] Keywords corresponding to the code check items, and check logic corresponding to the keywords.

[0045] Optionally, the identification unit is used to:

[0046] A first prompt word is input into the first large language model, where the first prompt word is used to prompt the first large language model to identify defective statements in the software code to be processed, and the first prompt word includes first indication information, where the first indication information instructs the first large language model to output a thinking process of identifying defective statements in the software code to be processed, and instructs the first large language model not to add assumptions in the process of identifying defective statements in the software code to be processed.

[0047] Optionally, the first set includes a second suspected defect statement, and the device further includes:

[0048] a first determining unit, configured to analyze the second suspected defective sentence by using a second language model, and determine that the second suspected defective sentence has a defect when a value of a target field is a target value, and the value of the target field comes from a target data table;

[0049] A generating unit is used to generate a first test code using the second large language model, wherein the first test code is used to test whether the value of the target field output by the target data table is equal to the target value.

[0050] Optionally, the device further comprises:

[0051] A first running unit, used for running the first test code;

[0052] A second determination unit is used to determine that the second suspected defect statement has a defect if the execution result of the first test code indicates that the value of the target field output by the target data table is equal to the target value; or to determine that the second suspected defect statement does not have a defect if the execution result of the first test code indicates that the value of the target field output by the target data table is not equal to the target value.

[0053] Optionally, the device further comprises:

[0054] A third determining unit, configured to determine a matching degree between the first test code and the second suspected defect sentence by using a third language model;

[0055] A code modification unit is used to modify the first test code using the third language model when the matching degree is less than or equal to a preset threshold, so as to obtain a second test code whose matching degree with the second suspected defect statement is greater than the preset threshold.

[0056] Optionally, the device further comprises:

[0057] A second running unit, used for running the second test code;

[0058] A fourth determination unit is used to determine that the second suspected defect statement has a defect if the execution result of the second test code indicates that the value of the target field output by the target data table is equal to the target value; or to determine that the second suspected defect statement does not have a defect if the execution result of the second test code indicates that the value of the target field output by the target data table is not equal to the target value.

[0059] Optionally, the device further comprises:

[0060] A periodic execution unit is used to periodically execute the target test code if the execution result of the target test code indicates that the value of the target field output by the target data table will not be equal to the target value, so as to periodically determine whether the second suspected defect statement has a defect, wherein the target test code is the first test code or the second test code.

[0061] Optionally, the device further comprises:

[0062] An output unit is used to output the second suspected defect statement and the location information of the second suspected defect statement in the software code to be processed if it is determined that the second suspected defect statement has a defect based on the execution result of the target test code, wherein the target test code is the first test code or the second test code.

[0063] Optionally, the device further comprises:

[0064] A second acquisition unit is used to acquire a manual inspection result of whether the second suspected defective sentence has a defect;

[0065] An information adding unit is used to add the second suspected defect sentence and a condition that makes the second suspected defect sentence non-defective to the auxiliary information if the manual inspection result indicates that the second suspected defect sentence does not have a defect.

[0066] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising a processor and a memory;

[0067] The processor is used to execute instructions stored in the memory so that the electronic device performs the method as described in any one of the first aspects above.

[0068] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, wherein the instructions instruct a device to execute a method as described in any one of the above first aspects.

[0069] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect above.

[0070] Compared with the prior art, the embodiments of the present application have the following advantages:

[0071] The present application provides a method for identifying defective statements in software code, the method comprising: obtaining software code to be processed, and after obtaining the software code to be processed, a large language model can be used to determine the defective statements in the software code to be processed. Specifically, the software code to be processed can be identified by using a first large language model to obtain a first set, wherein the first set includes at least one suspected defective statement in the software code to be processed. The first large language model can determine the first set in two stages, the first stage is to check the software code to be processed according to at least one code check item to obtain a second set, and the second set includes at least one suspected defective statement in the software code to be processed. The second stage is to further filter the suspected defective statements in the second set to obtain a first set, and the first set is a subset of the second set. For any suspected defective statement in the second set (referred to as the first suspected defective statement), the first large language model can determine whether the first suspected defective statement has a defect based on auxiliary information and associated statements related to the first suspected defective statement in the software code to be processed. If the first suspected defective statement does not have a defect, a first set that does not include the first suspected defective statement is obtained according to the second set. Wherein, the auxiliary information includes at least one statement and the conditions that each statement must meet to be defect-free. If the first suspected defect statement hits the first statement in the auxiliary information, and the aforementioned associated statement meets the conditions that the first statement must meet to be defect-free, then the first large language model can determine that the first suspected defect statement does not have a defect. Using this solution, the first large language model can combine the auxiliary information and at least one code check item to accurately determine the suspected defect statement in the software code to be processed. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0073] Figure 1 A flowchart of a method for identifying defective statements in software code provided by an embodiment of the present application;

[0074] Figure 2 A flowchart of a test code generation method provided in an embodiment of the present application;

[0075] Figure 3 A flowchart of another test code generation method provided in an embodiment of the present application;

[0076] Figure 4 A flowchart of a method for identifying defective statements in software code provided by an embodiment of the present application;

[0077] Figure 5 A schematic diagram of the structure of a device for identifying defective statements in software code provided in an embodiment of the present application. DETAILED DESCRIPTION

[0078] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0079] The inventor of the present application has found through research that currently, the software code can be checked manually or with a static code scanning tool to identify defective statements in the software code.

[0080] With manual inspection, the accuracy of the inspection results depends largely on personal experience and lacks a unified quality standard. Even if a strict code inspection process is established, manual inspection still relies on the experience and attention of the inspector. Manual inspection often focuses on the core part of the business logic. Inspectors lack sufficient resources and time to carefully inspect each line of software code, which may cause some defective statements to be ignored, or manual inspection is very likely to fail to discover problems hidden in complex logic or edge cases. Moreover, since software code inspection is an ongoing process, inspectors need to constantly learn and adapt to new programming practices and inspection processes during this process, which places high demands on inspectors.

[0081] The results of code inspection using static code scanning tools are also not very accurate. This is because static code scanning tools identify potential syntax errors and performance issues by analyzing the static structure of software code. However, static code scanning tools usually do not have the ability to perceive the context of the software code at runtime, so they cannot detect problems that only appear in specific data sets or under specific operating conditions. For example, static code scanning tools may not be able to identify logical errors that are only triggered under specific data patterns.

[0082] In order to solve or at least partially solve the above problems, the present application provides a method and device for identifying defective statements in software code, which can accurately identify suspected defective statements in software code.

[0083] Various non-limiting implementations of the present application are described in detail below in conjunction with the accompanying drawings.

[0084] Exemplary Methods

[0085] See also Figure 1 , which is a flow chart of a method for identifying defective statements in software code provided by an embodiment of the present application. The method provided by an embodiment of the present application can be applied to a software testing platform or a software testing system, for example. In the following example, the method is applied to a software testing platform as an example.

[0086] In this embodiment, the method may include the following steps: S101-S102.

[0087] S101: Obtain software code to be processed.

[0088] In one example, a code file input by a user may be obtained, and the initial software code included in the code file may be determined as the software code to be processed.

[0089] In another example, the initial software code may be preprocessed to obtain the software code to be processed. For example, one or more operations such as deleting comments in the initial software code, deleting statements configuring grammar at the beginning of the initial software code, adding line numbers to the code in the initial software code, etc. may be performed to obtain the software code to be processed.

[0090] In one example, the software code to be processed is a code for processing data, for example, a code for performing one or more operations such as screening and processing on data. The software code to be processed may be, for example, a structured query language (SQL) code.

[0091] S102: Using a first large language model to identify the software code to be processed to obtain a first set, wherein the first set includes at least one suspected defect statement in the software code to be processed.

[0092] After obtaining the software code to be processed, at least one large language model (LLM) can be used to identify the software code to be processed to determine the defective sentences in the software code to be processed. In a specific example, the at least one large language model includes at least a first large language model, and the software code to be processed and a first prompt word can be input into the first large language model to prompt the first large language model to identify the defective sentences in the software code to be processed.

[0093] The embodiment of the present application does not specifically limit the first prompt word. In one example, considering that the first large language model may autonomously add some assumptions when performing reasoning tasks (such as defective sentence identification), the assumptions autonomously added by the first large language model may affect the accuracy of the suspected defective sentences it identifies. Therefore, in order to ensure the accuracy of the suspected defective sentences identified by the first large language model, the first prompt word may include first indication information, the first indication information indicates the first large language model outputs the thinking process of identifying the defective sentences in the software code to be processed, and indicates that the first large language model does not add assumptions in the process of identifying the defective sentences in the software code to be processed.

[0094] On the one hand, the first indication information instructs the first large language model not to add assumptions in the process of identifying defective sentences in the software code to be processed, which can avoid the first large language model autonomously adding some assumptions when performing the defective sentence identification task.

[0095] On the other hand, the first indication information indicates the first language model outputs the thinking process of identifying the defective sentences in the software code to be processed, and can also be used to supervise whether the first language model autonomously adds some assumptions when performing the defective sentence identification task. In addition, outputting the thinking process can also help users understand the defective sentence identification ideas of the first language model, and facilitate users to perform corresponding optimization measures in subsequent defective sentence identification tasks, such as optimizing the prompt words input into the first language model, etc. These are not listed here one by one.

[0096] In the present application, after receiving the first prompt word and the software code to be processed, the first large language model may execute steps S1021-S1023 to obtain the first set.

[0097] S1021: Check the software code to be processed according to at least one code check item to obtain a second set, wherein the second set includes at least one suspected defect statement in the software code to be processed, and the first set is a subset of the second set.

[0098] In one example, the first large language model can obtain at least one code inspection item. The code inspection item mentioned in the embodiment of the present application is used to indicate the code to be inspected, wherein the code related to the code inspection item is the code to be inspected. The code to be inspected is the code that may have defects. In a specific example, the code inspection item may include the inspection logic corresponding to the code inspection item. For any code inspection item, the first large language model can determine whether the code inspection item passes the inspection based on the inspection logic corresponding to the code inspection item. In another specific example, the code inspection item may include: keywords corresponding to the code inspection item, and inspection logic corresponding to the keywords. In the process of reading and understanding the software code to be processed, if the first large language model determines that the software code includes the keywords in the code inspection item, it can determine whether the code inspection item passes the inspection based on the inspection logic corresponding to the keywords. In one example, if the code inspection item passes the inspection, the first large language model can determine that the code to be inspected corresponding to the code inspection item does not have defects, and if the code inspection item fails to pass the inspection, it can be determined that the statement included in the code corresponding to the code inspection item is a suspected defect statement. The suspected defective sentences mentioned in this application can be understood as sentences that may have defects, or sentences that are more likely to have defects.

[0099] The embodiment of the present application does not specifically limit the at least one code check item, and the at least one code check item can be obtained, for example, based on manually summarized code check experience. The at least one code check item can be understood in conjunction with the following Table 1, which shows the type of code check item, check logic, keywords, and description of defects in the code check item.

[0100] Table 1

[0101]

[0102]

[0103] It should be noted that Table 1 is only shown for the convenience of understanding the code check items proposed in the embodiment of the present application, and it does not constitute a limitation on the code check items. The code check items in the embodiment of the present application are not limited to the 11 code check items shown in Table 1.

[0104] After checking the software code to be processed according to the at least one code checking item, the large language model may obtain a second set.

[0105] S1022: Determine whether the first suspected defect statement has a defect based on the auxiliary information and the associated statements related to the first suspected defect statement in the software code to be processed, wherein the second set includes the first suspected defect statement, and the auxiliary information includes at least one statement and the conditions that must be met for the statement to not have a defect; if the first suspected defect statement hits the first statement in the auxiliary information, and the associated statements meet the conditions that must be met for the first statement to not have a defect, then determine that the first suspected defect statement does not have a defect.

[0106] In this application, it is considered that in actual applications, some statements will only have defects under specific circumstances. For example, for the statement "If (a + b> 100, 1, 0)", it will only be defective when the value of a or b is NULL. However, if the aforementioned specific situation has been circumvented in the software code, the statement will not pose a risk during operation, that is, the statement does not actually have a defect. In view of this, after obtaining the second set, the first large language model can further confirm whether each suspected defective statement in the second set is indeed defective.

[0107] For the convenience of description, any suspected defect statement in the second set is referred to as a "first suspected defect statement", and the first large language model can determine the associated statements related to the first suspected defect statement in the software code to be processed. Among them, the associated statements related to the first suspected defect statement can be, for example, statements including fields in the first suspected defect statement. For example, the first suspected defect statement is "If(a+b>100, 1, 0)", and the associated statements can be, for example, statements including field a in the software code to be processed, and / or statements including field b in the software code to be processed. Furthermore, the first large language model can combine the associated statements and auxiliary information to determine whether the first suspected defect statement has a defect.

[0108] In one example, the auxiliary information may be an input of the first large language model. The software testing platform may obtain the auxiliary information and input it into the first large language model.

[0109] In the present application, the auxiliary information includes at least one statement (e.g., an SQL statement), which is a statement that has defects in some cases but not in other cases. For each of the at least one statement, the auxiliary information also includes the conditions that must be met for the statement to be defect-free. In view of this, the first large language model can match the first suspected defective statement with the auxiliary information to determine whether the auxiliary information includes a statement that matches the first suspected defective statement. Specifically, if the first suspected defective statement hits the first statement included in the auxiliary information, it means that the first suspected defective statement matches the first statement.

[0110] Among them, the first suspected defect statement matches the first statement, which may mean that the first suspected defect statement has the same grammatical format as the first statement, and the names of the parameters included in the first suspected defect statement do not have to be exactly the same as the names of the parameters included in the first statement. The constants included in the first suspected defect statement do not have to have exactly the same values ​​as the constants included in the first statement. For example, the first suspected statement is "If(a+b>100, 1, 0)", and the first statement can be "If(a+b>c, d, e)", or "If(a+b1>c, d, e)", or "If(a1+b>c, d, e)", or "If(a1+b1>c, d, e)", where the values ​​of c, d, and e are not limited.

[0111] When the first suspected defective sentence matches the first sentence, the first large language model can determine whether the first suspected defective sentence has a defect through the associated sentence and the conditions that need to be met for the first sentence to not have a defect. Specifically, if the associated sentence meets the conditions that need to be met for the first sentence to not have a defect, it can be determined that the first suspected defective sentence does not have a defect. If the associated sentence does not meet the conditions that need to be met for the first sentence to not have a defect, it can be determined that the first suspected defective sentence has a high probability of having a defect. For example:

[0112] The first statement is "If(a+b>c, d, e)", and the conditions that must be met for the first statement to not have a defect are: when the value of a is NULL, a new value is assigned to a, and when the value of b is NULL, a new value is assigned to b. If the first suspected defect statement is "If(a+b>100, 1, 0)", and the associated statements include: "COALESCE(a, 0) AS a" or "NVL(b, 0)", the statement "COALESCE(a, 0) AS a" means that when the value of a is NULL, 0 is assigned to a, and the statement "NVL(b, 0)" means that when the value of b is NULL, the value 0 is returned (that is, 0 is assigned to b). Therefore, it can be determined that the associated statements meet the conditions that must be met for the first statement to not have a defect, and therefore it can be determined that the first suspected defect statement does not have a defect.

[0113] S1023: If the first suspected defect sentence does not have a defect, obtain the first set according to the second set, and the first set does not include the first suspected defect sentence.

[0114] In one example, if the aforementioned associated sentence does not meet the conditions that must be met for the first sentence to be free of defects, it indicates that the first suspected defect sentence is more likely to be defective, and the first suspected defect sentence can be retained. If it is determined based on S1022 that the first suspected defect sentence is free of defects, the first suspected defect sentence can be deleted from the second set to obtain the first set. It is not difficult to understand that the suspected defect sentences in the first set are sentences with a higher probability of being defective.

[0115] It can be seen from the above description that, using this solution, the first large language model can combine the auxiliary information and at least one code inspection item to accurately determine the suspected defective statements in the software code to be processed.

[0116] In one example, after determining the first set, the first large language model can output each suspected defect statement in the first set to facilitate manual inspection or optimization of the each suspected determined statement. Outputting each suspected defect statement in the first set, for example, can be outputting each suspected defect statement and the location information of the suspected defect statement in the software code to be processed, so that the tester can quickly locate the each suspected defect statement from the software code to be processed based on the location information. The location information of the suspected defect statement in the software code to be processed mentioned here includes but is not limited to the line number of the suspected defect statement in the software code to be processed.

[0117] Considering that in actual applications, for some statements that will only have defects under specific circumstances, the specific circumstances are related to the input fields of the statement. And these input fields may be output by the upstream data table. For example, for the statement "If (a + b> 100, 1, 0)", there is a defect when field a or field b is NULL, and the values ​​of field a and field b are both from the target data table. If the values ​​of a and b output by the target data table cannot be NULL, then even if there is no special processing for field a and field b in the software code to be processed (for example, reassigning the values ​​of field a and field b when they are NULL), the statement "If (a + b> 100, 1, 0)" is not defective.

[0118] In view of this, in one example, after obtaining the first set, for at least one (or each) suspected defect statement in the first set, it is possible to further verify whether the suspected defect statement does not have a defect. For the convenience of description, any suspected defect statement in the first set is referred to as a "second suspected defect statement", and for the second suspected defect statement, it is also possible to perform Figure 2 S201-S202 shown. Figure 2 A flowchart of a test code generation method provided in an embodiment of the present application.

[0119] S201: Analyze the second suspected defect sentence using the second largest language model to determine that the second suspected defect sentence has a defect when the value of the target field is a target value, and the value of the target field comes from a target data table.

[0120] In one example, the second largest language model and the first largest language model may be the same model. In another example, the second largest language model and the first largest language model may be two different models.

[0121] In one example, a second prompt word can be input into the second largest language model, and the second prompt word is used to prompt the second largest language model to analyze the conditions under which the second suspected defective sentence is defective. Accordingly, the second largest language model can analyze the second suspected defective sentence based on the second prompt word, and analyze the field that makes the second suspected defective sentence defective, and the specific value of the field. As a specific example, the second largest language model can analyze and obtain that the target field is the field that makes the second suspected defective sentence defective, and when the value of the target field is the target value, the second suspected defective sentence is defective. Taking the second suspected defective sentence as "If(a+b>100, 1, 0)" as an example, the target field may include field a and field b, the target value corresponding to field a is NULL, and the target value corresponding to field b is NULL.

[0122] S202: Generate a first test code using the second largest language model, where the first test code is used to test whether the value of the target field output by the target data table is equal to the target value.

[0123] In the present application, since the value of the target field comes from the target data table, the second largest language model can be used to generate a first test code for testing whether the value of the target field output by the target data table is equal to the target value. For example, if the second suspected defect statement is "If(a+b>100, 1, 0)", the first test code is used to test whether the value of field a output by the target data table is equal to NULL, and whether the value of field b output by the target data table is equal to NULL. In a specific example, the aforementioned second prompt word is also used to prompt the second largest language model to generate the first test code. The first test code mentioned here can be, for example, an SQL code.

[0124] In one example, after the second large language model generates the first test code, the first test code can be output. Accordingly, the software testing platform can further obtain and execute the first test code to obtain the execution result of the first test code. The execution result of the first test code can be used to indicate whether the value of the target field output by the target data table is equal to the aforementioned target value. Accordingly, the software testing platform can determine whether the second suspected defect statement has a defect based on the execution result of the first test code. As a specific example, if the execution result of the first test code indicates that the value of the target field output by the target data table will be equal to the target value (for example, the aforementioned a will be equal to NULL, or b will be equal to NULL), it can be determined that the second suspected defect statement has a defect. As another specific example, if the execution result of the first test code indicates that the value of the target field output by the target data table will not be equal to the target value, it can be determined that the second suspected defect statement does not have a defect.

[0125] In some embodiments, considering that the value output by the target data table may change for the target field, when the execution result of the first test code indicates that the value of the target field output by the target data table is not equal to the target value, the first test code may be executed periodically to periodically determine whether the second suspected defective statement has a defect. That is, the first test code is executed periodically to determine in real time whether the second suspected defective statement has a defect.

[0126] In other embodiments, if the second suspected defect statement is determined to be defective according to the execution result of the first test code, the software testing platform can output the second suspected defect statement and the location information of the second suspected defect statement in the software code to be processed, so as to facilitate the user (such as an inspector) to manually check the second suspected statement. Among them, the location information of the second suspected defect statement in the software code to be processed can be, for example, the line number of the second suspected defect statement in the software code to be processed. Outputting this location information can help the user quickly locate the second suspected defect statement in the software code to be processed, which is conducive to improving the efficiency of the user's manual inspection of the second suspected defect statement.

[0127] After the user manually checks the second suspected defective statement, the software testing platform can also obtain the manual inspection result for whether the second suspected defective statement has a defect. If the manual inspection result indicates that the second suspected defective statement does not have a defect, it means that the conclusion that "the second suspected defective statement has a defect" determined by the first large language model and the second large language model is not accurate. In this case, in order to optimize the reasoning effect of the first large language model, the software testing platform can add the second suspected defective statement and the conditions that make the second suspected defective statement non-defective to the aforementioned auxiliary information, so that the first large language model can refer to the second suspected defective statement and the conditions that make the second suspected defective statement non-defective in subsequent reasoning tasks. The condition that the second suspected defective statement does not have a defect mentioned here can be input by the user, which can be specifically determined by the user when manually checking the second suspected defective statement in combination with the software code to be processed.

[0128] In one example, considering that the large language model will add some assumptions when performing reasoning tasks. If the second large language model adds assumptions when generating the first test code, the first test code may not match the second suspected defect statement. For example, the first test code may include content that is irrelevant to the second suspected defect statement. In view of this, in one example, after obtaining the first test code, you can also execute Figure 3 S301-S302 shown. Figure 3 A flowchart of another test code generation method provided in an embodiment of the present application.

[0129] S301: Determine the matching degree between the first test code and the second suspected defect statement by using a third language model.

[0130] In one example, the software testing platform may input a third prompt word into the third language model, and the third prompt word may be used to instruct the third language model to determine the degree of match between the first test code and the suspected defect statement. The third prompt word may include at least the first test code and the second suspected defect statement.

[0131] Correspondingly, after receiving the third prompt word, the third language model can analyze the first test code and the second suspected defect statement according to the third prompt word to determine the degree of matching between the first test code and the second suspected defect statement. Wherein: the degree of matching between the first test code and the second suspected defect statement can be, for example, the degree of matching between the fields in the first test code and the second suspected defect statement. For example, if the second suspected defect statement includes field a and field b, but the first test code does not include field a or b, or, although the first test code includes field a and b, it also includes more other fields that are not related to a and b, it means that the degree of matching between the fields in the first test code and the second suspected defect statement is not very high. For another example, if the second suspected defect statement includes field a and field b, and the first test code includes field a or b, and does not include other fields that are not related to a and b, it means that the degree of matching between the fields in the first test code and the second suspected defect statement is high.

[0132] In one example, the third largest language model and the second largest language model may be the same model. In another example, the third largest language model and the second largest language model may be two different models.

[0133] In one example, the third largest language model and the first largest language model may be the same model. In another example, the third largest language model and the first largest language model may be two different models.

[0134] S302: When the matching degree is less than or equal to a preset threshold, modify the first test code using the third language model to obtain a second test code whose matching degree with the second suspected defect sentence is greater than the preset threshold.

[0135] The degree of match is less than or equal to the preset threshold, indicating that the degree of match between the fields in the first test code and the second suspected defect statement is not very high. In this case, on the one hand, if the first test code does not include statements related to the fields in the second suspected defect statement, executing the first test code may not accurately obtain the execution result of whether the value of the target field is equal to the target value. On the other hand, although the first test code does not include statements related to the fields in the second suspected defect statement, the first test code also includes more statements that are not related to the fields in the second suspected defect statement, the execution efficiency of the first test code is low.

[0136] In view of this, for this situation, the first test code can also be modified using the third language model to obtain a second test code whose matching degree with the second suspected defect statement is greater than the preset threshold. As a specific example, the aforementioned third prompt word can also include second indication information, and the second indication information is used to instruct the third largest language model to modify the first test code when the aforementioned matching degree is less than or equal to the preset threshold, so as to obtain a second test code whose matching degree with the second suspected defect statement is greater than the preset threshold. Accordingly, the third largest language model can generate a second test code based on the first test code based on the second indication information. Similar to the first test code, the second test code can also be an SQL code.

[0137] In one example, after the third language model generates the second test code, the second test code can be output. Accordingly, the software testing platform can further obtain and execute the second test code to obtain the execution result of the second test code. The execution result of the second test code can be used to indicate whether the value of the target field output by the target data table is equal to the aforementioned target value. Accordingly, the software testing platform can determine whether the second suspected defect statement has a defect based on the execution result of the second test code. As a specific example, if the execution result of the second test code indicates that the value of the target field output by the target data table will be equal to the target value (for example, the aforementioned a will be equal to NULL, or b will be equal to NULL), it can be determined that the second suspected defect statement has a defect. As another specific example, if the execution result of the second test code indicates that the value of the target field output by the target data table will not be equal to the target value, it can be determined that the second suspected defect statement does not have a defect.

[0138] In some embodiments, considering that the value output by the target data table may change for the target field, when the execution result of the second test code indicates that the value of the target field output by the target data table is not equal to the target value, the second test code may be executed periodically to periodically determine whether the second suspected defective statement has a defect. That is, the second test code is executed periodically to determine whether the second suspected defective statement has a defect in real time.

[0139] In other embodiments, if it is determined that the second suspected defect statement has a defect based on the execution result of the second test code, the software testing platform can output the second suspected defect statement and the location information of the second suspected defect statement in the software code to be processed, so as to facilitate the user (such as an inspector) to manually check the second suspected statement. Accordingly, after the user manually checks the second suspected defect statement, the software testing platform can also obtain the manual inspection result of whether the second suspected defect statement has a defect. And further perform corresponding processing based on the manual inspection result. Regarding the specific implementation of "performing corresponding processing based on the manual inspection result", please refer to the relevant description in the previous text, and no repeated description will be made here.

[0140] The above describes the method provided in the embodiment of the present application. Figure 4 , introduces a possible implementation method of the embodiment of the present application.

[0141] See also Figure 4 , which is a flow chart of a method for identifying defective statements in software code provided in an embodiment of the present application.

[0142] like Figure 4 As shown, the software testing platform can perform the following steps in sequence:

[0143] 1. First, the software code submitted by the user is obtained, and further, the software code submitted by the user is preprocessed to obtain the software code to be processed.

[0144] 2. Use the first language model to identify defective sentences in the software code to be processed to obtain a first set.

[0145] The first language model can combine the knowledge base and the auxiliary information when identifying the software code to be processed, and the knowledge base includes at least one code inspection item. The first set includes the second suspected defect statement.

[0146] 3. For the second suspected defect sentence, use the second largest language model to generate a first test code for testing the target data table.

[0147] For the target data table, the first test code, etc., please refer to the relevant description in the previous text, and no repeated description is made here.

[0148] 4. Use the third language model to check the first test code, and generate a second test code based on the first test code.

[0149] 5. Execute the second test code and collect the execution result of the second test code.

[0150] 6. According to the execution result of the second test code, determine whether the value of the target field is equal to the target value.

[0151] 7. If the execution result of the second test code indicates that the value of the target field will not be equal to the target value, the second test code is converted into a periodically executed task.

[0152] Converting the second test code into a periodically executed task refers to subsequently periodically executing the second test code.

[0153] 8. If the execution result of the second test code indicates that the value of the target field will be equal to the target value, the second suspected defect statement is manually checked.

[0154] 9. If the manual inspection determines that the second suspected defect statement does not have a defect, the second suspected defect statement and the condition that makes the second suspected defect statement not have a defect are added to the auxiliary information. 10. If the manual inspection determines that the second suspected defect statement does have a defect, the developer can further optimize the second suspected defect statement.

[0155] For the specific implementation of the above steps 1-10, please refer to the relevant description in the previous text, and no repetitive description is given here.

[0156] Exemplary Devices

[0157] Based on the method provided in the above embodiment, the embodiment of the present application further provides a device, which is described below in conjunction with the accompanying drawings.

[0158] See also Figure 5 , which is a structural diagram of a device for identifying defective statements in software code provided in an embodiment of the present application. Figure 5 The device 500 shown is used to execute the method for identifying defective statements in software code provided by the above method embodiment.

[0159] like Figure 5 As shown, the device 500 includes: a first acquisition unit 501 and an identification unit 502.

[0160] The first acquisition unit 501 is used to acquire the software code to be processed;

[0161] The identification unit 502 is used to identify the software code to be processed by using a first large language model to obtain a first set, wherein the first set includes at least one suspected defect statement in the software code to be processed, wherein the first large language model is used to:

[0162] Checking the software code to be processed according to at least one code check item to obtain a second set, wherein the second set includes at least one suspected defect statement in the software code to be processed, and the first set is a subset of the second set;

[0163] Based on the auxiliary information and the associated statements related to the first suspected defect statement in the software code to be processed, determining whether the first suspected defect statement has a defect, wherein the second set includes the first suspected defect statement, the auxiliary information includes at least one statement and the conditions that each of the statements must satisfy for not having a defect, and if the first suspected defect statement hits the first statement in the auxiliary information, and the associated statements meet the conditions that the first statement must satisfy for not having a defect, then determining that the first suspected defect statement does not have a defect;

[0164] If the first suspected defect sentence does not have a defect, the first set is obtained according to the second set, and the first set does not include the first suspected defect sentence.

[0165] Optionally, the code inspection items include:

[0166] The check logic corresponding to the code check item, or,

[0167] Keywords corresponding to the code check items, and check logic corresponding to the keywords.

[0168] Optionally, the identification unit 502 is used to:

[0169] A first prompt word is input into the first large language model, where the first prompt word is used to prompt the first large language model to identify defective statements in the software code to be processed, and the first prompt word includes first indication information, where the first indication information instructs the first large language model to output a thinking process of identifying defective statements in the software code to be processed, and instructs the first large language model not to add assumptions in the process of identifying defective statements in the software code to be processed.

[0170] Optionally, the first set includes a second suspected defect statement, and the device further includes:

[0171] a first determining unit, configured to analyze the second suspected defective sentence by using a second language model, and determine that the second suspected defective sentence has a defect when a value of a target field is a target value, and the value of the target field comes from a target data table;

[0172] A generating unit is used to generate a first test code using the second large language model, wherein the first test code is used to test whether the value of the target field output by the target data table is equal to the target value.

[0173] Optionally, the device further comprises:

[0174] A first running unit, used for running the first test code;

[0175] A second determination unit is used to determine that the second suspected defect statement has a defect if the execution result of the first test code indicates that the value of the target field output by the target data table is equal to the target value; or to determine that the second suspected defect statement does not have a defect if the execution result of the first test code indicates that the value of the target field output by the target data table is not equal to the target value.

[0176] Optionally, the device further comprises:

[0177] A third determining unit, configured to determine a matching degree between the first test code and the second suspected defect sentence by using a third language model;

[0178] A code modification unit is used to modify the first test code using the third language model when the matching degree is less than or equal to a preset threshold, so as to obtain a second test code whose matching degree with the second suspected defect statement is greater than the preset threshold.

[0179] Optionally, the device further comprises:

[0180] A second running unit, used for running the second test code;

[0181] A fourth determination unit is used to determine that the second suspected defect statement has a defect if the execution result of the second test code indicates that the value of the target field output by the target data table is equal to the target value; or to determine that the second suspected defect statement does not have a defect if the execution result of the second test code indicates that the value of the target field output by the target data table is not equal to the target value.

[0182] Optionally, the device further comprises:

[0183] A periodic execution unit is used to periodically execute the target test code if the execution result of the target test code indicates that the value of the target field output by the target data table will not be equal to the target value, so as to periodically determine whether the second suspected defect statement has a defect, wherein the target test code is the first test code or the second test code.

[0184] Optionally, the device further comprises:

[0185] An output unit is used to output the second suspected defect statement and the location information of the second suspected defect statement in the software code to be processed if it is determined that the second suspected defect statement has a defect based on the execution result of the target test code, wherein the target test code is the first test code or the second test code.

[0186] Optionally, the device further comprises:

[0187] A second acquisition unit is used to acquire a manual inspection result of whether the second suspected defective sentence has a defect;

[0188] An information adding unit is used to add the second suspected defect sentence and a condition that makes the second suspected defect sentence non-defective to the auxiliary information if the manual inspection result indicates that the second suspected defect sentence does not have a defect.

[0189] Since the device 500 is a device corresponding to the method for identifying defective statements in software code provided by the above method embodiment, the specific implementation of each unit of the device 500 is based on the same concept as the above method embodiment. Therefore, regarding the specific implementation of each unit of the device 500, please refer to the relevant description part of the above method embodiment, which will not be repeated here.

[0190] The embodiment of the present application also provides an electronic device, the electronic device comprising a processor and a memory;

[0191] The processor is used to execute the instructions stored in the memory, so that the electronic device executes the method for identifying defective statements in software codes provided by the above method embodiment.

[0192] An embodiment of the present application provides a computer-readable storage medium, including instructions, wherein the instructions instruct a device to execute the method for identifying defective statements in software code provided by the above method embodiment.

[0193] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the method for identifying defective statements in software code provided by the above method embodiment.

[0194] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0195] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0196] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for identifying defective statements in software code, characterized in that: The method comprises: Obtain the software code to be processed; The software code to be processed is identified by using a first large language model to obtain a first set, wherein the first set includes at least one suspected defect sentence in the software code to be processed, wherein the first large language model is used to: Checking the software code to be processed according to at least one code check item to obtain a second set, wherein the second set includes at least one suspected defect statement in the software code to be processed, and the first set is a subset of the second set; Based on the auxiliary information and the associated statements related to the first suspected defect statement in the software code to be processed, determining whether the first suspected defect statement has a defect, wherein the second set includes the first suspected defect statement, the auxiliary information includes at least one statement and the conditions that each of the statements must satisfy for not having a defect, and if the first suspected defect statement hits the first statement in the auxiliary information, and the associated statements meet the conditions that the first statement must satisfy for not having a defect, then determining that the first suspected defect statement does not have a defect; If the first suspected defect sentence does not have a defect, the first set is obtained according to the second set, and the first set does not include the first suspected defect sentence.

2. The method according to claim 1, characterized in that The code inspection items include: The check logic corresponding to the code check item, or, Keywords corresponding to the code check items, and check logic corresponding to the keywords.

3. The method according to claim 1, characterized in that The using the first large language model to identify the software code to be processed to obtain a first set includes: A first prompt word is input into the first large language model, where the first prompt word is used to prompt the first large language model to identify defective statements in the software code to be processed, and the first prompt word includes first indication information, where the first indication information instructs the first large language model to output a thinking process of identifying defective statements in the software code to be processed, and instructs the first large language model not to add assumptions in the process of identifying defective statements in the software code to be processed.

4. The method according to claim 1, characterized in that: The first set includes a second suspected defect statement, and the method further includes: Analyzing the second suspected defect sentence using the second largest language model to determine that the second suspected defect sentence has a defect when the value of the target field is a target value, and the value of the target field comes from a target data table; The second largest language model is used to generate a first test code, where the first test code is used to test whether the value of the target field output by the target data table is equal to the target value.

5. The method according to claim 4, characterized in that The method further comprises: Run the first test code; If the execution result of the first test code indicates that the value of the target field output by the target data table is equal to the target value, then it is determined that the second suspected defect statement has a defect; or, if the execution result of the first test code indicates that the value of the target field output by the target data table is not equal to the target value, then it is determined that the second suspected defect statement does not have a defect.

6. The method according to claim 4, characterized in that The method further comprises: Determine the matching degree between the first test code and the second suspected defect sentence by using a third language model; When the matching degree is less than or equal to a preset threshold, the first test code is modified by using the third language model to obtain a second test code whose matching degree with the second suspected defect sentence is greater than the preset threshold.

7. The method according to claim 6, characterized in that The method further comprises: Run the second test code; If the execution result of the second test code indicates that the value of the target field output by the target data table is equal to the target value, then it is determined that the second suspected defect statement has a defect; or, if the execution result of the second test code indicates that the value of the target field output by the target data table is not equal to the target value, then it is determined that the second suspected defect statement does not have a defect.

8. The method according to claim 5 or 7, characterized in that: The method further comprises: If the execution result of the target test code indicates that the value of the target field output by the target data table is not equal to the target value, the target test code is periodically executed to periodically determine whether the second suspected defect statement has a defect, wherein the target test code is the first test code or the second test code.

9. The method according to claim 5 or 7, characterized in that: The method further comprises: If it is determined that the second suspected defect statement has a defect based on the execution result of the target test code, the second suspected defect statement and the location information of the second suspected defect statement in the software code to be processed are output, wherein the target test code is the first test code or the second test code.

10. The method according to claim 9, characterized in that The method further comprises: Obtaining a manual inspection result on whether the second suspected defective sentence has a defect; If the manual inspection result indicates that the second suspected defect sentence does not have a defect, the second suspected defect sentence and a condition that makes the second suspected defect sentence not have a defect are added to the auxiliary information.

11. A device for identifying defective statements in software code, characterized in that: The device comprises: A first acquisition unit, used to acquire the software code to be processed; an identification unit, configured to identify the software code to be processed by using a first large language model to obtain a first set, wherein the first set includes at least one suspected defect sentence in the software code to be processed, wherein the first large language model is used to: Checking the software code to be processed according to at least one code check item to obtain a second set, wherein the second set includes at least one suspected defect statement in the software code to be processed, and the first set is a subset of the second set; Based on the auxiliary information and the associated statements related to the first suspected defect statement in the software code to be processed, determining whether the first suspected defect statement has a defect, wherein the second set includes the first suspected defect statement, the auxiliary information includes at least one statement and the conditions that each of the statements must satisfy for not having a defect, and if the first suspected defect statement hits the first statement in the auxiliary information, and the associated statements meet the conditions that the first statement must satisfy for not having a defect, then determining that the first suspected defect statement does not have a defect; If the first suspected defect sentence does not have a defect, the first set is obtained according to the second set, and the first set does not include the first suspected defect sentence.

12. An electronic device, characterized in that: The electronic device comprises a processor and a memory; The processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The method comprises instructions, wherein the instructions instruct a device to execute the method according to any one of claims 1 to 10.

Images