Method and system for detecting application database table validity

Abstract

The application discloses a kind of application program database table effectiveness detection method and system, this method includes: obtaining any two or all in first data set, second data set and third data set;Database address is extracted based on when application program connects database all database table information, to obtain first data set;The data file related to database operation in application program is parsed, to obtain all database table information recorded in the code of this application program, to obtain second data set;All database table information triggered in functional test by application program is obtained, to obtain third data set;Any two in first data set, second data set, third data set respectively calculates three coverage indexes.Based on the above detection method, the degree of effective database of application program can be automatically and quickly detected to help users to sort out application program in real time, effectively improve the maintenance efficiency of application program, and save labor cost.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of application program effective code detection, and in particular to an application program database table effectiveness detection method and system. BACKGROUND

[0002] With the popularization of distributed applications, the original single application program accessing a single database scene gradually evolves into the current micro-service accessing multiple database scene, the number of application programs is increasing, and the number of database tables is also increasing. However, while the business is iterated, the previous database tables may become obsolete and unused. If no one handles it, the number of obsolete database tables will increase, and it will be difficult to maintain in the end.

[0003] For example, in a certain application iteration, there are four modules of user module, group module, enterprise module and role module in the 1.0 version, which correspond to four tables of user, group, org and role; the group module is cancelled in the 2.0 version, and only the user module, the enterprise module and the role module are retained, which correspond to three tables of user, org and role, and the group table is deleted in the database in the 3.0 iteration, but some obsolete codes in the code that call the group table are not deleted, so there are four tables in the business code, and only three tables in the database.

[0004] In this regard, during the development and maintenance of the application program, the application program needs to be combed frequently to detect and delete the obsolete codes in the application program. In the prior art, there is a technology for automatically deleting obsolete codes in the application program, for example, the Chinese invention patent with the publication number CN111597149A discloses a "database data cleaning method and device", which finds all operable data shards by traversing the database and writes them into a newly created data file. When the cleaning instruction is executed, the newly created data file is browsed to quickly locate the operable data shards with a deletion mark, and the operable data shards are deleted in a preset executable cleaning instruction time period to achieve the purpose of cleaning the database. Although the patent scheme can automatically delete obsolete databases in the application program, for program maintenance personnel, it is not clear when the application program needs to be combed, therefore, at present, the effectiveness of the database tables in the application program is generally detected by manual detection at intervals, which not only has low detection efficiency, but also has low accuracy, especially when facing a large amount of data, it is particularly time-consuming and laborious. SUMMARY

[0005] The application aims to provide an application program database table effectiveness detection method and system which can automatically analyze and detect the effectiveness of the database tables in the application program.

[0006] In order to achieve the above object, the application discloses a method for detecting the effectiveness of an application database table, which comprises the following steps:

[0007] The application program to be tested is instrumented by using an instrumenting tool to weave the tracking detection logic into the application program;

[0008] Any two or all of the first data set, the second data set and the third data set are obtained;

[0009] The first data set is obtained by obtaining the database address when the application program connects to the database, and extracting all the database table information based on the database address to obtain the first data set;

[0010] The second data set is obtained by parsing the data file related to the database operation in the application program to obtain all the database table information recorded in the code of the application program to obtain the second data set;

[0011] The third data set is obtained by obtaining all the database table information triggered in the functional test of the application program to obtain the third data set;

[0012] The degree to which the second data set covers the first data set is calculated to obtain a first coverage degree; and / or the degree to which the third data set covers the second data set is calculated to obtain a second coverage degree; and / or the degree to which the third data set covers the first data set is calculated to obtain a third coverage degree.

[0013] Preferably, when the first coverage degree and the second coverage degree are obtained, the product of the first coverage degree and the second coverage degree is also calculated to obtain a fourth coverage degree.

[0014] Preferably, the first coverage degree and the third coverage degree can be calculated from one or more of the dimensions of the overall database, the dimensions of all the tables in a certain database and the dimensions of the tables of all the databases; the second coverage degree can be calculated from one or more of the dimensions of the tables in a certain application database and the dimensions of the tables in all the application databases.

[0015] Preferably, the code file and the xml file containing the SQL fragment in the application program are parsed to obtain all the database table information recorded in the code of the application program.

[0016] Preferably, the SQL statement information executed when the application program accesses the database is parsed to extract the database table information.

[0017] Preferably, the first coverage is calculated by finding the intersection of the second data set and the first data set to obtain a first arithmetic number representing the number of common objects in the second data set and the first data set, and calculating the ratio of the first arithmetic number to a second arithmetic number representing the number of corresponding objects in the first data set to obtain the first coverage; the second coverage is calculated by finding the intersection of the third data set and the second data set to obtain a third arithmetic number representing the number of common objects in the third data set and the second data set, and calculating the ratio of the third arithmetic number to a fourth arithmetic number representing the number of corresponding objects in the second data set to obtain the second coverage; and the third coverage is calculated by finding the intersection of the third data set and the first data set to obtain a fifth arithmetic number representing the number of functional objects in the third data set and the first data set, and calculating the ratio of the fifth arithmetic number to a sixth arithmetic number representing the number of corresponding objects in the first data set to obtain the third coverage.

[0018] The application further discloses a detection system of application database table coverage, which works based on the detection method.

[0019] The application further discloses a detection system of application database table coverage, which comprises:

[0020] one or more processors;

[0021] a memory;

[0022] and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs comprise instructions for executing the detection method of application database table coverage.

[0023] The application further discloses a computer readable storage medium, characterized by comprising a computer program, wherein the computer program can be executed by a processor to complete the detection method of application database table coverage.

[0024] Compared with the prior art, the detection method disclosed in the technical scheme of the application inserts the application program to be detected by the instrumentation tool, and automatically obtains database table information from different channels in combination with program analysis technology, and calculates a plurality of indexes respectively representing the effectiveness of the database table of the application program in different states, namely the first coverage, the second coverage and the third coverage. For any index, the current effectiveness of the application program to the database settings or application can be indicated in a certain aspect, thereby providing an indication for the user on whether the application program needs to be combed currently, and the user can also purposefully arrange the corresponding database according to the obtained indexes. Therefore, according to the detection method, the effectiveness of the effective database of the application program can be automatically and quickly detected, to help the user to comb the application program in real time, effectively improve the maintenance efficiency of the application program, and save the labor cost. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 The detection method flowchart in the embodiments of the application. DETAILED DESCRIPTION

[0026] To explain the technical content, structural features, achieved purposes and effects of the application in detail, the embodiments are combined with the drawings for detailed description.

[0027] The embodiment discloses a detection method for the effectiveness of the database table of an application program, to detect the code effectiveness of the application program by detecting the effectiveness of the database table in the application program, thereby providing a standard for the program maintenance personnel on whether the application program needs to be combed currently. As shown in Figure 1 The detection method comprises the following steps:

[0028] S1: An instrumentation tool is used to instrument the application program to be detected, to weave the tracking detection logic into the application program.

[0029] S2: Any two or all of the first data set, the second data set and the third data set are obtained, namely the first data set, the second data set, or the first data set, the third data set, or the second data set, the third data set, or the first data set, the second data set and the third data set.

[0030] The acquisition method of the first data set is to obtain the database address when the application program connects the database, to access the corresponding database according to the database address, thereby extracting all the database table information, to obtain the first data set. The first data set in the embodiment comprises the information of all the databases and the tables in each database which are adapted to the application program.

[0031] The method for obtaining the second data set is to parse the data file related to the database operation in the application program to obtain all the database table information recorded in the code of the application program, so as to obtain the second data set. In the embodiment, since the data file related to the database operation is parsed, the second data set obtained includes all the database table information recorded in the code of the application program and the information of the tables in each database.

[0032] The method for obtaining the third data set is to obtain all the database table information triggered by the application program in the function test, so as to obtain the third data set. In the embodiment, the application program is started and the function of the application program is tested, so that the application program accesses the database in the test running process. Therefore, the database information accessed by the application program in the stage is tracked and obtained by the plug-in tool, so as to obtain the database table information representing the running trigger of the application program.

[0033] S3: calculating the degree of coverage of the second data set on the first data set to obtain the first coverage degree; and / or calculating the degree of coverage of the third data set on the second data set to obtain the second coverage degree; and / or calculating the degree of coverage of the third data set on the first data set to obtain the third coverage degree. In the step, when only the first data set and the second data set are obtained, the first coverage degree is output. When only the second data set and the third data set are obtained, the second coverage degree is output. When only the first data set and the third data set are obtained, the third coverage degree is output. In addition, when the first data set, the second data set and the third data set are obtained at the same time, the first coverage degree, the second coverage degree and the third coverage degree can be output.

[0034] Through the automatic generation of the three indexes (the first coverage degree, the second coverage degree and the third coverage degree) in the above embodiment, the user can understand the effective degree of the code in the current application program from different angles. For example, for the first coverage degree, the static effectiveness of the code of the application program is represented, that is, the proportion of the database actually used by the application program, so that it can be quickly counted which databases are not used. For the second coverage degree and the third coverage degree, the dynamic effectiveness of the code of the application program is represented, that is, the proportion of the database used in the test process of the application program, so that it can be quickly counted which databases in the code of the application program and the created database system are not dynamically accessed.

[0035] Further, the calculation method of the first coverage degree is to obtain the intersection of the second data set and the first data set to obtain the first arithmetic number representing the number of common objects in the second data set and the first data set, and to calculate the ratio of the first arithmetic number to the second arithmetic number representing the number of corresponding objects in the first data set, so as to obtain the first coverage degree.

[0036] The second coverage is calculated by finding the intersection of the third data set and the second data set to obtain a third arithmetic number representing the number of common objects in the third data set and the second data set, and calculating the ratio of the third arithmetic number to a fourth arithmetic number representing the number of corresponding objects in the second data set to obtain the second coverage.

[0037] The third coverage is calculated by finding the intersection of the third data set and the first data set to obtain a fifth arithmetic number representing the number of functional objects in the third data set and the first data set, and calculating the ratio of the fifth arithmetic number to a sixth arithmetic number representing the number of corresponding objects in the first data set to obtain the third coverage.

[0038] Specifically, the first coverage P1 is calculated based on Formula One below, the second coverage P2 is calculated based on Formula Two below, and the third coverage P3 is calculated based on Formula Three below.

[0039]

[0040]

[0041]

[0042] wherein N1 is the first arithmetic number, N2 is the second arithmetic number, N3 is the third arithmetic number, N4 is the fourth arithmetic number, N5 is the fifth arithmetic number, and N6 is the sixth arithmetic number.

[0043] A is the first data set, B is the second data set, and C is the third data set.

[0044] Further, the first coverage and the third coverage can be calculated from one or more of the dimensions of the overall database, the dimensions of all tables in a certain database, and the dimensions of tables in all databases. The second coverage can be calculated from one or more of the dimensions of tables in a certain application database and the dimensions of tables in all application databases.

[0045] Specifically, for a certain application program, the obtained first data set A is as follows: [

[0047] {"dbName":"webgoat_db",

[0048] "tables":[

[0049] "user","product","config_info","app_user","app_info"

[0050] ]},

[0051] {"dbName":"test_db",

[0052] "tables": [

[0053] "test_user","test_group","test_role","test_config","test_car" ]

[0055] },

[0056] {"dbName":"navis_db",

[0057] "tables": [

[0058] "vul","risk","server","image" ]

[0060] }]

[0061] Thus, three databases are recorded in the first data set A, respectively: "webgoat_db", "test_db", "navis_db".

[0062] In addition, in the database "webgoat_db", five tables are recorded, respectively: "user", "product", "config_info", "app_user", "app_info".

[0063] In the database "test_db", five tables are recorded, respectively: "test_user", "test_group", "test_role", "test_config", "test_car".

[0064] In the database "navis_db", four tables are recorded, respectively: "vul", "risk", "server", "image".

[0065] The second data set B obtained is as follows:

[0066] [{"appName":"webgoat",

[0067] "dbName":"webgoat_db",

[0068] "tables": [

[0069] "user","product","config_info","app_user"

[0070] ]},

[0071] {"appName": "navis",

[0072] "dbName": "navis_db",

[0073] "tables": [

[0074] "vul", "risk", "server"

[0075] ]}

[0076] Thus, in the second data set B, two applications are recorded, which are "webgoat" and "navis" respectively.

[0077] The database in the application "webgoat" is "webgoat_db", and in the database "webgoat_db", four tables are recorded, which are "user", "product", "config_info", "app_user" respectively.

[0078] The database in the application "navis" is "navis_db", and in the database "navis_db", three tables are recorded, which are "vul", "risk", "server" respectively.

[0079] The third data set C obtained is as follows:

[0080] {"appName": "webgoat",

[0081] "dbName": "webgoat_db",

[0082] "tables": [

[0083] "user", "product", "config_info"

[0084] ]},

[0085] {"appName": "navis",

[0086] "dbName": "navis_db",

[0087] "tables": [

[0088] "vul", "risk", "server"

[0089] ]}

[0090] Thus, in the second data set C, two applications are recorded, respectively "webgoat" and "navis".

[0091] The database in the application "webgoat" is "webgoat_db", and in the database "webgoat_db", three tables are recorded, respectively "user", "product", "config_info".

[0092] The database in the application "navis" is "navis_db", and in the database "navis_db", three tables are recorded, respectively "vul", "risk", "server".

[0093] For the first coverage P1, if analyzed from the dimension of the overall database:

[0094] A: ["webgoat_db", "navis_db", "test_db"];

[0095] B: ["webgoat_db", "navis_db"];

[0096] Therefore, N1 is 2, N2 is 3, and P1 = 2 / 3.

[0097] For the first coverage P1, if analyzed from the dimension of all tables in a database, taking the database "webgoat_db" as an example:

[0098] A: ["user", "product", "config_info", "app_user", "app_info"];

[0099] B: ["user", "product", "config_info", "app_user"];

[0100] Therefore, N1 is 4, N2 is 5, and P1 = 4 / 5.

[0101] For the first coverage P1, if analyzed from the dimension of all tables in all databases:

[0102] A: ["user", "product", "config_info", "app_user", "app_info", "test_user", "test_group", "test_role", "test_config", "test_car", "vul", "risk", "server", "image"];

[0103] B: ["user", "product", "config_info", "app_user", "vul", "risk", "server"];

[0104] Thus, N1 is 7, N2 is 14, then P1 = 7 / 14.

[0105] For the second coverage P2, from the dimension analysis of the tables in the database of an application, take the application webgoat in the above example as an example:

[0106] B: ["user", "product", "config_info"];

[0107] C: ["user", "product", "config_info", "app_user"];

[0108] Thus, N3 is 3, N4 is 4, then P2 = 3 / 4.

[0109] For the second coverage P2, from the dimension analysis of the tables in the database of all applications:

[0110] B: ["user", "product", "config_info", "vul", "risk", "server"];

[0111] C: ["user", "product", "config_info", "app_user", "vul", "risk", "server"];

[0112] Thus, N3 is 6, N4 is 7, then P2 = 6 / 7.

[0113] For the third coverage P3, if from the dimension analysis of the overall database:

[0114] A: ["webgoat_db", "navis_db", "test_db"];

[0115] C: ["webgoat_db", "navis_db"];

[0116] Thus, N5 is 2, N6 is 3, then P3 = 2 / 3.

[0117] For the third coverage P3, if from the dimension analysis of all tables in a certain database, take the database "webgoat_db" as an example:

[0118] A: ["user", "product", "config_info", "app_user", "app_info"];

[0119] C: ["user", "product", "config_info"];

[0120] Therefore, N5 is 3, N6 is 5, and P3 = 3 / 5.

[0121] For the third coverage P3, if analyzing from the dimension of all the tables of the database:

[0122] A: ["user", "product", "config_info", "app_user", "app_info", "test_user", "test_group", "test_role", "test_config", "test_car", "vul", "risk", "server", "image"];

[0123] C: ["user", "product", "config_info", "vul", "risk", "server"];

[0124] Therefore, N5 is 6, N6 is 14, and P3 = 6 / 14.

[0125] Further, since all the actual database tables used by the application program cannot be represented in the functional test, because there can be some abandoned business code in the application program, containing non-existent database tables, in order to obtain the proportion of the database tables actually used by the application program in the total set of effective database tables in the application code during the actual running of the application program, when the first coverage and the second coverage are obtained, the product of the first coverage and the second coverage is calculated to obtain the fourth coverage.

[0126] For example, when the first coverage P1 obtained by analyzing from the dimension of all the tables of a certain database is 4 / 5, and the second coverage P2 is 3 / 4, then the fourth coverage P4 = (4 / 5)*(3 / 4) = 3 / 5. That is, the actual coverage ratio of the application navis is 3 / 4. For the fourth coverage, the greater the value, the less the abandoned code of the application program, the stronger the functional test coverage, and the closer the association between the application and the database.

[0127] In another aspect, when the second data set is acquired, the code files and xml files containing SQL fragments in the application program are obtained by the instrumentation tool and the keyword identification method, and the code files and xml files are parsed by the lexical analysis technique to obtain all the database table information recorded in the code of the application program.

[0128] In still another aspect, when the third data set is acquired, the SQL statement information executed by the application program when accessing the database is first obtained by the instrumentation tool, and the SQL statement information is parsed by the lexical analysis method to extract the database table information.

[0129] In summary, the application discloses a detection method for application program database table coverage, which instrumentates the application program to be tested by an instrumentation tool, and automatically obtains database table information from different channels in combination with program parsing techniques, and calculates a plurality of indexes representing the effectiveness of the database table of the application program in different states, namely the first coverage, the second coverage, the third coverage and the fourth coverage. For any index, the effective degree of the application program to the database settings or application can be represented in a certain aspect, thereby providing an indication for the user on whether the application program needs to be sorted at present, and the user can also purposefully arrange the corresponding database according to the obtained indexes. According to the above detection method, the effective degree of the database of the application program can be automatically and quickly detected, to help the user to sort the application program in real time, effectively improve the maintenance efficiency of the application program, and save the labor cost.

[0130] In another preferred embodiment of the application, a detection system for application program database table coverage is also disclosed, which works based on the above detection method.

[0131] The application also discloses another detection system, which includes one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs include instructions for executing the detection method as described above. The processor can adopt a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, for executing the related programs to realize the functions required by the modules in the detection system of the embodiments of the application, or to execute the detection method of the method embodiments of the application.

[0132] The application further discloses a computer readable storage medium comprising a computer program, the computer program being executable by a processor to complete the detection method as described above. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, a data center, etc. integrated with one or more available media. The available medium can be a read-only memory (ROM), or a random access memory (RAM), or a magnetic medium such as a floppy disk, a hard disk, a magnetic tape, a magnetic disc, or an optical medium such as a digital versatile disc (DVD), or a semiconductor medium such as a solid state disk (SSD), etc.

[0133] The application further discloses a computer program product or a computer program, the computer program product or the computer program comprising computer instructions stored in a computer readable storage medium. A processor of an electronic device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the electronic device to perform the detection method.

[0134] The above only discloses preferred embodiments of the application, and of course cannot limit the scope of the application. Any equivalent changes made within the scope of the patent application of the application are still within the scope of the application.

Claims

1. A method for detecting the effectiveness of an application database table, characterized by, Comprising: instrumenting a to-be-tested application program with an instrumentation tool to weave tracking detection logic into the application program; acquiring any two or all of a first data set, a second data set and a third data set; the first data set is acquired by acquiring a database address when the application program connects to a database, and extracting all database table information based on the database address to obtain the first data set; the second data set is acquired by parsing data files related to database operations in the application program to obtain all database table information recorded in the code of the application program to obtain the second data set; the third data set is acquired by acquiring all database table information triggered by the application program in the functional test to obtain the third data set; calculating the degree to which the second data set covers the first data set to obtain a first coverage degree; and / or calculating the degree to which the third data set covers the second data set to obtain a second coverage degree; and / or calculating the degree to which the third data set covers the first data set to obtain a third coverage degree.

2. The method of claim 1, wherein the application database table effectiveness is detected by: When the first coverage degree and the second coverage degree are obtained, the product of the first coverage degree and the second coverage degree is also calculated to obtain a fourth coverage degree.

3. The method of claim 1, wherein the application database table effectiveness is detected by: The first coverage degree and the third coverage degree can be calculated from one or more of the dimensions of the overall database, the dimensions of all tables in a certain database, and the dimensions of the tables of all databases; the second coverage degree can be calculated from one or more of the dimensions of the tables in a certain application database and the dimensions of the tables in all application databases.

4. The application program database table effectiveness detection method according to claim 1, wherein parsing code files and xml files containing SQL fragments in the application program to obtain all database table information recorded in the code of the application program.

5. The method of claim 1, wherein the application database table effectiveness is detected by: parsing SQL statement information executed when the application program accesses the database to extract database table information corresponding to the third data set.

6. The method of claim 1, wherein the application database table effectiveness is detected by: The first coverage is calculated by finding the intersection of the second data set and the first data set to obtain a first arithmetic number representing the number of common objects in the second data set and the first data set, and calculating the ratio of the first arithmetic number to a second arithmetic number representing the number of corresponding objects in the first data set to obtain the first coverage; the second coverage is calculated by finding the intersection of the third data set and the second data set to obtain a third arithmetic number representing the number of common objects in the third data set and the second data set, and calculating the ratio of the third arithmetic number to a fourth arithmetic number representing the number of corresponding objects in the second data set to obtain the second coverage; and the third coverage is calculated by finding the intersection of the third data set and the first data set to obtain a fifth arithmetic number representing the number of functional objects in the third data set and the first data set, and calculating the ratio of the fifth arithmetic number to a sixth arithmetic number representing the number of corresponding objects in the first data set to obtain the third coverage.

7. A system for detecting application database table overwrites, the system comprising: The detection system works based on the detection method of any one of claims 1 to 6.

8. A system for detecting application database table overwrites, the system comprising: Comprising: one or more processors; memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs comprising instructions for performing the detection method of application database table coverage as claimed in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, A computer program is included, which can be executed by a processor to complete the detection method of application database table coverage as claimed in any one of claims 1 to 6. A computer program is included, which can be executed by a processor to complete the detection method of application database table coverage as claimed in any one of claims 1 to 6.

Images