Anomaly detection model training method, anomaly detection method and device

Abstract

The present disclosure provides a training method for an anomaly detection model, which is applied to the field of artificial intelligence, the field of computer technology or other fields. The training method for the anomaly detection model includes obtaining an anomaly scan report; obtaining an anomaly data set according to the anomaly scan report, wherein the anomaly data set includes a plurality of anomaly data, and the anomaly data is used to characterize the parameter information of the anomaly in the anomaly scan report; using the anomaly data set as a training sample data set to train an initial anomaly detection model to be trained, and obtaining a trained anomaly detection model. The present disclosure also provides an anomaly detection method, an anomaly detection model training device, an anomaly detection device, an electronic device, a computer-readable storage medium and a computer program product.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence and computer technology, and more specifically, to an anomaly detection model training method, an anomaly detection method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Black box scanning refers to scanning without knowing the internal structure and internal features of the application. It is generally applicable to scanning web applications (i.e. web applications) and mobile applications. When performing black box scanning on web applications and mobile applications, the built-in attack vectors of the black box scanning tool are used to attack the full URL (Uniform Resource Locator) of the application, and each URL is analyzed according to the preset response classification of the scanning tool to see if there are security risks, and a scanning report is generated.

[0003] In the related art, anomalies reported in a scanning report are usually screened by manual screening. In the process of realizing the concept of the present disclosure, the inventors found that the related art has the problem of high cost and low accuracy of manual screening. Summary of the invention

[0004] In view of this, the present disclosure provides an anomaly detection model training method, an anomaly detection method and an anomaly detection device.

[0005] One aspect of the present disclosure provides a method for training an anomaly detection model, comprising:

[0006] Get anomaly scan reports;

[0007] Acquire an abnormal data set according to the abnormal scanning report, wherein the abnormal data set includes a plurality of abnormal data, and the abnormal data are used to characterize the abnormal parameter information in the abnormal scanning report;

[0008] The above anomaly data set is used as a training sample data set to train the initial anomaly detection model to be trained, thereby obtaining a trained anomaly detection model.

[0009] According to an embodiment of the present disclosure, the above-mentioned obtaining an abnormal data set according to the above-mentioned abnormal scanning report includes:

[0010] Acquire multiple original abnormal data according to the abnormal scanning report;

[0011] Preprocessing the plurality of original abnormal data respectively to generate the plurality of abnormal data, wherein the data types of the abnormal data include numerical type and / or Boolean type;

[0012] The abnormal data set is generated according to the plurality of abnormal data.

[0013] According to an embodiment of the present disclosure, the above-mentioned multiple original abnormal data include: request data and response data;

[0014] The request data includes original request data and modified request data; the response data includes response data corresponding to the original request data and response data corresponding to the modified request data.

[0015] According to an embodiment of the present disclosure, the preprocessing of the plurality of original abnormal data to generate the plurality of abnormal data includes:

[0016] Perform feature extraction on the original request data, the response data corresponding to the original request data, the modified request data, and the response data corresponding to the modified request data, respectively, to determine first message length data, second message length data, third message length data, and fourth message length data;

[0017] Determine the first message similarity based on the original request data and the modified request data;

[0018] Determining a second message similarity based on the response data corresponding to the original request data and the response data corresponding to the modified request data;

[0019] Determine the parameter type of the first message header of the original request data according to the message body data of the original request data;

[0020] Determine the parameter type of the second message header of the modified request data according to the message body data of the modified request data;

[0021] Determining first recurrence data according to the original request data and the response data corresponding to the original request data, wherein the first recurrence data is used to indicate whether a keyword appearing in the original request data appears in the response data corresponding to the original request data;

[0022] Determining second recurrence data according to the modified request data and the response data corresponding to the modified request data, wherein the second recurrence data is used to indicate whether the keyword appearing in the modified request data appears in the response data corresponding to the modified request data;

[0023] Determine the first return code data according to the original response data;

[0024] Determine the second return code data according to the response data corresponding to the modified request data;

[0025] The first message length data, the second message length data, the third message length data, the fourth message length data, the first message similarity, the second message similarity, the parameter type of the first message header, the parameter type of the second message header, the first recurrence data, the second recurrence data, the first return code data and the second return code data are taken as the multiple abnormal data.

[0026] According to an embodiment of the present disclosure, the anomaly detection model to be trained is trained using the training sample data set to obtain the anomaly detection model, including:

[0027] Divide the above training sample data set into a training set and a test set;

[0028] Using the training set to train the anomaly detection model to be trained to obtain an anomaly detection model to be tested;

[0029] The above-mentioned anomaly detection model to be tested is tested using the above-mentioned test set to obtain a verification result. If the above-mentioned verification result does not meet the iteration stopping condition, the above-mentioned initial anomaly detection model is continuously trained until the above-mentioned verification result meets the iteration stopping condition, wherein the above-mentioned verification result is used to characterize whether the abnormal data in the above-mentioned test set is true or false, the above-mentioned true characterizes that the abnormal data in the above-mentioned scanning report is true abnormal data, and the above-mentioned false characterizes that the abnormal data in the above-mentioned scanning report is false alarm abnormal data;

[0030] The model corresponding to the above iteration stopping conditions is used as the anomaly detection model.

[0031] According to an embodiment of the present disclosure, the above-mentioned initial anomaly detection model is constructed based on a naive Bayes classification algorithm.

[0032] Another aspect of the present disclosure provides an anomaly detection method, including: obtaining a scan report to be tested;

[0033] Obtain a sample data set to be tested according to the above-mentioned scan report to be tested;

[0034] The above-mentioned sample data set to be tested is input into the above-mentioned anomaly detection model, and the detection result is output, wherein the above-mentioned anomaly detection model is trained by the training method of the anomaly detection model provided by the embodiment of the present disclosure.

[0035] According to an embodiment of the present disclosure, the above-mentioned obtaining a sample data set to be tested according to the scan report to be tested includes:

[0036] Acquire an original sample to be tested according to the above-mentioned scan report to be tested, wherein the original sample to be tested includes a plurality of original data;

[0037] Preprocessing the above-mentioned multiple original data respectively to obtain multiple test data, wherein the test samples in the above-mentioned test sample data set are composed of the above-mentioned multiple test data, and the data type of the above-mentioned test data includes numerical type and / or Boolean type;

[0038] The above multiple test data are used as test sample data sets.

[0039] According to an embodiment of the present disclosure, it also includes:

[0040] According to the above test results, the above anomaly detection model is optimized using the above test sample data set to obtain an optimized anomaly detection model.

[0041] According to an embodiment of the present disclosure, the above-mentioned abnormality detection model is optimized using the above-mentioned sample data set to be tested according to the above-mentioned detection results, and the optimized detection model is obtained, including:

[0042] Determine whether the sample to be tested in the sample data set to be tested overlaps with the training sample of the anomaly detection model;

[0043] When the test samples in the above test sample data set do not overlap with the training samples of the above anomaly detection model:

[0044] Divide the above-mentioned sample data set to be tested into an optimized training set and an optimized test set;

[0045] Optimize the above anomaly detection model using the above optimized training set to obtain an anomaly detection model to be optimized for testing;

[0046] The above-mentioned anomaly detection model to be optimized is tested using the above-mentioned optimized test set to obtain a verification result. If the above-mentioned verification result does not meet the iteration stop condition, the above-mentioned anomaly detection model to be optimized is continuously optimized until the above-mentioned verification result meets the iteration stop condition, wherein the above-mentioned verification result is used to characterize whether the abnormal data in the above-mentioned optimized test set is true or false, the above-mentioned true characterizes that the abnormal data in the above-mentioned scan report to be tested is true abnormal data, and the above-mentioned false characterizes that the abnormal data in the above-mentioned scan report to be tested is false alarm abnormal data; and

[0047] The model that meets the above iteration stop conditions is used as the optimized anomaly detection model;

[0048] When the test samples in the above test sample data set overlap with the training samples of the above anomaly detection model:

[0049] The anomaly detection model is optimized using the training sample data set to be tested to obtain an optimized anomaly detection model.

[0050] According to an embodiment of the present disclosure, it also includes:

[0051] A test report is generated based on the test results, and the test report is used to characterize the abnormality of each sample to be tested in the sample data set to be tested.

[0052] According to an embodiment of the present disclosure, the scan report to be tested and the scan report used to train the anomaly detection model are scan reports from different versions of the same application.

[0053] Another aspect of the present disclosure provides a training device for an anomaly detection model, comprising:

[0054] A first acquisition module is used to acquire an abnormal data set, wherein the abnormal data set includes a plurality of abnormal data, and the abnormal data is used to characterize abnormal parameter information;

[0055] A second acquisition module is used to acquire a training sample data set, wherein the training sample data set includes a plurality of the above-mentioned abnormal data sets;

[0056] The training module is used to train the anomaly detection model to be trained using the above training sample data set to obtain the anomaly detection model.

[0057] Another aspect of the present disclosure provides an abnormality detection device, comprising:

[0058] The third acquisition module is used to obtain the scan report to be tested;

[0059] A fourth acquisition module is used to acquire a sample data set to be tested according to the above-mentioned scan report to be tested;

[0060] The detection module is used to input the above-mentioned sample data set to be tested into the anomaly detection model and output the detection result, wherein the above-mentioned anomaly detection model is trained by the training method of the anomaly detection model provided by the embodiment of the present disclosure.

[0061] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more instructions, wherein when the one or more instructions are executed by the one or more processors, the one or more processors implement the above-mentioned method.

[0062] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0063] Another aspect of the present disclosure provides a computer program product, the computer program product comprising computer executable instructions, and the instructions are used to implement the above method when executed.

[0064] According to the embodiments of the present disclosure, an anomaly detection model is trained by using an anomaly data set as a training sample, wherein the anomaly data set can characterize the parameter information of the anomaly provided in the scan report. When the anomaly detection model is used to detect anomalies in the scan report to be tested, it is possible to accurately screen out whether the anomaly indicated in the anomaly detection report is a true anomaly or a false alarm anomaly, thereby achieving the technical effect of improving the accuracy of anomaly detection and reducing the cost of manual screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0066] Figure 1 Schematically illustrates an exemplary system architecture to which an anomaly detection model training method, an anomaly detection model training device, an anomaly detection method, and an anomaly detection device according to an embodiment of the present disclosure can be applied;

[0067] Figure 2 A flowchart of a method for training an anomaly detection model according to an embodiment of the present disclosure is schematically shown;

[0068] Figure 3 A flowchart of obtaining an abnormal data set according to an abnormal scanning report according to an embodiment of the present disclosure is schematically shown;

[0069] Figure 4 A flowchart of preprocessing a plurality of original abnormal data to generate a plurality of abnormal data according to an embodiment of the present disclosure is schematically shown;

[0070] Figure 5 A flowchart of obtaining an anomaly detection model by using a training sample data set to train an anomaly detection model to be trained is schematically shown according to an embodiment of the present disclosure;

[0071] Figure 6 A flowchart of an abnormality detection method according to an embodiment of the present disclosure is schematically shown;

[0072] Figure 7 A flowchart of obtaining a sample data set to be tested according to a scan report to be tested according to an embodiment of the present disclosure is schematically shown;

[0073] Figure 8 A flowchart of optimizing an anomaly detection model using a sample data set to be tested based on the detection results to obtain an optimized detection model is schematically shown according to an embodiment of the present disclosure;

[0074] Fig. 9 A block diagram of a training device for an anomaly detection model according to an embodiment of the present disclosure is schematically shown;

[0075] Fig.10 A block diagram schematically shows an abnormality detection device according to an embodiment of the present disclosure; and

[0076] Fig.11 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0077] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0078] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0079] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0080] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0081] The present disclosure provides a training method for an anomaly detection model, which is applied to the field of artificial intelligence, the field of computer technology or other fields. The training method for the anomaly detection model includes obtaining an anomaly scan report; obtaining an anomaly data set according to the anomaly scan report, wherein the anomaly data set includes a plurality of anomaly data, and the anomaly data is used to characterize the parameter information of the anomaly in the anomaly scan report; using the anomaly data set as a training sample data set to train an initial anomaly detection model to be trained, and obtaining a trained anomaly detection model. The present disclosure also provides an anomaly detection method, an anomaly detection model training device, an anomaly detection device, an electronic device, a computer-readable storage medium and a computer program product.

[0082] Figure 1 The exemplary system architecture 100 to which the training method of the anomaly detection model, the training device of the anomaly detection model, the anomaly detection method and the anomaly detection device according to the embodiment of the present disclosure can be applied is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0083] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0084] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software, etc. (only as examples).

[0085] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0086] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0087] It should be noted that the training method of the anomaly detection model and the anomaly detection method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the training device of the anomaly detection model and the anomaly detection device provided in the embodiment of the present disclosure can generally be set in the server 105. The training method of the anomaly detection model and the anomaly detection method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal device 101, 102, 103 and / or the server 105. Correspondingly, the training device of the anomaly detection model and the anomaly detection device provided in the embodiment of the present disclosure can be set in a server or server cluster that is different from the server 105 and can communicate with the terminal device 101, 102, 103 and / or the server 105. Alternatively, the training method of the anomaly detection model and the anomaly detection method provided in the embodiment of the present disclosure can also be executed by the terminal device 101, 102, or 103, or can also be executed by other terminal devices different from the terminal device 101, 102, or 103. Correspondingly, the training device of the anomaly detection model and the anomaly detection device provided in the embodiments of the present disclosure may also be arranged in the terminal device 101, 102, or 103, or in other terminal devices different from the terminal device 101, 102, or 103.

[0088] For example, the abnormal data set may be originally stored in any one of the terminal devices 101, 102, or 103 (for example, the terminal device 101, but not limited thereto), or stored on an external storage device and may be imported into the terminal device 101. Then, the terminal device 101 may locally execute the training method of the abnormal detection model provided in the embodiment of the present disclosure, or send the abnormal data set to other terminal devices, servers, or server clusters, and the other terminal devices, servers, or server clusters that receive the abnormal data set may execute the training method of the abnormal detection model provided in the embodiment of the present disclosure.

[0089] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0090] Figure 2The flowchart of the training method of the anomaly detection model according to the embodiment of the present disclosure is schematically shown.

[0091] It should be noted that, unless it is explicitly stated that there is a sequence of execution between different operations shown in the flowchart in the embodiments of the present disclosure, or there is a sequence of execution between different operations in technical implementation, otherwise, the execution order of multiple operations may not be prioritized, and multiple operations may also be executed simultaneously.

[0092] like Figure 2 As shown, the method includes operations S201 to S203.

[0093] In operation S201, an abnormal scanning report is obtained.

[0094] According to an embodiment of the present disclosure, the abnormal scanning report may include an abnormal scanning report generated after a black box scanning is performed on an application program using a black box scanning tool.

[0095] According to an embodiment of the present disclosure, the exception scanning report generated after black box scanning of an application using a black box scanning tool will provide request data and response data corresponding to the discovered exceptions, and mark the request differences, and problematic paragraphs in the request data and response data.

[0096] According to an embodiment of the present disclosure, the exception scanning report may provide, for example, one exception existing in the application, but is not limited thereto. The exception scanning report may also provide, for example, multiple exceptions existing in the application.

[0097] In operation S202, an abnormal data set is acquired according to the abnormal scanning report, wherein the abnormal data set includes a plurality of abnormal data, and the abnormal data are used to characterize the abnormal parameter information in the abnormal scanning report.

[0098] According to an embodiment of the present disclosure, there may be a one-to-one correspondence between anomaly data sets and anomalies provided in anomaly scan reports. For example, if anomaly scan reports provide N anomalies, N anomaly data sets may be obtained based on the anomaly scan reports, where N is a positive integer.

[0099] According to an embodiment of the present disclosure, after obtaining the abnormal scanning report, abnormal data of the abnormalities provided in the abnormal scanning report can be extracted through a text analysis method to form an abnormal data set.

[0100] According to an embodiment of the present disclosure, all abnormal data in the abnormal data set are used together to characterize an abnormality corresponding to the abnormal data set; each abnormal data in the abnormal data set can characterize a parameter information of an abnormality corresponding to the abnormal data set.

[0101] In operation S203, the abnormal data set is used as a training sample data set to train the initial abnormality detection model to be trained, so as to obtain a trained abnormality detection model.

[0102] According to the embodiments of the present disclosure, the black box scanning tool generates different anomaly scanning reports for different applications and different anomalies, so that different anomaly detection models can be trained by adjusting the model parameters of the initial anomaly detection model, thereby improving the degree of freedom of anomaly detection.

[0103] According to an embodiment of the present disclosure, an abnormal data set corresponding to an abnormality in a scan report may be used as a training sample to train an initial abnormality detection model.

[0104] According to an embodiment of the present disclosure, a plurality of abnormal data sets corresponding to a plurality of abnormalities in an abnormal scanning report may be used as a training sample data set to train an initial abnormality detection model.

[0105] According to an embodiment of the present disclosure, the anomaly detection model can detect whether an anomaly in a scan report corresponding to an abnormal data set is a true anomaly or a false positive anomaly.

[0106] According to an embodiment of the present disclosure, the initial anomaly detection model may be, for example, an anomaly detection model constructed based on a naive Bayes classification algorithm. It should be noted that the initial anomaly detection model is not limited thereto, and may also be an anomaly detection model constructed using other algorithms, for example, a random forest algorithm, etc.

[0107] The disclosed embodiment provides a method for training an anomaly detection model, wherein an anomaly data set is used as a training sample to train an anomaly detection model, wherein the anomaly data set can characterize the parameter information of the anomaly provided in the scan report. When the anomaly detection model is used to detect anomalies in the scan report to be tested, it is possible to accurately screen whether the anomaly indicated in the anomaly detection report is a true anomaly or a false alarm anomaly, thereby achieving the technical effect of improving the accuracy of anomaly detection and reducing the cost of manual screening.

[0108] Reference below Figure 3 to Figure 5 , combined with specific embodiments Figure 2 The method shown is further explained.

[0109] Figure 3 The flowchart of obtaining an abnormal data set according to an abnormal scanning report according to an embodiment of the present disclosure is schematically shown.

[0110] like Figure 3 As shown, the method includes operations S301 to S303.

[0111] In operation S301 , a plurality of original abnormal data are acquired according to an abnormal scanning report.

[0112] According to an embodiment of the present disclosure, the original abnormality data obtained according to the abnormality scanning report includes string type data. Since the string type data cannot be directly used to train the initial abnormality detection model, the multiple original abnormality data can be preprocessed separately to convert the string type data into data that can directly train the initial abnormality detection model.

[0113] In operation S302 , a plurality of original abnormal data are preprocessed respectively to generate a plurality of abnormal data, wherein the data types of the abnormal data include numerical type and / or Boolean type.

[0114] According to an embodiment of the present disclosure, by preprocessing the original abnormal data of string type, numerical and / or Boolean abnormal data are obtained, so that the initial abnormal scanning model can be trained directly using the abnormal data.

[0115] In operation S303 , an abnormal data set is generated according to the plurality of abnormal data.

[0116] According to an embodiment of the present disclosure, multiple original exception data include request data and response data; wherein the request data includes original request data and modified request data; the response data includes response data corresponding to the original request data and response data corresponding to the modified request data.

[0117] According to an embodiment of the present disclosure, by using the original request data, the modified request data, the response data corresponding to the original request data, and the response data corresponding to the modified request data as the original exception data, the anomalies provided in the exception scanning report can be fully characterized, thereby improving the training quality of the anomaly detection model.

[0118] Figure 4 The flowchart of preprocessing a plurality of original abnormal data respectively to generate a plurality of abnormal data according to an embodiment of the present disclosure is schematically shown.

[0119] like Figure 4 As shown, the method includes operations S401 to S4010.

[0120] In operation S401, feature extraction is performed on the original request data, the response data corresponding to the original request data, the modified request data, and the response data corresponding to the modified request data to determine first message length data, second message length data, third message length data, and fourth message length data.

[0121] According to an embodiment of the present disclosure, feature extraction can be performed on the original request data to obtain first message length data; feature extraction can be performed on the response data corresponding to the original request data to obtain second message length data; feature extraction can be performed on the modified request data to obtain third message length data; feature extraction can be performed on the response data corresponding to the modified request data to obtain fourth message length data.

[0122] According to an embodiment of the present disclosure, the first message length data may include the total number of characters in the message data of the original request data. The total number of characters may be obtained by, for example, counting the total number of spaces and special characters in the message data.

[0123] According to an embodiment of the present disclosure, the second message length data may include the total number of characters of the message data of the response data corresponding to the original request data. The total number of characters may be obtained by, for example, counting the total number of spaces and special characters in the message data.

[0124] According to an embodiment of the present disclosure, the third message length data may include the total number of characters of the message data of the modified request data. The total number of characters may be obtained, for example, by counting the total number of spaces and special characters in the message data.

[0125] According to an embodiment of the present disclosure, the fourth message length data may include the total number of characters of the message data of the response data corresponding to the modified request data. The total number of characters may be obtained by, for example, counting the total number of spaces and special characters in the message data.

[0126] In operation S402, a first message similarity is determined based on the original request data and the modified request data.

[0127] According to an embodiment of the present disclosure, the first message similarity may be determined based on the original request data and the modified request data using a Levenshtein Distance algorithm.

[0128] In operation S403, a second message similarity is determined based on the response data corresponding to the original request data and the response data corresponding to the modified request data.

[0129] According to an embodiment of the present disclosure, the second message similarity may be determined based on the response data corresponding to the original request data and the response data corresponding to the modified request data using a Levenshtein Distance algorithm.

[0130] In operation S404, a parameter type of a first message header of the original request data is determined according to the message body data of the original request data.

[0131] According to an embodiment of the present disclosure, the message data of the original request data includes message header data and message body data, and the parameter type of the message header data has a fixed range, and the message body data and the message header data have a corresponding relationship.

[0132] According to an embodiment of the present disclosure, the message body data of the original request data may be determined first, and then a traversal search may be performed in a data dictionary based on the message body data of the original request data to determine the parameter type of the first message header of the original request data.

[0133] In operation S405, a parameter type of a second message header of the modified request data is determined according to the message body data of the modified request data.

[0134] According to an embodiment of the present disclosure, the message data of the modified request data includes a message header and a message body, and the parameter type of the message header has a fixed range, and the message body and the message header have a corresponding relationship.

[0135] According to an embodiment of the present disclosure, the message body data of the modified request data may be determined first, and then a traversal search may be performed in a data dictionary based on the message body data of the modified request data to determine the parameter type of the second message header of the modified request data.

[0136] In operation S406, first recurrence data is determined according to the original request data and the response data corresponding to the original request data, wherein the first recurrence data is used to indicate whether the keyword appearing in the original request data appears in the response data corresponding to the original request data.

[0137] According to an embodiment of the present disclosure, for example, the original request data includes the keyword "(target="_blank")", and the keyword "(target="_blank")" can be searched in the response data corresponding to the original request, so as to determine whether the keyword "(target="_blank")" appears in the response data corresponding to the original request;

[0138] According to an embodiment of the present disclosure, the first recurrence data may, for example, include 1 or 0, where 1 may indicate that a keyword appearing in the original request data appears in the response data corresponding to the original request data, and 0 may indicate that a keyword appearing in the original request data does not appear in the response data corresponding to the original request data; but not limited to this, the first recurrence data may also, for example, include true or false, where true may indicate that a keyword appearing in the original request data appears in the response data corresponding to the original request data, and false may indicate that a keyword appearing in the original request data does not appear in the response data corresponding to the original request data.

[0139] In operation S407, second recurrence data is determined according to the modified request data and the response data corresponding to the modified request data, wherein the second recurrence data is used to indicate whether the keyword appearing in the modified request data appears in the response data corresponding to the modified request data.

[0140] According to an embodiment of the present disclosure, for example, the modified request data includes the keyword "(target="_blank")", and the keyword "(target="_blank")" can be searched in the response data corresponding to the modified request data, so as to determine whether the keyword "(target="_blank")" appears in the response data corresponding to the modified request data;

[0141] According to an embodiment of the present disclosure, the second recurrence data may, for example, include 1 or 0, where 1 may indicate that a keyword appearing in the modified request data appears in the response data corresponding to the modified request data, and 0 may indicate that a keyword appearing in the modified request data does not appear in the response data corresponding to the modified request data; but not limited to this, the second recurrence data may also, for example, include true or false, where true may indicate that a keyword appearing in the modified request data appears in the response data corresponding to the modified request data, and false may indicate that a keyword appearing in the modified request data does not appear in the response data corresponding to the modified request data.

[0142] According to an embodiment of the present disclosure, the first reproduction data and the second reproduction data may be, for example, Boolean data.

[0143] In operation S408, first return code data is determined according to the original response data.

[0144] According to an embodiment of the present disclosure, in the original response data, the first return code data may start with HTTP / 1.X, wherein X may include any character, and the first return code data may include three characters, so that the first return code data may be extracted from the original response data according to the format of the first return code.

[0145] In operation S409, second return code data is determined according to the response data corresponding to the modified request data.

[0146] According to an embodiment of the present disclosure, in the response data corresponding to the modified request data, the second return code data may start with HTTP / 1.X, wherein X may include any character, and the second return code data may include three characters, so that the second return code data may be extracted from the response data corresponding to the modified request data according to the format of the second return code.

[0147] In operation S4010, the first message length data, the second message length data, the third message length data, the fourth message length data, the first message similarity, the second message similarity, the parameter type of the first message header, the parameter type of the second message header, the first recurrence data, the second recurrence data, the first return code data, and the second return code data are taken as multiple abnormal data.

[0148] According to an embodiment of the present disclosure, Table 1 schematically shows abnormal data and a corresponding preprocessing method according to an embodiment of the present disclosure.

[0149] Table 1

[0150]

[0151] Figure 5 The flowchart of obtaining the anomaly detection model by using the training sample data set to train the anomaly detection model to be trained according to an embodiment of the present disclosure is schematically shown.

[0152] like Figure 5 As shown, the method includes operations S501 to S504.

[0153] In operation S501, a training sample data set is divided into a training set and a test set.

[0154] According to an embodiment of the present disclosure, a part of the training samples in the training sample data set can be used as a training set, and the training samples in the training sample data set other than the training set can be used as a test set; for example, 80% of the training samples in the training sample data set can be used as a training set, and 20% of the training samples in the training sample data set can be used as a test set, but not limited to this, and 70% of the training samples in the training sample data set can also be used as a training set, and 30% of the training samples in the training sample data set can be used as a test set.

[0155] In operation S502, the anomaly detection model to be trained is trained using the training set to obtain an anomaly detection model to be tested.

[0156] In operation S503, the test set is used to test the anomaly detection model to be tested to obtain a verification result. If the verification result does not meet the iteration stop condition, the initial anomaly detection model continues to be trained until the verification result meets the iteration stop condition, wherein the verification result is used to characterize whether the anomaly data in the test set is true or false, where true represents that the anomaly data in the scan report is true anomaly data, and false represents that the anomaly data in the scan report is false positive anomaly data.

[0157] According to the embodiments of the present disclosure, the iteration stop condition can be flexibly set by those skilled in the art according to actual application conditions, and the embodiments of the present disclosure do not specifically limit the iteration stop condition.

[0158] In operation S504, the model corresponding to the condition for stopping the iteration is used as an anomaly detection model.

[0159] According to an embodiment of the present disclosure, the initial anomaly detection model is constructed based on a naive Bayes classification algorithm.

[0160] Figure 6 The flowchart of the abnormality detection method according to the embodiment of the present disclosure is schematically shown.

[0161] like Figure 6 As shown, the method includes operations S601 to S603.

[0162] In operation S601, a scan report to be tested is obtained.

[0163] According to an embodiment of the present disclosure, the scan report to be tested may include a scan report to be tested generated after a black box scan is performed on an application program using a black box scanning tool.

[0164] In operation S602, a sample data set to be tested is obtained according to the scan report to be tested.

[0165] In operation S603, the sample data set to be tested is input into the anomaly detection model, and the detection result is output, wherein the anomaly detection model is trained by the anomaly detection model training method provided by the embodiment of the present disclosure.

[0166] According to the embodiments of the present disclosure, the anomalies in the scan report to be tested are scanned by using the pre-trained anomaly detection model, which can accurately screen out whether the anomaly prompted in the scan report to be tested is a true anomaly or a false alarm anomaly, thereby achieving the technical effect of improving the accuracy of anomaly detection and reducing the cost of manual screening.

[0167] Figure 7 The flowchart of obtaining a sample data set to be tested according to a scan report to be tested according to an embodiment of the present disclosure is schematically shown.

[0168] like Figure 7 As shown, the method includes operations S701 to S703.

[0169] In operation S701, an original sample to be tested is obtained according to a scan report to be tested, where the original sample to be tested includes a plurality of original data.

[0170] In operation S702, a plurality of original data are preprocessed respectively to obtain a plurality of test data, wherein the test samples in the test sample data set are composed of a plurality of test data, and the data types of the test data include numerical type and / or Boolean type.

[0171] In operation S703, a plurality of to-be-tested data are used as to-be-tested sample data sets.

[0172] According to an embodiment of the present disclosure, the anomaly detection method further includes optimizing the anomaly detection model using the sample data set to be tested according to the detection result to obtain an optimized anomaly detection model.

[0173] According to the embodiments of the present disclosure, the anomaly detection model may be optimized and trained according to the accuracy requirement of anomaly detection to further improve the anomaly detection accuracy of the anomaly detection model.

[0174] Figure 8 The flowchart of optimizing the anomaly detection model by using the sample data set to be tested according to the detection results to obtain the optimized detection model according to an embodiment of the present disclosure is schematically shown.

[0175] like Figure 8 As shown, the method includes operations S801 to S806.

[0176] In operation S801, it is determined whether the sample to be tested in the sample data set to be tested overlaps with the training sample of the anomaly detection model.

[0177] According to an embodiment of the present disclosure, before optimizing the training of the anomaly detection model, it may be determined whether the sample to be tested in the sample data set to be tested is used for the training of the anomaly detection model for the first time.

[0178] If the sample to be tested in the sample data set does not overlap with the training sample of the anomaly detection model, operation S802 is performed. If the sample to be tested in the sample data set overlaps with the training sample of the anomaly detection model, operation S806 is performed.

[0179] In operation S802, the sample data set to be tested is divided into an optimized training set and an optimized test set.

[0180] According to an embodiment of the present disclosure, a part of the samples to be tested in the sample data set to be tested can be used as an optimized training set, and another part of the samples to be tested in the sample data set except the optimized training set can be used as an optimized test set; for example, 80% of the samples to be tested in the sample data set to be tested can be used as an optimized training set, and 20% of the samples to be tested in the sample data set to be tested can be used as an optimized test set, but not limited to this, 70% of the samples to be tested in the sample data set to be tested can also be used as an optimized training set, and 30% of the samples to be tested in the sample data set to be tested can be used as an optimized test set.

[0181] In operation S803, the anomaly detection model is optimized using the optimized training set to obtain an anomaly detection model to be tested for optimization.

[0182] In operation S804, the optimized test set is used to test the abnormal detection model to be optimized and obtain a verification result. If the verification result does not meet the iteration stop condition, the abnormal detection model to be optimized continues to be optimized until the verification result meets the iteration stop condition. The verification result is used to characterize whether the abnormal data in the optimized test set is true or false. True indicates that the abnormal data in the scan report to be tested is true abnormal data, and false indicates that the abnormal data in the scan report to be tested is false alarm abnormal data.

[0183] In operation S805, the model that meets the iteration stop condition is used as the optimized anomaly detection model.

[0184] According to the embodiments of the present disclosure, the iteration stop condition can be flexibly set by those skilled in the art according to actual application conditions, and the embodiments of the present disclosure do not specifically limit the iteration stop condition.

[0185] In operation S806, the anomaly detection model is optimized using the training sample data set to be tested to obtain an optimized anomaly detection model.

[0186] According to an embodiment of the present disclosure, if the test samples in the test sample data set are used for the first time to train the anomaly detection model, the test sample data set can be divided into an optimized training set and an optimized test set to determine the optimization effect of the anomaly detection model, thereby determining whether to continue to optimize the anomaly detection model; if the test samples in the test sample data set are not used for the first time to train the anomaly detection model, the test training sample data set can be used to optimize the anomaly detection model to obtain an optimized anomaly detection model to speed up the optimization training.

[0187] According to an embodiment of the present invention, the anomaly detection method further includes generating a detection report according to the detection result, where the detection report is used to characterize the anomaly of each sample to be tested in the sample data set to be tested.

[0188] According to an embodiment of the present disclosure, the detection report may include parameter information of the anomalies provided in the scan report to be tested, for example, which anomalies are true anomalies and which anomalies are false positive anomalies among multiple anomalies in the scan report to be tested, and a paragraph where the true anomaly has problems in the full amount of URLs may also be provided to analyze the true anomaly.

[0189] According to an embodiment of the present invention, the scan report to be tested and the scan report used to train the anomaly detection model are scan reports from different versions of the same application.

[0190] According to an embodiment of the present disclosure, the scan report to be tested may, for example, include a scan report of the application A2.0 version, and the scan report used to train the anomaly detection model may, for example, include a scan report of the application A1.0 version.

[0191] According to the embodiments of the present disclosure, during the application process of the anomaly detection model, as the application is updated, the anomaly detection model can be continuously optimized to improve the robustness of the anomaly detection model.

[0192] Fig. 9 A block diagram 900 schematically shows a training apparatus for an anomaly detection model according to an embodiment of the present disclosure.

[0193] like Fig. 9 As shown, the training device of the anomaly detection model includes a first acquisition module 901, a second acquisition module 902 and a training module 903.

[0194] The first acquisition module 901 is used to acquire an abnormal data set, wherein the abnormal data set includes a plurality of abnormal data, and the abnormal data is used to characterize abnormal parameter information.

[0195] The second acquisition module 902 is used to acquire a training sample data set, wherein the training sample data set includes a plurality of abnormal data sets.

[0196] The training module 903 is used to train the anomaly detection model to be trained using the training sample data set to obtain the anomaly detection model.

[0197] According to an embodiment of the present disclosure, the training device of the anomaly detection model also includes a fifth acquisition module, a preprocessing module and a generation module.

[0198] The fifth acquisition module is used to acquire a plurality of original abnormal data according to the abnormal scanning report.

[0199] The first preprocessing module is used to preprocess the multiple original abnormal data respectively to generate multiple abnormal data, wherein the data type of the abnormal data includes a numerical type and / or a Boolean type.

[0200] The first generating module is used to generate an abnormal data set according to a plurality of abnormal data.

[0201] According to an embodiment of the present disclosure, the multiple original exception data include: request data and response data; wherein the request data includes original request data and modified request data; the response data includes response data corresponding to the original request data and response data corresponding to the modified request data.

[0202] According to an embodiment of the present disclosure, the preprocessing module includes a feature extraction unit, a first determination unit, a second determination unit, a third determination unit, a fourth determination unit, a fifth determination unit, a sixth determination unit, a seventh determination unit, an eighth determination unit and a generation unit.

[0203] A feature extraction unit is used to extract features from the original request data, the response data corresponding to the original request data, the modified request data and the response data corresponding to the modified request data, respectively, to determine the first message length data, the second message length data, the third message length data and the fourth message length data.

[0204] The first determining unit is used to determine a first message similarity based on the original request data and the modified request data.

[0205] The second determining unit is used to determine the second message similarity based on the response data corresponding to the original request data and the response data corresponding to the modified request data.

[0206] The third determining unit is used to determine the parameter type of the first message header of the original request data according to the message body data of the original request data.

[0207] The fourth determining unit is used to determine the parameter type of the second message header of the modified request data according to the message body data of the modified request data.

[0208] The fifth determining unit is used to determine first recurrence data according to the original request data and the response data corresponding to the original request data, wherein the first recurrence data is used to indicate whether a keyword appearing in the original request data appears in the response data corresponding to the original request data.

[0209] The sixth determination unit is used to determine second recurrence data based on the modified request data and the response data corresponding to the modified request data, wherein the second recurrence data is used to characterize whether a keyword appearing in the modified request data appears in the response data corresponding to the modified request data.

[0210] The seventh determining unit is used to determine the first return code data according to the original response data.

[0211] The eighth determining unit is used to determine the second return code data according to the response data corresponding to the modified request data.

[0212] A generating unit is used to use the first message length data, the second message length data, the third message length data, the fourth message length data, the first message similarity, the second message similarity, the parameter type of the first message header, the parameter type of the second message header, the first recurrence data, the second recurrence data, the first return code data and the second return code data as multiple abnormal data.

[0213] According to an embodiment of the present disclosure, the training module 903 includes a first dividing unit, a training unit, a testing unit and a second generating unit.

[0214] The first division unit is used to divide the training sample data set into a training set and a test set.

[0215] The training unit is used to train the anomaly detection model to be trained using the training set to obtain the anomaly detection model to be tested.

[0216] The testing unit is used to test the anomaly detection model to be tested using the test set to obtain a verification result. If the verification result does not meet the iteration stop condition, the initial anomaly detection model continues to be trained until the verification result meets the iteration stop condition. The verification result is used to characterize whether the anomaly data in the test set is true or false. True represents that the anomaly data in the scan report is true anomaly data, and false represents that the anomaly data in the scan report is false alarm anomaly data.

[0217] The second generating unit is used to use the corresponding model when the iteration stopping condition is met as the anomaly detection model.

[0218] According to an embodiment of the present disclosure, the initial anomaly detection model is constructed based on a naive Bayes classification algorithm.

[0219] Fig.10 A block diagram 1000 of an abnormality detection device according to an embodiment of the present disclosure is schematically shown.

[0220] like Fig.10 As shown, the abnormality detection device includes a third acquisition module 1001 , a fourth acquisition module 1002 and a detection module 1003 .

[0221] The third acquisition module 1001 acquires the scan report to be tested.

[0222] The fourth acquisition module 1002 is used to acquire a sample data set to be tested according to the scan report to be tested.

[0223] The detection module 1003 is used to input the sample data set to be tested into the anomaly detection model and output the detection result, wherein the anomaly detection model is trained by the training method of the anomaly detection model provided by the embodiment of the present disclosure.

[0224] According to an embodiment of the present disclosure, the abnormality detection device further includes a sixth acquisition module, a second preprocessing module and a second generation module.

[0225] The sixth acquisition module is used to acquire the original sample to be tested according to the scan report to be tested, where the original sample to be tested includes a plurality of original data.

[0226] The second preprocessing module is used to preprocess the multiple original data respectively to obtain multiple test data, wherein the test samples in the test sample data set are composed of multiple test data, and the data type of the test data includes numerical type and / or Boolean type.

[0227] The second generating module is used to use a plurality of to-be-tested data as a to-be-tested sample data set.

[0228] According to an embodiment of the present disclosure, the anomaly detection device further includes an optimization module.

[0229] The optimization module is used to optimize the anomaly detection model using the sample data set to be tested according to the detection results to obtain an optimized anomaly detection model.

[0230] According to an embodiment of the present disclosure, the optimization module includes a first judgment unit, a second division unit, a first optimization training unit, an optimization testing unit, a third generation unit and a second optimization training unit.

[0231] The first judgment unit is used to judge whether the sample to be tested in the sample data set to be tested overlaps with the training sample of the anomaly detection model.

[0232] When the test samples in the test sample data set do not overlap with the training samples of the anomaly detection model:

[0233] The second division unit is used to divide the sample data set to be tested into an optimized training set and an optimized test set.

[0234] The first optimization training unit is used to optimize the anomaly detection model using the optimized training set to obtain an anomaly detection model to be optimized for testing.

[0235] The optimization test unit is used to test the anomaly detection model to be optimized using the optimization test set to obtain a verification result. If the verification result does not meet the iteration stop condition, the anomaly detection model to be optimized will continue to be optimized until the verification result meets the iteration stop condition. The verification result is used to characterize whether the abnormal data in the optimization test set is true or false. True indicates that the abnormal data in the scan report to be tested is true abnormal data, and false indicates that the abnormal data in the scan report to be tested is false alarm abnormal data. And

[0236] The third generating unit is used to use the model that meets the iteration stopping condition as the optimized anomaly detection model.

[0237] When the test samples in the test sample data set overlap with the training samples of the anomaly detection model:

[0238] The second optimization training unit is used to optimize the anomaly detection model using the training sample data set to be tested to obtain an optimized anomaly detection model.

[0239] According to an embodiment of the present disclosure, the abnormality detection device 1000 further includes a detection report generating unit.

[0240] The test report generating unit is used to generate a test report according to the test result, and the test report is used to characterize the abnormal situation of each sample to be tested in the sample data set to be tested.

[0241] According to an embodiment of the present disclosure, the scan report to be tested and the scan report used to train the anomaly detection model are scan reports from different versions of the same application.

[0242] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0243] For example, any multiple of the first acquisition module 901, the second acquisition module 902, the training module 903, the third acquisition module 1001, the fourth acquisition module 1002, and the detection module 1003 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the first acquisition module 901, the second acquisition module 902, the training module 903, the third acquisition module 1001, the fourth acquisition module 1002 and the detection module 1003 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the first acquisition module 901, the second acquisition module 902, the training module 903, the third acquisition module 1001, the fourth acquisition module 1002 and the detection module 1003 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0244] It should be noted that the anomaly detection model training device part in the embodiment of the present disclosure corresponds to the anomaly detection model training method part in the embodiment of the present disclosure, and the description of the anomaly detection model training device part specifically refers to the anomaly detection model training method part, which will not be repeated here; the anomaly detection device part in the embodiment of the present disclosure corresponds to the anomaly detection method part in the embodiment of the present disclosure, and the description of the anomaly detection device part specifically refers to the anomaly detection method part, which will not be repeated here.

[0245] Fig.11 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Fig.11 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0246] like Fig.11As shown, the electronic device 1100 according to an embodiment of the present disclosure includes a processor 1101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage part 1108 into a random access memory (RAM) 1103. The processor 1101 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), and the like. The processor 1101 may also include an onboard memory for caching purposes. The processor 1101 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0247] In RAM 1103, various programs and data required for the operation of electronic device 1100 are stored. Processor 1101, ROM 1102 and RAM 1103 are connected to each other through bus 1104. Processor 1101 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1102 and / or RAM 1103. It should be noted that the program can also be stored in one or more memories other than ROM 1102 and RAM 1103. Processor 1101 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0248] According to an embodiment of the present disclosure, the electronic device 1100 may further include an input / output (I / O) interface 1105, and the input / output (I / O) interface 305 is also connected to the bus 1104. The electronic device 1100 may also include one or more of the following components connected to the I / O interface 1105: an input part 1106 including a keyboard, a mouse, etc.; an output part 1107 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 1108 including a hard disk, etc.; and a communication part 1109 including a network interface card such as a LAN card, a modem, etc. The communication part 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed, so that a computer program read therefrom is installed into the storage part 1108 as needed.

[0249] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1109, and / or installed from the removable medium 1111. When the computer program is executed by the processor 1101, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0250] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0251] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0252] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 1102 and / or the RAM 1103 described above and / or one or more memories other than the ROM 1102 and the RAM 1103 .

[0253] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the character recognition model training method and character recognition method provided by the embodiment of the present disclosure.

[0254] When the computer program is executed by the processor 1101, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0255] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1109, and / or installed from a removable medium 1111. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0256] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0257] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0258] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A training method for an anomaly detection model, comprising: Obtain anomaly scan reports generated by performing black box scans on applications using black box scanning tools; Acquire a plurality of original abnormal data according to the abnormal scanning report; Preprocessing the plurality of original abnormal data respectively to generate a plurality of abnormal data, wherein the abnormal data is used to characterize the abnormal parameter information in the abnormal scanning report; generating an abnormal data set according to the plurality of abnormal data; Using the abnormal data set as a training sample data set to train an initial abnormality detection model to be trained, thereby obtaining a trained abnormality detection model; The multiple original abnormal data include: request data and response data, wherein the request data includes original request data and modified request data; the response data includes response data corresponding to the original request data and response data corresponding to the modified request data; The preprocessing of the plurality of original abnormal data respectively to generate the plurality of abnormal data comprises: Performing feature extraction on the original request data, the response data corresponding to the original request data, the modified request data, and the response data corresponding to the modified request data, respectively, to determine first message length data, second message length data, third message length data, and fourth message length data; Determining a first message similarity based on the original request data and the modified request data; Determining a second message similarity based on the response data corresponding to the original request data and the response data corresponding to the modified request data; Determining a parameter type of a first message header of the original request data according to the message body data of the original request data; Determining a parameter type of a second message header of the modified request data according to the message body data of the modified request data; Determining first recurrence data according to the original request data and the response data corresponding to the original request data, wherein the first recurrence data is used to indicate whether a keyword appearing in the original request data appears in the response data corresponding to the original request data; determining second recurrence data according to the modified request data and the response data corresponding to the modified request data, wherein the second recurrence data is used to indicate whether a keyword appearing in the modified request data appears in the response data corresponding to the modified request data; Determining first return code data according to the response data corresponding to the original request data; Determine second return code data according to the response data corresponding to the modified request data, wherein the first return code data is data with HTTP / 1 as a header in the response data corresponding to the original request data, and the second return code data is data with HTTP / 1 as a header in the response data corresponding to the modified request data; The first message length data, the second message length data, the third message length data, the fourth message length data, the first message similarity, the second message similarity, the parameter type of the first message header, the parameter type of the second message header, the first reproduction data, the second reproduction data, the first return code data and the second return code data are taken as the multiple abnormal data.

2. The method according to claim 1, wherein: The data type of the abnormal data includes a numeric type and / or a Boolean type.

3. The method according to claim 1, wherein: The method of using the abnormal data set as a training sample data set to train the initial abnormality detection model to be trained to obtain the trained abnormality detection model includes: Dividing the training sample data set into a training set and a test set; Using the training set to train the initial anomaly detection model to be trained to obtain an anomaly detection model to be tested; The anomaly detection model to be tested is tested using the test set to obtain a verification result. If the verification result does not meet the iteration stop condition, the initial anomaly detection model is continuously trained until the verification result meets the iteration stop condition, wherein the verification result is used to characterize whether the anomaly data in the test set is true or false, wherein true characterizes that the anomaly data in the scan report is true anomaly data, and false characterizes that the anomaly data in the scan report is false alarm anomaly data; The model corresponding to the condition that stops the iteration is used as the anomaly detection model.

4. The method according to any one of claims 1 to 3, wherein: The initial anomaly detection model is constructed based on the Naive Bayes classification algorithm.

5. An anomaly detection method, comprising: Get the scan report to be tested; Acquire a sample data set to be tested according to the scan report to be tested; The sample data set to be tested is input into the anomaly detection model, and a detection result is output, wherein the anomaly detection model is trained by the anomaly detection model training method according to any one of claims 1 to 3.

6. The method according to claim 5, wherein: The step of obtaining a sample data set to be tested according to the scan report to be tested comprises: Acquire an original sample to be tested according to the scan report to be tested, wherein the original sample to be tested includes a plurality of original data; Preprocessing the plurality of original data respectively to obtain a plurality of test data, wherein the test samples in the test sample data set are composed of the plurality of test data, and the data types of the test data include numerical type and / or Boolean type; The plurality of to-be-tested data are used as to-be-tested sample data sets.

7. The method according to claim 5, wherein: Also includes: According to the detection result, the anomaly detection model is optimized using the sample data set to obtain an optimized anomaly detection model.

8. The method according to claim 7, wherein: The optimizing the anomaly detection model by using the sample data set to be tested according to the detection result to obtain the optimized detection model comprises: Determine whether the sample to be tested in the sample data set to be tested overlaps with the training sample of the anomaly detection model; When the samples to be tested in the sample data set to be tested do not overlap with the training samples of the anomaly detection model: Dividing the sample data set to be tested into an optimized training set and an optimized test set; Optimizing the anomaly detection model using the optimized training set to obtain an anomaly detection model to be optimized for testing; Using the optimized test set to test the test anomaly detection model to be optimized to obtain a verification result, if the verification result does not meet the iteration stop condition, continue to optimize the test anomaly detection model to be optimized until the verification result meets the iteration stop condition, wherein the verification result is used to characterize whether the abnormal data in the optimized test set is true or false, the true one characterizes that the abnormal data in the scan report to be tested is true abnormal data, and the false one characterizes that the abnormal data in the scan report to be tested is false positive abnormal data; and The model that meets the iteration stop condition is used as the optimized anomaly detection model; When the samples to be tested in the sample data set to be tested overlap with the training samples of the anomaly detection model: The anomaly detection model is optimized using the sample data set to obtain an optimized anomaly detection model.

9. The method according to claim 5, wherein: Also includes: A test report is generated according to the test result, and the test report is used to characterize the abnormal situation of each sample to be tested in the sample data set to be tested.

10. The method according to claim 5, wherein: The scan report to be tested and the scan report used to train the anomaly detection model are scan reports from different versions of the same application.

11. A training device for an anomaly detection model, comprising: A fifth acquisition module, configured to acquire a plurality of original abnormality data according to an abnormality scanning report generated after performing a black box scanning on the application based on a black box scanning tool; A first preprocessing module, used for preprocessing the plurality of original abnormal data respectively to generate a plurality of abnormal data; A first generating module, configured to generate an abnormal data set according to the plurality of abnormal data; A first acquisition module is used to acquire an abnormal data set, wherein the abnormal data set includes a plurality of abnormal data, and the abnormal data is used to characterize abnormal parameter information; A second acquisition module is used to acquire a training sample data set, wherein the training sample data set includes a plurality of the abnormal data sets; A training module, used to train the anomaly detection model to be trained using the training sample data set to obtain the anomaly detection model; The multiple original abnormal data include: request data and response data, wherein the request data includes original request data and modified request data; the response data includes response data corresponding to the original request data and response data corresponding to the modified request data; Wherein, the first preprocessing module includes: a feature extraction unit, configured to perform feature extraction on the original request data, the response data corresponding to the original request data, the modified request data, and the response data corresponding to the modified request data, respectively, to determine first message length data, second message length data, third message length data, and fourth message length data; A first determining unit, configured to determine a first message similarity based on the original request data and the modified request data; A second determining unit, configured to determine a second message similarity based on the response data corresponding to the original request data and the response data corresponding to the modified request data; A third determining unit, configured to determine a parameter type of a first message header of the original request data according to message body data of the original request data; a fourth determining unit, configured to determine a parameter type of a second message header of the modified request data according to the message body data of the modified request data; a fifth determining unit, configured to determine first recurrence data according to the original request data and the response data corresponding to the original request data, wherein the first recurrence data is used to indicate whether a keyword appearing in the original request data appears in the response data corresponding to the original request data; a sixth determining unit, configured to determine second recurring data according to the modified request data and the response data corresponding to the modified request data, wherein the second recurring data is used to indicate whether a keyword appearing in the modified request data appears in the response data corresponding to the modified request data; a seventh determining unit, configured to determine first return code data according to the response data corresponding to the original request data; an eighth determining unit, configured to determine second return code data according to the response data corresponding to the modified request data, wherein the first return code data is data with HTTP / 1 as a header in the response data corresponding to the original request data, and the second return code data is data with HTTP / 1 as a header in the response data corresponding to the modified request data; A generating unit, configured to use the first message length data, the second message length data, the third message length data, the fourth message length data, the first message similarity, the second message similarity, the parameter type of the first message header, the parameter type of the second message header, the first reproduction data, the second reproduction data, the first return code data, and the second return code data as the multiple abnormal data.

12. An abnormality detection device, comprising: The third acquisition module is used to obtain the scan report to be tested; A fourth acquisition module, used to acquire a sample data set to be tested according to the scan report to be tested; A detection module is used to input the sample data set to be tested into an anomaly detection model and output a detection result, wherein the anomaly detection model is trained by the training method of the anomaly detection model described in any one of claims 1 to 3.

13. An electronic device, comprising: one or more processors; A memory for storing one or more instructions, Wherein, when the one or more instructions are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 10.

14. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 10.

15. A computer program product, comprising computer executable instructions, which when executed are used to implement the method of any one of claims 1 to 10.

Images