Feature extraction method and apparatus for vulnerability reproduction screenshot, electronic device, and medium

Abstract

The present application relates to the technical field of network security, and provides a feature extraction method and apparatus for a vulnerability reproduction screenshot, an electronic device, and a medium. The feature extraction method for a vulnerability reproduction screenshot comprises: acquiring a vulnerability reproduction screenshot to be processed; determining a target vulnerability reproduction tool corresponding to said vulnerability reproduction screenshot; if the target vulnerability reproduction tool is a first-type vulnerability reproduction tool, extracting features of said vulnerability reproduction screenshot on the basis of a large language model; and if the target vulnerability reproduction tool is a second-type vulnerability reproduction tool, extracting features of said vulnerability reproduction screenshot on the basis of a pre-trained request response information recognition model and the large language model. Because the contents of vulnerability reproduction screenshots generated using different types of vulnerability reproduction tools may be different, different methods are used to perform feature extraction on the vulnerability reproduction screenshots generated using different types of vulnerability reproduction tools, so that features extracted can be more accurate.

Description

Feature extraction methods, devices, electronic equipment, and media from vulnerability reproduction screenshots

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 2024114769959, filed on October 22, 2024, entitled “Method, Apparatus, Electronic Device and Medium for Feature Extraction from Vulnerability Reproduction Screenshots”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the technical field of network security, and more specifically, to a method, apparatus, electronic device, and medium for feature extraction from vulnerability reproduction screenshots. Background Technology

[0004] A vulnerability refers to a flaw in the specific implementation of hardware, software, protocols, or system security policies, which allows attackers to access or damage the system without authorization. Based on their business attributes, vulnerabilities can be categorized into application vulnerabilities and general component vulnerabilities. Application vulnerabilities indicate security issues in the design, implementation, or configuration of applications within specific business scenarios. These vulnerabilities are typically related to specific functions or business logic of the application, and attackers can exploit them to perform unauthorized operations, tamper with data, or steal sensitive information at the application layer. General component vulnerabilities refer to security flaws in widely used third-party libraries, frameworks, protocols, or system components. Due to their prevalence, these vulnerabilities pose a threat to multiple systems and applications.

[0005] Currently, whether it's a vulnerability reproduction report for application business vulnerabilities or a vulnerability analysis report for general component vulnerabilities, technical personnel typically paste screenshots of the vulnerability reproduction into the report. However, in the current environment, extracting key information from vulnerability reproduction screenshots relies on manual analysis. Security researchers need to manually review the screenshots and extract important information related to the vulnerability, making the process time-consuming and susceptible to human factors, leading to oversights or misunderstandings. Summary of the Invention

[0006] This application provides a feature extraction method, apparatus, electronic device, and medium for vulnerability reproduction screenshots, in order to solve the problem that the extraction of important vulnerability-related information from vulnerability reproduction screenshots is easily affected by human factors, leading to oversights or misunderstandings.

[0007] In a first aspect, this application provides a feature extraction method for vulnerability reproduction screenshots, comprising: acquiring a vulnerability reproduction screenshot to be processed; determining a target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed; if the target vulnerability reproduction tool is a first type of vulnerability reproduction tool, extracting features of the vulnerability reproduction screenshot to be processed based on a large language model; if the target vulnerability reproduction tool is a second type of vulnerability reproduction tool, extracting features of the vulnerability reproduction screenshot to be processed based on a pre-trained request-response information recognition model and the large language model.

[0008] In this embodiment, the content of vulnerability reproduction screenshots generated by different types of vulnerability reproduction tools may differ. Therefore, the type of the target vulnerability reproduction tool is first determined, and then different methods are used to extract features from the vulnerability reproduction screenshots generated by different types of vulnerability reproduction tools, thereby making the extracted features more accurate. Furthermore, this process does not require direct intervention from staff, thus reducing the impact of human factors and minimizing the occurrence of oversights or misunderstandings.

[0009] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, determining the target vulnerability reproduction tool corresponding to the screenshot of the vulnerability to be processed includes: extracting text information from the screenshot of the vulnerability to be processed; using the large language model to extract tool keywords from the text information, and identifying the target vulnerability reproduction tool as either the first type of vulnerability reproduction tool or the second type of vulnerability reproduction tool based on the tool keywords.

[0010] In this embodiment of the application, the text information in the vulnerability reproduction screenshot is processed by a large language model, thereby quickly and accurately determining the tool keywords, which makes it more accurate to determine the type of vulnerability reproduction tool based on the tool keywords.

[0011] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the vulnerability reproduction tool of the second type is an HTTP testing tool, and the vulnerability reproduction tool of the first type is a non-HTTP testing tool.

[0012] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, based on the pre-trained request-response information recognition model and the large language model, the features of the vulnerability reproduction screenshot to be processed are extracted, including: segmenting and extracting the vulnerability reproduction screenshot to be processed based on the request-response information recognition model to obtain a request information screenshot containing request information and a response information screenshot containing response information; and obtaining the features of the vulnerability reproduction screenshot to be processed based on the large language model, the request information screenshot, and the response information screenshot.

[0013] In this embodiment of the application, a request information screenshot and a response information screenshot are obtained by using a request-response information recognition model, thereby distinguishing the request information and response information in the vulnerability reproduction screenshot, which facilitates subsequent feature extraction.

[0014] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, based on the large language model, the request information screenshot, and the response information screenshot, the features of the vulnerability reproduction screenshot to be processed are obtained, including: extracting the text from the request information screenshot and the response information screenshot respectively to obtain the request information text and the response information text; and using the large language model to extract the features from the request information text and the response information text to obtain the features of the vulnerability reproduction screenshot to be processed.

[0015] In this embodiment, by extracting text from the request and response screenshots respectively, a large language model can be used to extract features from the request and response texts, thus obtaining the features of the screenshot for reproducing the vulnerability to be processed. Because the request and response information are distinguished, the large language model avoids confusing the request and response information during feature extraction, preventing errors in the extracted features and improving the accuracy of the extracted features.

[0016] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the step of extracting features from the screenshot of the vulnerability to be processed based on the large language model includes: extracting the text content of the screenshot of the vulnerability to be processed; and using the large language model to extract features from the text content to obtain the features of the screenshot of the vulnerability to be processed.

[0017] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the features of the screenshot of the vulnerability to be processed include at least one of the following: HTTP request content, HTTP response content, command parameters, Uniform Resource Locator in the screenshot of the vulnerability to be processed, description of the function of the screenshot of the vulnerability to be processed, vulnerability reproduction tool name, and vulnerability reproduction tool type.

[0018] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the training process of the request-response information recognition model includes: obtaining a training dataset, which includes multiple screenshots of vulnerability reproduction, and each screenshot of vulnerability reproduction is labeled with a response information range and a request information range; training an initial model based on the training dataset to obtain a trained request-response information recognition model.

[0019] In this embodiment, the initial model is trained using screenshots of vulnerability reproduction labeled with the range of response information and the range of request information. This enables the initial model to learn the characteristics of the range of request information and response information, thereby allowing the trained request-response information recognition model to accurately identify the range of request information and response information in the vulnerability reproduction screenshots.

[0020] Secondly, this application provides a feature extraction device for vulnerability reproduction screenshots, comprising: an acquisition module and a processing module; the acquisition module is used to acquire a vulnerability reproduction screenshot to be processed; the processing module is used to determine the target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed; if the target vulnerability reproduction tool is a first type of vulnerability reproduction tool, the features of the vulnerability reproduction screenshot to be processed are extracted based on a large language model; if the target vulnerability reproduction tool is a second type of vulnerability reproduction tool, the features of the vulnerability reproduction screenshot to be processed are extracted based on a pre-trained request-response information recognition model and the large language model.

[0021] Thirdly, this application provides an electronic device, including: a memory and a processor, the memory and the processor being connected; the memory being used to store a program; the processor being used to invoke the program stored in the memory to execute the method described in the first aspect and / or in combination with any possible implementation of the first aspect.

[0022] Fourthly, this application provides a computer-readable storage medium, characterized in that it stores a computer program thereon, which, when executed by a computer, performs the method described in the first aspect and / or in combination with any possible implementation of the first aspect. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 is a flowchart illustrating the feature extraction method of the first vulnerability reproduction screenshot according to an embodiment of this application;

[0025] Figure 2 is a flowchart illustrating the feature extraction method for the second type of vulnerability reproduction screenshot shown in the embodiments of this application;

[0026] Figure 3 is a structural block diagram of a feature extraction device for vulnerability reproduction screenshots according to an embodiment of this application;

[0027] Figure 4 is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0028] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0029] It should be noted that similar reference numerals and letters in the following figures denote similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, relational terms such as "first," "second," etc., in the description of this application are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one…" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0030] The technical solution of this application will now be described in detail with reference to the accompanying drawings.

[0031] Please refer to Figure 1, which is a flowchart illustrating a feature extraction method for a vulnerability reproduction screenshot according to an embodiment of this application. The steps included in the method will be described below with reference to Figure 1.

[0032] S100: Obtain screenshots of the vulnerability to be reproduced.

[0033] Screenshots of vulnerability reproduction to be addressed can be pre-acquired and stored on local storage media, and then directly accessed when needed. Alternatively, screenshots of vulnerability reproduction to be addressed can be obtained in real-time from third-party devices or networks when required.

[0034] A vulnerability reproduction screenshot is a screenshot generated by a vulnerability reproduction tool after reproducing the vulnerability.

[0035] S200: Identify the target vulnerability reproduction tool corresponding to the screenshot of the vulnerability to be processed.

[0036] In one implementation, the specific method for determining the target vulnerability reproduction tool corresponding to the screenshot of the vulnerability to be processed can be as follows: First, extract the text information from the screenshot of the vulnerability to be processed. Then, use a large language model to extract tool keywords from the text information, and identify whether the target vulnerability reproduction tool corresponds to a first type of vulnerability reproduction tool or a second type of vulnerability reproduction tool based on the tool keywords.

[0037] The method for extracting text information from screenshots of the vulnerability to be reproduced can be any method used to extract text information from images, such as OCR (Optical Character Recognition) technology and online image-to-text tools. No specific restrictions are placed on the method used here.

[0038] In this implementation, only the tool keywords in the text information can be extracted using a large language model, and then the type of the target vulnerability reproduction tool can be determined according to the preset correspondence between the keywords and the vulnerability reproduction tool types.

[0039] One way to extract tool keywords from text information using a large language model is to input the extracted text information into the large language model, confirm the tool keyword prompts through preset representations, and then have the large language model output the tool keywords.

[0040] The prompts are, for example, "confirm the tool keywords in the text information" or "extract the tool keywords in the text information." The specific prompts can be set according to actual needs, as long as they have similar meanings.

[0041] Alternatively, the type of target vulnerability reproduction tool can be directly obtained using a large language model. For example, after inputting the extracted text information into the large language model, the type of vulnerability reproduction tool can be confirmed by pre-defined prompts, causing the large language model to output the type of the target vulnerability reproduction tool.

[0042] The prompts are, for example, "confirm the tool keywords in the text information and determine the type of vulnerability reproduction tool based on the tool keywords" or "confirm the type of vulnerability reproduction tool corresponding to the text information". The specific format of the prompts can be set according to actual needs, as long as their meanings are similar.

[0043] In one implementation, if the large language model has the ability to recognize text information in an image, the specific method for determining the target vulnerability reproduction tool corresponding to the screenshot of the vulnerability to be processed can also be as follows: First, input the screenshot of the vulnerability to be processed into the large language model. Then, use the large language model to extract tool keywords from the screenshot of the vulnerability to be processed, and identify the target vulnerability reproduction tool as either a first-type vulnerability reproduction tool or a second-type vulnerability reproduction tool based on the tool keywords.

[0044] Optionally, the method of using a large language model to extract tool keywords from the screenshot of the vulnerability to be processed can be as follows: First, use the prompt words of the representation to extract text information to control the large language model to extract text information from the screenshot of the vulnerability to be processed. Then, use the preset representation to confirm the prompt words of the tool keywords, so that the large language model outputs the tool keywords.

[0045] The prompts for extracting text information can be phrases such as "extract text from input image" or "obtain text information from input image," and there are no restrictions on their specific methods.

[0046] Alternatively, the method of extracting tool keywords from the screenshot of the vulnerability reproduction to be processed using a large language model can also be: directly confirm the tool keyword prompts in the image based on the representation, and then output the tool keywords by the large language model.

[0047] The prompts that identify the tool keywords in the image can be phrases such as "extract tool keywords from the input image" or "extract text information from the input image and obtain tool keywords from the text information," and there are no restrictions on the specific methods used here.

[0048] In one implementation, the specific method for determining the target vulnerability reproduction tool corresponding to the screenshot of the vulnerability to be processed can be as follows: first, extract the text information from the screenshot of the vulnerability to be processed; then, detect whether there are preset tool keywords in the text information.

[0049] If no tool keyword is found in the text information, it is determined that the screenshot of the vulnerability to be reproduced does not contain a corresponding target vulnerability reproduction tool.

[0050] If the text information contains keywords related to the target tool, the target vulnerability reproduction tool corresponding to the screenshot of the vulnerability to be processed is determined based on the preset correspondence between the tool keywords and the vulnerability reproduction tool.

[0051] Since different types of vulnerability reproduction tools have different tool keywords, the type of target vulnerability reproduction tool can be determined by identifying the tool keywords.

[0052] Optionally, tool keywords can represent information about different vulnerability reproduction tools. For example, they can be the name or number of the vulnerability reproduction tool, or the toolset information of different vulnerability reproduction tools. There are no restrictions on their specific types here.

[0053] Vulnerability reproduction tools can generally be divided into two main categories: HTTP testing tools and non-HTTP testing tools. Screenshots generated by HTTP testing tools typically include complete request and response information. However, screenshots generated by non-HTTP testing tools usually do not directly include request and response information.

[0054] S300: If the target vulnerability reproduction tool is a type 1 vulnerability reproduction tool, extract the features of the vulnerability reproduction screenshot to be processed based on the large language model.

[0055] Among them, the first type of vulnerability reproduction tool is a non-HTTP testing tool.

[0056] In one implementation, the specific method for extracting features from a screenshot of a vulnerability reproduction to be processed based on a large language model can be as follows: the screenshot of the vulnerability reproduction to be processed is used as input to the large language model, and then prompt words with features are extracted from the image based on a preset representation, so that the large language model outputs the features of the screenshot of the vulnerability reproduction to be processed.

[0057] In this case, the large language model is required to have the ability to recognize text information in the image.

[0058] The prompts for extracting features from an image can be phrases such as "extract key information from the input image" or "extract feature information from the input image," and there are no restrictions on their specific implementation methods here.

[0059] In one implementation, if the large language model has the ability to recognize text information in an image, the specific method for extracting features from the screenshot of the vulnerability to be reproduced based on the large language model can be as follows: First, the screenshot of the vulnerability to be reproduced is used as input to the large language model. Then, based on the prompt words for extracting text information from the representation, the large language model is controlled to extract the text information from the screenshot of the vulnerability to be reproduced. Afterwards, based on the preset prompt words for extracting features from the text information, the large language model outputs the features of the screenshot of the vulnerability to be reproduced.

[0060] The prompts used to extract text information are implemented in the same way as described above, and will not be repeated here for the sake of brevity.

[0061] The prompts that represent the extraction of features from text information can be, for example, "extract features from text information" or "extract key information from text information," and there are no restrictions on their specific implementation methods here.

[0062] In one implementation, when the target vulnerability reproduction tool is a type 1 vulnerability reproduction tool, the specific method for extracting features from the vulnerability reproduction screenshot based on a large language model can be as follows: First, extract the text content of the vulnerability reproduction screenshot. Then, use a large language model to extract features from the text content to obtain the features of the vulnerability reproduction screenshot.

[0063] The methods for extracting text content from screenshots and for extracting features from text content using large language models have been clearly described above and will not be repeated here for the sake of brevity.

[0064] In one implementation, if the target vulnerability reproduction tool corresponding to the screenshot of the vulnerability to be processed cannot be determined, features are extracted from the screenshot of the vulnerability to be processed in accordance with the method of S300.

[0065] In one implementation, the features of the screenshot of the vulnerability to be processed include at least one of the following: HTTP request content, HTTP response content, command parameters, Uniform Resource Locator in the screenshot of the vulnerability to be processed, description of the function of the screenshot of the vulnerability to be processed, vulnerability reproduction tool name, and vulnerability reproduction tool type.

[0066] S400: If the target vulnerability reproduction tool is a second-type vulnerability reproduction tool, extract the features of the vulnerability reproduction screenshot to be processed based on the pre-trained request-response information recognition model and large language model.

[0067] The second type of vulnerability reproduction tool is an HTTP testing tool.

[0068] In one implementation, the specific method for extracting features from a screenshot of a vulnerability reproduction to be processed, based on a pre-trained request-response information recognition model and a large language model, can be as follows: First, the screenshot of the vulnerability reproduction to be processed is segmented and extracted based on the request-response information recognition model to obtain a request information screenshot containing request information and a response information screenshot containing response information. Then, based on the large language model, the request information screenshot, and the response information screenshot, the features of the screenshot of the vulnerability reproduction to be processed are obtained.

[0069] The screenshots of the request and response information are both part of the screenshots showing the reproduction of the vulnerability to be processed.

[0070] Optionally, the specific implementation method for segmenting and extracting the vulnerability reproduction screenshot based on the request-response information recognition model can be as follows: input the vulnerability reproduction screenshot to be processed into the request-response information recognition model so that the request-response information recognition model processes the vulnerability reproduction screenshot to obtain the request information screenshot and response information screenshot output by the request-response information recognition model.

[0071] Accordingly, the training process for the request-response information recognition model is as follows: First, a training dataset is obtained, which includes multiple screenshots of vulnerability reproduction, as well as the corresponding request and response information screenshots for each vulnerability reproduction screenshot. Then, the initial model is trained based on the training dataset to obtain the trained request-response information recognition model.

[0072] The specific methods and principles of training the model are well known to those skilled in the art, and will not be elaborated upon here for the sake of brevity.

[0073] Alternatively, the specific implementation of segmenting and extracting the vulnerability reproduction screenshot based on the request-response information recognition model can also be as follows: The screenshot of the vulnerability reproduction to be processed is input into the request-response information recognition model, which processes the screenshot to obtain a screenshot labeled with the scope of the request information and the scope of the response information. Then, based on the labeled scope of the request information and the scope of the response information, the screenshot is segmented to obtain request information screenshots and response information screenshots respectively.

[0074] The scope of the request information / response information can be information containing multiple coordinates, and the area enclosed by the multiple coordinates represents the scope of the request information / response information.

[0075] Alternatively, the request / response information range can also include a reference point coordinate and length and width information. In this approach, the request / response information range can be a rectangle (with the length and width corresponding to the length and width in the request / response information range) enclosed by the point corresponding to the reference point coordinate (for example, any corner or center point of a rectangle).

[0076] Accordingly, the coordinate system can be a conventional machine vision coordinate system such as the image coordinate system, world coordinate system, or camera coordinate system. Alternatively, it can be a custom-defined coordinate system, for example, one that specifies the top-left corner of the image as the origin, the direction of horizontally arranged pixels as the x-axis, and the direction of vertically arranged pixels as the y-axis. This example is provided for ease of understanding.

[0077] Accordingly, the training process for the request-response information recognition model is as follows: First, a training dataset is obtained, which includes multiple screenshots of vulnerability reproduction, and each screenshot is labeled with the range of response information and the range of request information. Then, the initial model is trained based on the training dataset to obtain the trained request-response information recognition model.

[0078] The range of response information and the range of request information corresponding to each vulnerability reproduction screenshot can be manually marked using annotation tools (such as LabelImg).

[0079] Since the scope of the request information and the scope of the response information are known, the specific method for segmenting the screenshot of the vulnerability reproduction to be processed based on the marked scope of the request information and the scope of the response information can be as follows: Cropping the image of the region corresponding to the scope of the request information from the screenshot of the vulnerability reproduction to obtain the request information screenshot; and cropping the image of the region corresponding to the scope of the response information from the screenshot of the vulnerability reproduction to obtain the response information screenshot.

[0080] The cropping operation can be achieved using various screenshot software or programs with screenshot or cropping functions. For example, Burp Suite can be used to segment the screenshot of the vulnerability reproduction to be processed based on the range of the labeled request information and the range of the labeled response information. This example is only for illustrative purposes and should not be construed as a limitation of this application.

[0081] In one implementation, the specific method for obtaining the features of the vulnerability reproduction screenshot to be processed based on a large language model, request information screenshots, and response information screenshots can be as follows: First, extract the text from the request information screenshots and response information screenshots respectively to obtain request information text and response information text. Then, aggregate the request information text and response information text to obtain vulnerability reproduction information text. Finally, use a large language model to extract features from the vulnerability reproduction information text to obtain the features of the vulnerability reproduction screenshot to be processed.

[0082] The method of extracting text from the screenshots of request and response information is the same as the method of extracting text information from the screenshots of the vulnerability to be processed, as described above. For the sake of brevity, it will not be repeated here.

[0083] Optionally, the feature extraction of vulnerability reproduction information text using a large language model to obtain the feature of the vulnerability reproduction screenshot to be processed can be: inputting the vulnerability reproduction information text into the large language model, and then extracting feature prompts from the vulnerability reproduction information text based on preset representations, so that the large language model outputs the feature of the vulnerability reproduction screenshot to be processed.

[0084] The prompts that indicate the extraction of features from the vulnerability reproduction information text can be, for example, "extract features from the vulnerability reproduction information text" or "extract key information from the vulnerability reproduction information text." No specific restrictions are placed on their implementation here.

[0085] In one implementation, if the large language model has the ability to recognize text information in an image, the specific way to obtain the features of the screenshot of the vulnerability reproduction to be processed based on the large language model, the request information screenshot, and the response information screenshot can be as follows: First, the request information screenshot and / or the response information screenshot are input into the large language model. Then, based on a preset representation, prompt words for text information are extracted from the request information screenshot and the response information screenshot, enabling the large language model to extract the request information text and the response information text. Afterwards, the prompt words for feature extraction are used to extract features from the request information text and the response text, enabling the large language model to extract the features of the screenshot of the vulnerability reproduction to be processed.

[0086] The prompts indicating the extraction of text information from the request and response screenshots can be, for example, "extract text information from the input image," "extract text information from the request screenshot and the response screenshot," or "extract text information from both the request and response screenshots." No specific implementation details are specified here.

[0087] The prompts that represent the features extracted from the request and response texts can be, for example, "extract features from all extracted text information" or "extract key information from the request and response texts," and there are no restrictions on their specific implementation methods here.

[0088] To facilitate understanding of the training process of the request-response information recognition model described above, an example will be provided below.

[0089] First, you can collect screenshots of vulnerability reproductions publicly available on the internet. For example, you can collect relevant technical reports and articles from cybersecurity forums like Freebuf, open-source repositories on GitHub, and technical blogs from security research institutes, and extract and store the vulnerability reproduction screenshots from them.

[0090] Then, the collected screenshots of vulnerability reproduction are annotated. The Labelme client can be used for image annotation. Objects in the images are categorized into "request" and "response." Labelme's polygon tool is used to annotate request and response information blocks in the images. "request" is used to annotate request information, and "response" is used to annotate response information.

[0091] The processed JSON annotation files are then converted to a YOLO (You Only Look Once, a detection algorithm) compatible annotation file format. Since Labelme typically exports annotation files in JSON format, they need to be converted to a YOLO-compatible TXT file format. Each YOLO annotation file is a plain text (.txt) file, corresponding to the image filename. Its format can be...<class_id><center_x><center_y> <width> <height>,<class_id> This indicates the category of the image tag box (i.e., "request" or "response").<center_x><center_y> Indicates the x and y coordinates of the bounding box reference point. <width> <height>This represents the height and width of the bounding box. This method can generate annotation files for multiple training images. These training images and their corresponding annotation files are used as a training dataset, which is then used to train the initial model. The training images are screenshots reproduced from publicly available vulnerability data collected from the internet.

[0092] You can use YOLOv8 model training tools to build a request-response information block recognition model, such as choosing YOLOv8n, YOLOv8m, etc. as the base training.

[0093] The model is trained using training images and labeled training files, and its weights and parameters are updated to obtain the final recognition model. The optimal weights and parameters are then selected as the final model.

[0094] To facilitate understanding of the feature extraction method for the vulnerability reproduction screenshots described above, the following explanation will be provided in conjunction with Figure 2.

[0095] As shown in Figure 2, first, obtain a screenshot of the vulnerability reproduction to be processed, and then use an OCR tool to extract the text information from the screenshot. This can be done using the Chinese and English recognition models of the open-source OCR software Tesseract.

[0096] Then identify the vulnerability reproduction tool used in the image. This can be done by checking if the extracted text information contains keywords such as Burp Suite or SQL Map. If so, the vulnerability reproduction tool used in the image can be identified based on these keywords.

[0097] Alternatively, a prompt word approach can be used, sending text information combined with the prompt word to a large language model. The large language model then makes a toolset determination.

[0098] The prompt used here could be: "The following is a my request:"

[0099] The following are the OCR extraction results of the image,Please extract the body of the response:{{OCR TEXT}};

[0100] You need to return the results to me in JSON objects JSON objects of type"UseTool"according to the following JSON schema definitions:{{JSON SCHEMA}};

[0101] Return JSON object with 2 spaces of indentation.”

[0102] Here, {{OCR_TEXT}} refers to the previously extracted text information, and {{JSON_SCHEMA}} is the schema protocol for the required data class. This prompt indicates that the large language model should analyze the text information and return the name of the vulnerability reproduction tool used. This prompt returns the vulnerability reproduction tool name in JSON format. The returned result uses `{` and `}` as the start and end characters for content extraction, and then converts the result to Python's dict or dataclass data type.

[0103] Next, the tool used to reproduce the vulnerability is used to determine whether it is an HTTP request testing tool. For example, Burp Suite is an HTTP request testing tool, and this type of tool generally contains clear request and response information.

[0104] For screenshots of vulnerability reproduction using HTTP testing tools, a request-response information recognition model is used to identify the scope of request and response information in the screenshots. The image regions corresponding to the respective request and response information scopes are extracted from the recognition results, resulting in and saving the request and response information screenshots.

[0105] The request and response screenshots are subjected to OCR recognition to obtain the request and response information, respectively. Then, a prompt word engineering process is used to allow the large language model to return structured request and response information. The prompt word here is similar to that used previously, namely "The following is a my request:

[0106] The following are the OCR extraction results of the image,Please extract the body of the response:{{OCR TEXT}};

[0107] You need to return the results to me in JSON objects JSON objects of type"UseTool"according to the following JSON schema definitions:{{JSON SCHEMA}};

[0108] Return JSON object with 2 spaces of indentation.”

[0109] The Chinese translation of the above prompt is: "The following are my requirements:"

[0110] The following are the OCR extraction results of the image. Please extract the response text: {{OCR TEXT}};

[0111] Your output needs to conform to the JSON Scheme definition I provided, returning a JSON object of type "UseTool": {{JSON SCHEMA}};

[0112] The JSON result is indented to two spaces.

[0113] However, the OCR text here is structured request and response information, while the JSON schema is a predefined JSON schema of the HttpRequest and HttpResponse data models. The JSON schema passed to the large language model needs to be descriptive information that the large language model can understand.

[0114] For example, a JSON schema could be: "HttpRequest(Http request object):Http_method(str):extract Http request method, like:GET,POST"; url(str):Http request url"; headers(dict):Http request headers"; body:Http request body". HttpResponse(Http response object):status_code(str):Http response status code"; headers(dict):Http response headers"; body(str):Http response body". Correspondingly, the large language model will extract the information defined in the JSON schema from the input request and response information, namely, extracting the Http request object, Http request method, Http request url, Http request headers, Http request body, Http response object, Http response status code, Http response headers, and Http response body. This example is for illustrative purposes only; actual usage can be more flexible than this example.

[0115] The large language model aggregates and extracts the acquired structured request-response information and tool information, returning structured key image information (i.e., the characteristics of the aforementioned screenshot of the vulnerability reproduction to be processed). The structured key image information results include fields and descriptions such as: "HTTP request content: If possible, you should extract the complete HTTP request content that conforms to the specifications; HTTP response content: If possible, you should extract the complete HTTP response content; Command parameters: Possible terminal commands in the image; Uniform Resource Locator: If possible, you should extract the verifiable URL address; Description of the function of the screenshot of the vulnerability reproduction to be processed: If possible, you should briefly describe the content of the image; Vulnerability reproduction tool name: The name of the vulnerability reproduction tool used in the image."

[0116] For non-HTTP request testing tools, the text information of the screenshot of the vulnerability to be addressed, along with the JSON schema model data of the key information in the structured image, can be combined using prompt word engineering to allow the Large Language Model (LLM) to return structured result data. The prompt words are the same as those mentioned above, but here the OCR TEXT is text information, and the JSON SCHEMA is the predefined JSON schema information of the HTTPRequest and HTTPResponse data models.

[0117] Additionally, the following instructions can be included as a System message to enhance the understanding capabilities of the large language model: "You are a security vulnerability image expert. Although you do not have the ability to directly analyze and process;"

[0118] images, users will upload the extracted content after OCR processing, so you can assist customers in analysis;

[0119] vulnerability recurrence or verification by analyzing the OCR processed content.Screenshot.Generally speaking, the user's OCR;

[0120] results may not be accurate. You'd better optimize based on the content.”

[0121] By using System Message, the role of the large language model is restricted to network security vulnerability image analysis experts, and the system informs them that it will provide text information for analysis and result optimization.

[0122] Based on the same technical concept, this application also provides a feature extraction device for vulnerability reproduction screenshots, as shown in Figure 3. The feature extraction device 100 for vulnerability reproduction screenshots includes an acquisition module 110 and a processing module 120.

[0123] Module 110 is used to obtain screenshots of the vulnerability reproduction to be processed.

[0124] The processing module 120 is used to determine the target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed; if the target vulnerability reproduction tool is a first type of vulnerability reproduction tool, the features of the vulnerability reproduction screenshot to be processed are extracted based on a large language model; if the target vulnerability reproduction tool is a second type of vulnerability reproduction tool, the features of the vulnerability reproduction screenshot to be processed are extracted based on a pre-trained request-response information recognition model and the large language model.

[0125] The processing module 120 is specifically used to extract text information from the screenshot of the vulnerability reproduction to be processed; to extract tool keywords from the text information using the large language model; and to identify the target vulnerability reproduction tool as either the first type of vulnerability reproduction tool or the second type of vulnerability reproduction tool based on the tool keywords.

[0126] In one implementation, the second type of vulnerability reproduction tool is an HTTP testing tool, while the first type of vulnerability reproduction tool is a non-HTTP testing tool.

[0127] The processing module 120 is specifically used to segment and extract the screenshot of the vulnerability reproduction to be processed based on the request-response information recognition model to obtain a screenshot of request information containing request information and a screenshot of response information containing response information; and to obtain the features of the screenshot of the vulnerability reproduction to be processed based on the large language model, the screenshot of request information and the screenshot of response information.

[0128] The processing module 120 is specifically used to extract the text from the request information screenshot and the response information screenshot respectively to obtain the request information text and the response information text; and to extract the features from the request information text and the response information text using the large language model to obtain the features of the vulnerability reproduction screenshot to be processed.

[0129] The processing module 120 is specifically used to extract the text content of the screenshot of the vulnerability reproduction to be processed; and to extract the features in the text content using the large language model to obtain the features of the screenshot of the vulnerability reproduction to be processed.

[0130] In one implementation, the features of the screenshot of the vulnerability to be processed include at least one of the following: HTTP request content, HTTP response content, command parameters, Uniform Resource Locator in the screenshot of the vulnerability to be processed, description of the function of the screenshot of the vulnerability to be processed, vulnerability reproduction tool name, and vulnerability reproduction tool type.

[0131] The processing module 120 is also used to acquire a training dataset, which includes multiple screenshots of vulnerability reproduction, and each screenshot of vulnerability reproduction is labeled with a response information range and a request information range; the initial model is trained based on the training dataset to obtain a trained request-response information recognition model.

[0132] The feature extraction device 100 for vulnerability reproduction screenshots provided in this application embodiment has the same implementation principle and technical effect as the aforementioned feature extraction method embodiment for vulnerability reproduction screenshots. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned feature extraction method embodiment for vulnerability reproduction screenshots.

[0133] Please refer to Figure 4, which shows an electronic device 200 provided in an embodiment of this application. The electronic device 200 includes: a processor 210 and a memory 220.

[0134] The components of the memory 220 and processor 210 are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The memory 220 stores computer programs, such as the software functional module shown in Figure 3, namely the feature extraction device 100 for vulnerability reproduction screenshots. The feature extraction device 100 for vulnerability reproduction screenshots includes at least one software functional module that can be stored in the memory 220 in the form of software or firmware or embedded in the operating system (OS) of the electronic device 200. The processor 210 executes the executable modules stored in the memory 220, such as the software functional modules or computer programs included in the feature extraction device 100 for vulnerability reproduction screenshots. At this time, the processor 210 is used to acquire a screenshot of the vulnerability to be processed; determine the target vulnerability reproduction tool corresponding to the screenshot of the vulnerability to be processed; if the target vulnerability reproduction tool is a first type of vulnerability reproduction tool, extract the features of the screenshot of the vulnerability to be processed based on a large language model; if the target vulnerability reproduction tool is a second type of vulnerability reproduction tool, extract the features of the screenshot of the vulnerability to be processed based on a pre-trained request-response information recognition model and the large language model.

[0135] The memory 220 can be, but is not limited to, RAM (Random Access Memory), ROM (Read Only Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0136] Processor 210 may be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be a DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor, or processor 210 can be any conventional processor.

[0137] Among them, the aforementioned electronic devices 200 include, but are not limited to, personal computers, servers, etc.

[0138] This application also provides a computer-readable storage medium (hereinafter referred to as the storage medium) storing a computer program. When the computer program is run by a computer, such as the electronic device 200 described above, it executes the feature extraction method for vulnerability reproduction screenshots as described above. The computer-readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0139] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.< / height> < / width> < / height> < / width>

Claims

1. A method for feature extraction from screenshots of vulnerability reproduction, characterized in that, include: Obtain screenshots of the vulnerability to be reproduced; Identify the target vulnerability reproduction tool corresponding to the screenshot of the vulnerability to be processed; If the target vulnerability reproduction tool is a first type of vulnerability reproduction tool, the features of the vulnerability reproduction screenshot to be processed are extracted based on the large language model. If the target vulnerability reproduction tool is a second type of vulnerability reproduction tool, the features of the vulnerability reproduction screenshot to be processed are extracted based on the pre-trained request-response information recognition model and the large language model.

2. The method according to claim 1, characterized in that, The target vulnerability reproduction tool corresponding to the screenshot of the vulnerability to be processed includes: Extract the text information from the screenshot of the vulnerability reproduction to be addressed; The tool keywords in the text information are extracted using the large language model, and the target vulnerability reproduction tool is identified as either a first type or a second type of vulnerability reproduction tool based on the tool keywords.

3. The method according to claim 1, characterized in that, The vulnerability reproduction tool of the second type is an HTTP testing tool, while the vulnerability reproduction tool of the first type is a non-HTTP testing tool.

4. The method according to claim 1, characterized in that, Based on the pre-trained request-response information recognition model and the large language model, features are extracted from the screenshot of the vulnerability reproduction to be processed, including: Based on the request-response information recognition model, the screenshot of the vulnerability reproduction to be processed is segmented and extracted to obtain a screenshot of request information containing request information and a screenshot of response information containing response information. Based on the large language model, the screenshot of the request information, and the screenshot of the response information, the characteristics of the screenshot of the vulnerability reproduction to be processed are obtained.

5. The method according to claim 4, characterized in that, Based on the large language model, the request information screenshot, and the response information screenshot, the characteristics of the screenshot of the vulnerability reproduction to be processed are obtained, including: Extract the text from the screenshot of the request information and the screenshot of the response information respectively to obtain the request information text and the response information text; The features of the request information text and the response information text are extracted using the large language model to obtain the features of the screenshot of the vulnerability to be reproduced.

6. The method according to claim 1, characterized in that, The feature extraction of the screenshot of the vulnerability reproduction to be processed based on the large language model includes: Extract the text content from the screenshot of the vulnerability reproduction to be addressed; The features of the text content are extracted using the large language model to obtain the features of the screenshot of the vulnerability to be reproduced.

7. The method according to claim 1, characterized in that, The features of the vulnerability reproduction screenshot to be processed include at least one of the following: HTTP request content, HTTP response content, command parameters, Uniform Resource Locator in the vulnerability reproduction screenshot to be processed, description of the vulnerability reproduction screenshot function, vulnerability reproduction tool name, and vulnerability reproduction tool type.

8. The method according to any one of claims 1-7, characterized in that, The training process of the request-response information recognition model includes: Obtain a training dataset, which includes multiple screenshots of vulnerability reproduction, and each screenshot of vulnerability reproduction is labeled with the range of response information and the range of request information. The initial model is trained based on the training dataset to obtain a trained request-response information recognition model.

9. A feature extraction device for vulnerability reproduction screenshots, characterized in that, include: The acquisition module is used to acquire screenshots of the vulnerability reproduction to be processed; The processing module is used to determine the target vulnerability reproduction tool corresponding to the screenshot of the vulnerability to be processed; If the target vulnerability reproduction tool is a first type of vulnerability reproduction tool, the features of the vulnerability reproduction screenshot to be processed are extracted based on the large language model; if the target vulnerability reproduction tool is a second type of vulnerability reproduction tool, the features of the vulnerability reproduction screenshot to be processed are extracted based on the pre-trained request-response information recognition model and the large language model.

10. An electronic device, characterized in that, include: A memory and a processor, wherein the memory and the processor are connected; The memory is used to store programs; The processor is configured to invoke a program stored in the memory to execute the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a computer, performs the method as described in any one of claims 1-8.

Images