Feature extraction method and device for vulnerability reproduction screenshot, electronic equipment and medium

Abstract

The invention provides a feature extraction method and device for a vulnerability reproduction screenshot, electronic equipment and a medium, and relates to the technical field of network security. The feature extraction method for the vulnerability reproduction screenshot comprises the following steps: acquiring a to-be-processed vulnerability reproduction screenshot; determining a target vulnerability reproduction tool corresponding to the to-be-processed vulnerability reproduction screenshot; if the target vulnerability reproduction tool is the vulnerability reproduction tool of the first type, extracting features of the to-be-processed vulnerability reproduction screenshot based on a large language model; and if the target vulnerability reproduction tool is the vulnerability reproduction tool of the second type, extracting features of the to-be-processed vulnerability reproduction screenshot based on a pre-trained request response information identification model and a large language model. The contents of the vulnerability reproduction screenshots generated by different types of vulnerability reproduction tools may be different, so that feature extraction is performed on the vulnerability reproduction screenshots generated by different types of vulnerability reproduction tools in different modes, and the extracted features can be more accurate.

Description

Technical Field

[0001] The present application relates to the technical field of network security, and more specifically, to a method, device, electronic device, and medium for extracting features of vulnerability reproduction screenshots. Background Art

[0002] Vulnerabilities refer to defects in the specific implementation of hardware, software, or protocols or in system security policies, which can enable attackers to access or damage the system without authorization. According to the business attributes of the vulnerabilities, they can be divided into application business vulnerabilities and common component vulnerabilities. Application business vulnerabilities refer to security issues in the design, implementation, or configuration of applications in specific business scenarios. Such vulnerabilities are usually related to specific functions or business logic of the application. Attackers can exploit these vulnerabilities to perform unauthorized operations, tamper with data, or steal sensitive information at the application level. Common component vulnerabilities refer to security defects in widely used third-party libraries, frameworks, protocols, or system components. Due to their universality, these vulnerabilities pose a threat to multiple systems and applications.

[0003] At present, whether it is a vulnerability reproduction report of an application business vulnerability or a vulnerability analysis report of a common component vulnerability, technicians generally paste vulnerability reproduction screenshots of the vulnerability in the report. However, in the current environment, the extraction of key information from vulnerability reproduction screenshots relies on manual analysis, requiring security researchers to manually view the screenshots and extract important information related to the vulnerability, making the process time-consuming and easily affected by human factors, leading to problems such as negligence or misunderstanding. Summary of the invention

[0004] The present application provides a feature extraction method, device, electronic device and medium for vulnerability reproduction screenshots to solve the problem that the extraction of important vulnerability-related information from vulnerability reproduction screenshots is easily affected by human factors, resulting in negligence or misunderstanding.

[0005] In a first aspect, the present application provides a feature extraction method for vulnerability reproduction screenshots, comprising: obtaining a vulnerability reproduction screenshot to be processed; determining a target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed; if the target vulnerability reproduction tool is a first type of vulnerability reproduction tool, based on a large language model, extracting features of the vulnerability reproduction screenshot to be processed; if the target vulnerability reproduction tool is a second type of vulnerability reproduction tool, based on a pre-trained request response information recognition model and the large language model, extracting features of the vulnerability reproduction screenshot to be processed.

[0006] In the embodiment of the present application, the contents of the vulnerability reproduction screenshots generated by different types of vulnerability reproduction tools may be different, therefore, the type of the target vulnerability reproduction tool is first determined, and then different methods are used to extract features for the vulnerability reproduction screenshots generated by different types of vulnerability reproduction tools, so that the extracted features can be more accurate. In addition, this process does not require direct participation by staff, which reduces the impact of human factors, thereby reducing the occurrence of negligence or misunderstanding.

[0007] In combination with the technical solution provided in the first aspect above, in some possible implementations, determining the target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed includes: extracting text information in the vulnerability reproduction screenshot to be processed; using the large language model to extract tool keywords in the text information, and identifying the target vulnerability reproduction tool as the first type of vulnerability reproduction tool or the second type of vulnerability reproduction tool based on the tool keywords.

[0008] In an embodiment of the present application, the text information in the vulnerability reproduction screenshot is processed by a large language model, so that the tool keywords can be quickly and accurately determined, thereby making it more accurate to determine the type of vulnerability reproduction tool based on the tool keywords.

[0009] In combination with the technical solution provided in the first aspect above, in some possible implementations, the second type of vulnerability reproduction tool is an Http test type tool, and the first type of vulnerability reproduction tool is a non-Http test type tool.

[0010] In combination with the technical solution provided in the first aspect above, in some possible implementations, based on a pre-trained request-response information recognition model and the large language model, features of the vulnerability reproduction screenshot to be processed are extracted, including: segmenting and extracting the vulnerability reproduction screenshot to be processed based on the request-response information recognition model to obtain a request information screenshot containing request information and a response information screenshot containing response information; and obtaining features of the vulnerability reproduction screenshot to be processed based on the large language model, the request information screenshot, and the response information screenshot.

[0011] In an embodiment of the present application, a request information screenshot and a response information screenshot of the response information are obtained through a request response information recognition model, so as to distinguish the request information and the response information in the vulnerability reproduction screenshot, thereby facilitating subsequent feature extraction.

[0012] In combination with the technical solution provided in the first aspect above, in some possible implementations, based on the large language model, the request information screenshot and the response information screenshot, the features of the vulnerability reproduction screenshot to be processed are obtained, including: extracting the text in the request information screenshot and the response information screenshot respectively to obtain the request information text and the response information text; using the large language model to extract the features in the request information text and the response information text to obtain the features of the vulnerability reproduction screenshot to be processed.

[0013] In the embodiment of the present application, by extracting the text in the request information screenshot and the response information screenshot respectively, the large language model can be used to extract the features in the request information text and the response information text respectively, and obtain the features of the vulnerability reproduction screenshot to be processed. Since the request information and the response information are distinguished, the large language model will not confuse the request information and the response information when extracting features, resulting in errors in the extracted features, thereby improving the accuracy of the extracted features.

[0014] In combination with the technical solution provided in the first aspect above, in some possible implementations, extracting features of the vulnerability reproduction screenshot to be processed based on a large language model includes: extracting text content of the vulnerability reproduction screenshot to be processed; using the large language model to extract features in the text content to obtain features of the vulnerability reproduction screenshot to be processed.

[0015] In combination with the technical solution provided in the first aspect above, in some possible implementations, the characteristics of the vulnerability reproduction screenshot to be processed include at least one of Http request content, Http response content, command parameters, a uniform resource locator in the vulnerability reproduction screenshot to be processed, a description of the function of the vulnerability reproduction screenshot to be processed, a name of a vulnerability reproduction tool, and a type of vulnerability reproduction tool.

[0016] In combination with the technical solution provided in the first aspect above, in some possible implementations, the training process of the request-response information recognition model includes: obtaining a training data set, the training data set including multiple vulnerability reproduction screenshots, and each vulnerability reproduction screenshot is annotated with a response information range and a request information range; training the initial model based on the training data set to obtain a trained request-response information recognition model.

[0017] In an embodiment of the present application, the initial model is trained by taking vulnerability reproduction screenshots marked with response information ranges and request information ranges, so that the initial model can learn the characteristics of the request information and response information ranges, and thus the trained request response information recognition model can accurately recognize the request information range and response information range in the vulnerability reproduction screenshots.

[0018] In a second aspect, the present application provides a feature extraction device for a vulnerability reproduction screenshot, comprising: an acquisition module and a processing module, the acquisition module being used to acquire a vulnerability reproduction screenshot to be processed; the processing module being used to determine a target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed; if the target vulnerability reproduction tool is a first type of vulnerability reproduction tool, based on a large language model, extracting features of the vulnerability reproduction screenshot to be processed; if the target vulnerability reproduction tool is a second type of vulnerability reproduction tool, based on a pre-trained request response information recognition model and the large language model, extracting features of the vulnerability reproduction screenshot to be processed.

[0019] In a third aspect, the present application provides an electronic device, comprising: a memory and a processor, the memory and the processor being connected; the memory being used to store programs; and the processor being used to call the programs stored in the memory to execute the method described in the first aspect above and / or in combination with any possible implementation of the first aspect above.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium, characterized in that a computer program is stored thereon, and when the computer program is executed by a computer, it executes the method described in the first aspect above and / or in combination with any possible implementation method of the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A schematic diagram of the process of extracting features of a first vulnerability reproduction screenshot shown in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of the process of extracting features of a second vulnerability reproduction screenshot shown in an embodiment of the present application;

[0024] Figure 3 A structural block diagram of a feature extraction device for a vulnerability reproduction screenshot shown in an embodiment of the present application;

[0025] Figure 4 A structural block diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0027] It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it is not necessary to further define and explain it in the subsequent drawings. At the same time, in the description of this application, relational terms such as "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements that are not clearly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.

[0028] The technical solution of the present application will be described in detail below with reference to the accompanying drawings.

[0029] See also Figure 1 , Figure 1 This is a flow chart of a feature extraction method for a vulnerability reproduction screenshot shown in an embodiment of the present application. Figure 1 Describe the steps it involves.

[0030] S100: Get a screenshot of the vulnerability to be processed.

[0031] The screenshots of the vulnerability reproduction to be processed can be obtained in advance and stored in a local storage medium, and can be directly called when needed. Alternatively, the screenshots of the vulnerability reproduction to be processed can also be obtained from a third-party device or network in real time when needed.

[0032] The vulnerability reproduction screenshot is a screenshot of the result generated after the vulnerability reproduction tool reproduces the vulnerability.

[0033] S200: Determine a target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed.

[0034] In one implementation, a specific method for determining the target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed may be: first extracting text information in the vulnerability reproduction screenshot to be processed. Then, using a large language model to extract tool keywords in the text information, and identifying the target vulnerability reproduction tool as the first type of vulnerability reproduction tool or the second type of vulnerability reproduction tool based on the tool keywords.

[0035] The method of extracting text information from the screenshot of the vulnerability reproduction to be processed may be any method of extracting text information from an image, such as OCR (Optical Character Recognition) technology and online image-to-text tools, etc. The specific method is not limited here.

[0036] In this implementation, only the large language model may be used to extract tool keywords in text information, and then the type of the target vulnerability reproduction tool may be confirmed based on the correspondence between preset keywords and the type of vulnerability reproduction tool.

[0037] The method of extracting tool keywords from text information using the large language model may be: after inputting the extracted text information into the large language model, confirming the tool keyword prompt word through a preset representation, so that the large language model outputs the tool keyword.

[0038] Among them, the prompt words are, for example, "confirm tool keywords in text information", "extract tool keywords in text information", etc. The specific method of the prompt words can be set according to actual needs as long as the meanings they represent are similar.

[0039] Alternatively, the type of the target vulnerability reproduction tool may be directly obtained using the large language model. For example, after the extracted text information is input into the large language model, a preset prompt word representing the type of the vulnerability reproduction tool is used to make the large language model output the type of the target vulnerability reproduction tool.

[0040] Among them, the prompt words are, for example, "confirm the tool keywords in the text information, and determine the type of vulnerability reproduction tool based on the tool keywords", "confirm the type of vulnerability reproduction tool corresponding to the text information", etc. The specific method of the prompt words can be set according to actual needs, as long as the meaning they represent is similar.

[0041] In one implementation, if the large language model has the ability to recognize text information in an image, the specific method for determining the target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed may also be: first, the vulnerability reproduction screenshot to be processed is input into the large language model. Then, the large language model is used to extract tool keywords in the vulnerability reproduction screenshot, and the target vulnerability reproduction tool is identified as the first type of vulnerability reproduction tool or the second type of vulnerability reproduction tool according to the tool keywords.

[0042] Optionally, the method of using the large language model to extract tool keywords in the screenshot of the vulnerability reproduction to be processed can be: first, use the prompt words that represent the extracted text information to control the large language model to extract the text information in the screenshot of the vulnerability reproduction to be processed, and then confirm the prompt words of the tool keywords through a preset representation, so that the large language model outputs the tool keywords.

[0043] Among them, the prompt words representing the extraction of text information can be "extract text from the input image", "obtain text information from the input image", etc., and the specific method is not limited here.

[0044] Optionally, the method of using the large language model to extract the tool keywords in the screenshot of the vulnerability reproduction to be processed may also be: directly based on the prompt words representing the tool keywords in the confirmation picture, the large language model outputs the tool keywords.

[0045] Among them, the prompt words representing the tool keywords in the confirmation picture can be "extract tool keywords from the input picture", "extract text information from the input picture and obtain tool keywords from the text information", etc., and the specific method is not limited here.

[0046] In one implementation manner, a specific method for determining a target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed may also be: first extracting text information in the vulnerability reproduction screenshot to be processed; and then detecting whether there is a preset tool keyword in the text information.

[0047] If any tool keyword does not exist in the text information, it is determined that the vulnerability reproduction screenshot to be processed does not have a corresponding target vulnerability reproduction tool.

[0048] If the target tool keyword exists in the text information, the target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed is determined based on the correspondence between the preset tool keyword and the vulnerability reproduction tool.

[0049] Since different types of vulnerability reproduction tools correspond to different tool keywords, the type of the target vulnerability reproduction tool can be determined by determining the tool keyword.

[0050] Optionally, the tool keyword may represent information of different vulnerability reproduction tools, such as the name and number of the vulnerability reproduction tool, or tool set information of different vulnerability reproduction tools, and its specific type is not limited here.

[0051] Vulnerability reproduction tools can generally be divided into two categories, namely, HTTP test type tools and non-Http test type tools. The vulnerability reproduction screenshots generated by HTTP test type tools usually include complete request information and response information. However, the vulnerability reproduction screenshots generated by non-Http test type tools usually do not directly include request information and response information.

[0052] S300: If the target vulnerability reproduction tool is a first type of vulnerability reproduction tool, extract features of the vulnerability reproduction screenshot to be processed based on the large language model.

[0053] Among them, the first type of vulnerability reproduction tool is a non-Http testing type tool.

[0054] In one implementation, based on the large language model, a specific method for extracting features of the screenshot of the vulnerability reproduction to be processed may be: using the screenshot of the vulnerability reproduction to be processed as the input of the large language model, and then extracting feature prompt words from the image based on a preset representation, so that the large language model outputs the features of the screenshot of the vulnerability reproduction to be processed.

[0055] In this case, the large language model is required to have the ability to recognize text information in images.

[0056] The prompt words representing the extraction of features from the image may be “extract key information from the input image”, “extract feature information from the input image”, etc., and the specific implementation method thereof is not limited here.

[0057] In one implementation, if the large language model has the ability to recognize text information in an image, the specific method of extracting the features of the vulnerability reproduction screenshot to be processed based on the large language model may also be: first, the vulnerability reproduction screenshot to be processed is used as the input of the large language model, and then the large language model is controlled to extract the text information in the vulnerability reproduction screenshot to be processed based on the prompt words representing the extracted text information. Then, the prompt words for extracting features from the text information based on the preset representation are used to make the large language model output the features of the vulnerability reproduction screenshot to be processed.

[0058] The prompt words representing the extracted text information are consistent with the aforementioned implementation method, and are not repeated here for the sake of brevity.

[0059] The prompt words representing the extraction of features from text information may be, for example, “extract features from text information”, “extract key information from text information”, etc., and the specific implementation method thereof is not limited here.

[0060] In one implementation, when the target vulnerability reproduction tool is a first type of vulnerability reproduction tool, the specific implementation method of extracting the features of the vulnerability reproduction screenshot to be processed based on the large language model may be: first extract the text content of the vulnerability reproduction screenshot to be processed. Then use the large language model to extract features in the text content to obtain the features of the vulnerability reproduction screenshot to be processed.

[0061] Among them, the method of extracting text content from the screenshot and the method of extracting features in the text content using a large language model have been clearly described in the previous article. For the sake of brief description, they will not be repeated here.

[0062] In one implementation manner, if the target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed cannot be determined, features are extracted from the vulnerability reproduction screenshot to be processed in the manner of S300.

[0063] In one implementation, the features of the pending vulnerability reproduction screenshot include at least one of Http request content, Http response content, command parameters, a uniform resource locator in the pending vulnerability reproduction screenshot, a description of the function of the pending vulnerability reproduction screenshot, a name of a vulnerability reproduction tool, and a type of vulnerability reproduction tool.

[0064] S400: If the target vulnerability reproduction tool is a second type of vulnerability reproduction tool, extract features of the vulnerability reproduction screenshot to be processed based on a pre-trained request response information recognition model and a large language model.

[0065] Among them, the second type of vulnerability reproduction tool is the Http test type tool.

[0066] In one implementation, based on the pre-trained request-response information recognition model and the large language model, the specific method of extracting the features of the vulnerability reproduction screenshot to be processed can be: first, based on the request-response information recognition model, segment and extract the vulnerability reproduction screenshot to be processed to obtain a request information screenshot containing request information and a response information screenshot containing response information. Then, based on the large language model, the request information screenshot and the response information screenshot, the features of the vulnerability reproduction screenshot to be processed are obtained.

[0067] Among them, the request information screenshot and the response information screenshot are both part of the vulnerability reproduction screenshot to be processed.

[0068] Optionally, the specific implementation method of segmenting and extracting the vulnerability reproduction screenshot to be processed based on the request response information recognition model may be: inputting the vulnerability reproduction screenshot to be processed into the request response information recognition model, so that the request response information recognition model processes the vulnerability reproduction screenshot to be processed, and obtains the request information screenshot and response information screenshot output by the request response information recognition model.

[0069] Correspondingly, the training process of the request-response information recognition model is as follows: first, a training data set is obtained, which includes multiple vulnerability reproduction screenshots, and request information screenshots and response information screenshots corresponding to each vulnerability reproduction screenshot. Then, the initial model is trained based on the training data set to obtain a trained request-response information recognition model.

[0070] The specific method and principle of training the model are well known to those skilled in the art and will not be described in detail here for the sake of brevity.

[0071] Alternatively, the specific implementation method of segmenting and extracting the vulnerability reproduction screenshot to be processed based on the request response information recognition model can also be: input the vulnerability reproduction screenshot to be processed into the request response information recognition model, so that the request response information recognition model processes the vulnerability reproduction screenshot to be processed, and obtains the vulnerability reproduction screenshot to be processed marked with the request information range and the response information range. Then, based on the marked request information range and response information range, the vulnerability reproduction screenshot to be processed is segmented to obtain the request information screenshot and the response information screenshot.

[0072] The request information range / response information range may be information including a plurality of coordinates, and the area enclosed by the plurality of coordinates represents the request information range / response information range.

[0073] Alternatively, the request information range / response information range may also include a reference point coordinate and length and width information. In this manner, a rectangle (with a length and width corresponding to the length and width in the request information range / response information range) formed by taking a point corresponding to the reference point coordinate as a reference point (for example, any corner point or center point of a rectangle) may be used as the request information range / response information range.

[0074] Correspondingly, the coordinate system can be a conventional machine vision coordinate system such as an image coordinate system, a world coordinate system, a camera coordinate system, etc. Alternatively, it can be a self-established coordinate system, for example, it can be specified that the upper left corner of the image is the origin of the coordinate system, the direction of the horizontally arranged pixels is the x-axis, and the direction of the vertically arranged pixels is the y-axis. The examples here are only for ease of understanding.

[0075] Correspondingly, the training process of the request-response information recognition model is as follows: first, a training data set is obtained, which includes multiple vulnerability reproduction screenshots, and each vulnerability reproduction screenshot is annotated with a response information range and a request information range. Then, the initial model is trained based on the training data set to obtain a trained request-response information recognition model.

[0076] The response information range and request information range marked corresponding to each vulnerability reproduction screenshot may be manually marked by a marking tool (such as LabelImg, etc.).

[0077] Since the request information range and the response information range are known, the specific method of segmenting the pending vulnerability reproduction screenshot based on the marked request information range and response information range can be: cropping the image of the area corresponding to the request information range in the pending vulnerability reproduction screenshot to obtain the request information screenshot. And cropping the image of the area corresponding to the response information range in the pending vulnerability reproduction screenshot to obtain the response information screenshot.

[0078] Among them, the cropping operation can be implemented by using various screenshot software or programs with screenshot or cropping functions, for example, the operation of segmenting the vulnerability reproduction screenshot to be processed based on the marked request information range and response information range can be implemented by using burp suite, etc. The examples here are only for ease of understanding and should not be used as a limitation to this application.

[0079] In one implementation, based on the large language model, the request information screenshot and the response information screenshot, the specific method of obtaining the features of the vulnerability reproduction screenshot to be processed can be: first, the text in the request information screenshot and the response information screenshot are extracted respectively to obtain the request information text and the response information text. Then, the request information text and the response information text are aggregated to obtain the vulnerability reproduction information text. After that, the large language model is used to extract the features in the vulnerability reproduction information text to obtain the features of the vulnerability reproduction screenshot to be processed.

[0080] The method of extracting text from the request information screenshot and the response information screenshot is the same as the aforementioned method of extracting text information from the pending vulnerability reproduction screenshot, and is not repeated here for the sake of brief description.

[0081] Optionally, using a large language model to extract features in the vulnerability reproduction information text to obtain features of the vulnerability reproduction screenshot to be processed can be: inputting the vulnerability reproduction information text into the large language model, and then extracting feature prompt words from the vulnerability reproduction information text based on a preset representation, so that the large language model outputs the features of the vulnerability reproduction screenshot to be processed.

[0082] The prompt words representing the extraction of features from the vulnerability reproduction information text may be, for example, "extracting features from the vulnerability reproduction information text", "extracting key information from the vulnerability reproduction information text", etc. The specific implementation method is not limited here.

[0083] In one implementation, if the large language model has the ability to recognize text information in an image, the specific method of obtaining the features of the vulnerability reproduction screenshot to be processed based on the large language model, the request information screenshot and the response information screenshot may be: first, the request information screenshot and / or the response information screenshot are input into the large language model. Then, based on the preset representation, the prompt words of the text information are extracted from the request information screenshot and the response information screenshot, so that the large language model extracts the request information text and the response information text. Then, the prompt words that represent the features are extracted from the request information text and the response information text, so that the large language model extracts the features of the vulnerability reproduction screenshot to be processed.

[0084] The prompt words for extracting text information from the request information screenshot and the response information screenshot may be, for example, "extract text information from the input image", "extract text information from the request information screenshot, and extract text information from the response information screenshot", "extract text information from the request information screenshot and the response information screenshot respectively", etc. The specific implementation method is not limited here.

[0085] The prompt words representing the extraction of features from the request information text and the response information text may be, for example, "extract features from all extracted text information", "extract key information from the request information text and the response information text", etc., and the specific implementation method is not limited here.

[0086] In order to facilitate understanding of the training process of the above-mentioned request response information recognition model, an example is given below to illustrate.

[0087] First, you can collect vulnerability reproduction screenshots that are publicly available on the Internet. For example, you can collect relevant technical report articles from the network security technology forum Freebuf, the Github open source repository, and the Security Research Institute's technical blog and extract the vulnerability reproduction screenshots for storage.

[0088] Then, annotate the collected vulnerability reproduction screenshots. You can use the Labelme client to annotate the images. The object marking categories in the image are divided into two types: "request" and "response". Use the Labelme polygon tool to mark the request and response information blocks in the image. Among them, request is used to annotate request information, and response is used to annotate response information.

[0089] After that, convert the JSON annotation file to a YOLO (You Only Look Once, a detection algorithm) compatible annotation file format. Since the annotation files exported by Labelme are generally in JSON format, they need to be converted to a YOLO compatible txt file format. Each YOLO annotation file is a plain text (.txt) file corresponding to the image file name. Its format can be<class_id><center_x><center_y> <width> <height>,<class_id> Indicates the category of the image marker box (i.e. "request" or "response"),<center_x><center_y> represents the x, y coordinates of the reference point of the bounding box, <width> <height>Indicates the height and width of the bounding box. In this way, multiple annotation files corresponding to training images can be generated, and multiple training images and corresponding annotation files can be used as training data sets, and the initial model can be trained using the training data sets. The training images are obtained by collecting the vulnerability reproduction screenshots disclosed on the Internet as mentioned above.

[0090] You can use the yolov8 model training tool to build a request response information block recognition model, for example, select YOLOv8n, YOLOv8m, etc. as the basic training.

[0091] The model is trained through training images and training annotation files, and the weights and parameters of the model are updated to obtain the final recognition model. The optimal weights and parameters are selected as the final model.

[0092] In order to facilitate the understanding of the feature extraction method of the above vulnerability reproduction screenshots, the following will combine Figure 2 Explain it.

[0093] like Figure 2 As shown, first obtain a screenshot of the vulnerability to be processed, and use an OCR tool to extract text information from the screenshot of the vulnerability to be processed. The Chinese and English recognition models of the open source OCR software tesseract can be used to extract text information from the screenshot of the vulnerability to be processed.

[0094] Then identify the vulnerability reproduction tool used in the image. This can be done by determining whether the extracted text information contains tool keywords such as burpsuite and sqlmap. If so, the vulnerability reproduction tool used in the image can be determined based on the tool keyword.

[0095] Or use the prompt word method to send the text information combined with the prompt word to the large language model, so that the large language model can make a judgment on the tool set.

[0096] The following is a my request:

[0097] The following are the OCR extraction results of the image,Please extract the body of the response:{{OCR TEXT}};

[0098] You need to return the results to me in JSON objects JSON objects oftype"UseTool"according to the following JSON schema definitions:{{JSONSCHEMA}};

[0099] Return JSON object with 2spaces of indentation.”

[0100] {{OCR_TEXT}} is the text information extracted above, and {{JSON_SCHEMA}} is the schema protocol of the data class in the required scope. The meaning of this prompt word is to let the large language model analyze the text information and return the name of the vulnerability reproduction tool used. This prompt can return the name information of the vulnerability reproduction tool in JSON format. The returned result uses `{` and `}` as the start and end characters of the extracted content to extract the content, and then converts the result into the Python dict data type or dataclass data type.

[0101] After that, we can judge whether the tool is an Http request testing tool based on the vulnerability reproduction tool we obtained. For example, burpsuite is an Http request testing tool, which generally contains obvious request response information.

[0102] For screenshots of vulnerability reproduction using HTTP test type tools, the request and response information recognition model is used to identify the request information range and response information range in the vulnerability reproduction screenshot to be processed. The image areas corresponding to the request information range and response information range contained in the recognition result are extracted respectively to obtain and save the request information screenshot and response information screenshot.

[0103] Perform OCR recognition on the request information screenshot and the response information screenshot to obtain the request information and response information. Then use the prompt word project (Prompt) to let the large language model return structured request and response information. The prompt word here is similar to the prompt word used in the previous example, which is "The following is a my request:

[0104] The following are the OCR extraction results of the image,Please extract the body of the response:{{OCR TEXT}};

[0105] You need to return the results to me in JSON objects JSON objects oftype"UseTool"according to the following JSON schema definitions:{{JSONSCHEMA}};

[0106] Return JSON object with 2spaces of indentation.”

[0107] The Chinese meaning of the above prompt words is: "Here are my requirements:

[0108] The following is the OCR extraction result of the image. Please extract the response text: {{OCR TEXT}};

[0109] The result you output needs to conform to the JSON Schem definition I provided, and return a JSON object of the "UseTool" type: {{JSON SCHEMA}};

[0110] JSON results are indented with two spaces. ”

[0111] However, the OCR TEXT here is the structured request and response information, and the JSON SCHEMA is the Json schema information of the pre-defined HttpRequest and HttpResponse data models. Among them, the Json schema passed to the big language model needs to be the description information that the big language model can understand.

[0112] For example, the Json schema can be "HttpRequest (Http request object): Http_method (str): extract Http request method, like: GET, POST (extract Http request method, such as: GET, POST); url (str): Http request url (Http request url); headers (dict): Http request headers (Http request header); body: Http request body (Http request body). HttpResponse (Http response object): status_code (str): Http response status code (Http response status code); headers (dict): Httpresponse headers (Http response header); body (str): Http response body (Http response body)". Correspondingly, the large language model will extract the information defined in the Json schema from the input request information and response information, that is, extract the Http request object, Http request method, Http request url, Http request header, Http request body, Http response object, Http response status code, Http response header, and Http response body. The examples here are only for ease of understanding, and actual use is not limited to the examples given here.

[0113] The large language model aggregates and extracts the obtained structured request response information and tool information, and returns structured image key information (i.e., the features of the aforementioned pending vulnerability reproduction screenshots). The fields and field descriptions contained in the structured image key information results may include "Http request content: if possible, you'd better be able to extract the complete Http request content that complies with the specifications; Http response content: if possible, you'd better be able to extract the complete Http response content; command parameters: possible terminal commands in the image; uniform resource locator: if possible, you'd better extract a verifiable URL address; description of the pending vulnerability reproduction screenshot function: if possible, you'd better briefly describe the content of the image; vulnerability reproduction tool name: the name of the vulnerability reproduction tool used in the image."

[0114] For non-Http request testing tools, you can combine the text information of the vulnerability screenshot to be processed and the JSON schema model data of the key information of the structured image through the prompt word project to let the large language model (llm) return structured result data. The prompt words are the same as the above prompt words, but the OCR TEXT here is text information, and the JSON SCHEMA is the Json schema information of the pre-defined HttpRequest and HttpResponse data models.

[0115] At the same time, you can also include the following instructions as system messages to enhance the understanding ability of the large language model. The system message is "You are a security vulnerability image expert. Although you do not have the ability to directly analyze and process;

[0116] images, users will upload the extracted content after OCR processing, so you can assist customers in analysis;

[0117] vulnerability recurrence or verification by analyzing the OCRprocessed content.Screenshot.Generally speaking, the user's OCR;

[0118] results may not be accurate.You'd better optimize based on thecontent.”

[0119] Through System messag, the role of the large language model is limited to a network security vulnerability image analysis expert, and it is informed that text information will be provided for analysis and result optimization.

[0120] Based on the same technical concept, the present application also provides a feature extraction device for vulnerability reproduction screenshots, such as Figure 3 As shown, the feature extraction device 100 of the vulnerability reproduction screenshot includes an acquisition module 110 and a processing module 120.

[0121] The acquisition module 110 is used to obtain a screenshot of the vulnerability to be processed.

[0122] The processing module 120 is used to determine the target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed; if the target vulnerability reproduction tool is a first type of vulnerability reproduction tool, based on the large language model, extract the features of the vulnerability reproduction screenshot to be processed; if the target vulnerability reproduction tool is a second type of vulnerability reproduction tool, based on a pre-trained request response information recognition model and the large language model, extract the features of the vulnerability reproduction screenshot to be processed.

[0123] The processing module 120 is specifically used to extract text information from the vulnerability reproduction screenshot to be processed; use the large language model to extract tool keywords in the text information, and identify the target vulnerability reproduction tool as the first type of vulnerability reproduction tool or the second type of vulnerability reproduction tool based on the tool keywords.

[0124] In one implementation manner, the second type of vulnerability reproduction tool is an Http test type tool, and the first type of vulnerability reproduction tool is a non-Http test type tool.

[0125] The processing module 120 is specifically used to segment and extract the vulnerability reproduction screenshot to be processed based on the request response information recognition model to obtain a request information screenshot containing request information and a response information screenshot containing response information; based on the large language model, the request information screenshot and the response information screenshot, obtain the characteristics of the vulnerability reproduction screenshot to be processed.

[0126] The processing module 120 is specifically used to extract the text in the request information screenshot and the response information screenshot respectively to obtain the request information text and the response information text; use the large language model to extract features in the request information text and the response information text to obtain the features of the vulnerability reproduction screenshot to be processed.

[0127] The processing module 120 is specifically used to extract the text content of the vulnerability reproduction screenshot to be processed; use the large language model to extract features in the text content to obtain features of the vulnerability reproduction screenshot to be processed.

[0128] In one implementation manner, the features of the pending vulnerability reproduction screenshot include at least one of Http request content, Http response content, command parameters, a uniform resource locator in the pending vulnerability reproduction screenshot, a description of the function of the pending vulnerability reproduction screenshot, a vulnerability reproduction tool name, and a vulnerability reproduction tool type.

[0129] The processing module 120 is also used to obtain a training data set, which includes multiple vulnerability reproduction screenshots, and each vulnerability reproduction screenshot is marked with a response information range and a request information range; the initial model is trained based on the training data set to obtain a trained request response information recognition model.

[0130] The feature extraction device 100 for vulnerability reproduction screenshots provided in the embodiment of the present application has the same implementation principle and technical effects as those of the aforementioned feature extraction method embodiment for vulnerability reproduction screenshots. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned feature extraction method embodiment for vulnerability reproduction screenshots.

[0131] See also Figure 4 , which is an electronic device 200 provided in an embodiment of the present application. The electronic device 200 includes: a processor 210 and a memory 220.

[0132] The memory 220 and the processor 210 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory 220 is used to store computer programs, such as storing Figure 3 The software function module shown in is the feature extraction device 100 for vulnerability reproduction screenshots. Among them, the feature extraction device 100 for vulnerability reproduction screenshots includes at least one software function module that can be stored in the memory 220 in the form of software or firmware or solidified in the operating system (OS) of the electronic device 200. The processor 210 is used to execute the executable module stored in the memory 220, such as the software function module or computer program included in the feature extraction device 100 for vulnerability reproduction screenshots. At this time, the processor 210 is used to obtain the vulnerability reproduction screenshot to be processed; determine the target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed; if the target vulnerability reproduction tool is a vulnerability reproduction tool of the first type, extract the features of the vulnerability reproduction screenshot to be processed based on the large language model; if the target vulnerability reproduction tool is a vulnerability reproduction tool of the second type, extract the features of the vulnerability reproduction screenshot to be processed based on the pre-trained request response information recognition model and the large language model.

[0133] Among them, the memory 220 can be, but is not limited to, RAM (Random Access Memory), ROM (Read Only Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0134] The processor 210 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a CPU (Central Processing Unit), an NP (Network Processor), etc.; it may also be a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. A general-purpose processor may be a microprocessor or the processor 210 may also be any conventional processor, etc.

[0135] The electronic device 200 mentioned above includes but is not limited to a personal computer, a server, etc.

[0136] The embodiment of the present application also provides a computer-readable storage medium (hereinafter referred to as storage medium), on which a computer program is stored. When the computer program is run by a computer such as the above-mentioned electronic device 200, the feature extraction method of the vulnerability reproduction screenshot shown above is executed. The computer-readable storage medium includes: a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., which can store program codes.

[0137] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.< / height> < / width> < / height> < / width>

Claims

1. A feature extraction method for vulnerability reproduction screenshots, characterized in that: include: Get screenshots of the vulnerability to be processed; Determine a target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed; If the target vulnerability reproduction tool is a first type of vulnerability reproduction tool, extracting features of the vulnerability reproduction screenshot to be processed based on a large language model; If the target vulnerability reproduction tool is a second type of vulnerability reproduction tool, the features of the vulnerability reproduction screenshot to be processed are extracted based on a pre-trained request response information recognition model and the large language model.

2. The method according to claim 1, characterized in that Determine the target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed, including: Extracting text information from the screenshot of the vulnerability to be processed; The large language model is used to extract tool keywords in the text information, and the target vulnerability reproduction tool is identified as a first type of vulnerability reproduction tool or a second type of vulnerability reproduction tool according to the tool keywords.

3. The method according to claim 1, characterized in that The second type of vulnerability reproduction tool is an Http test type tool, and the first type of vulnerability reproduction tool is a non-Http test type tool.

4. The method according to claim 1, characterized in that Based on the pre-trained request response information recognition model and the large language model, the features of the vulnerability reproduction screenshot to be processed are extracted, including: Segment and extract the vulnerability reproduction screenshot to be processed based on the request response information recognition model to obtain a request information screenshot containing request information and a response information screenshot containing response information; Based on the large language model, the request information screenshot and the response information screenshot, features of the vulnerability reproduction screenshot to be processed are obtained.

5. The method according to claim 4, characterized in that Based on the large language model, the request information screenshot and the response information screenshot, the features of the vulnerability reproduction screenshot to be processed are obtained, including: Extracting text from the request information screenshot and the response information screenshot respectively to obtain a request information text and a response information text; The large language model is used to extract features in the request information text and the response information text to obtain features of the vulnerability reproduction screenshot to be processed.

6. The method according to claim 1, characterized in that The extracting features of the vulnerability reproduction screenshot to be processed based on the large language model includes: Extracting the text content of the screenshot of the vulnerability to be processed; The large language model is used to extract features in the text content to obtain features of the vulnerability reproduction screenshot to be processed.

7. The method according to claim 1, characterized in that The features of the pending vulnerability reproduction screenshot include at least one of Http request content, Http response content, command parameters, a uniform resource locator in the pending vulnerability reproduction screenshot, a description of the function of the pending vulnerability reproduction screenshot, a vulnerability reproduction tool name, and a vulnerability reproduction tool type.

8. The method according to any one of claims 1 to 7, characterized in that: The training process of the request response information recognition model includes: Obtain a training data set, wherein the training data set includes multiple vulnerability reproduction screenshots, and each vulnerability reproduction screenshot is marked with a response information range and a request information range; The initial model is trained based on the training data set to obtain a trained request response information recognition model.

9. A feature extraction device for vulnerability reproduction screenshots, characterized in that: include: The acquisition module is used to obtain screenshots of the vulnerability to be processed; A processing module, used to determine a target vulnerability reproduction tool corresponding to the vulnerability reproduction screenshot to be processed; If the target vulnerability reproduction tool is a first type of vulnerability reproduction tool, the features of the vulnerability reproduction screenshot to be processed are extracted based on a large language model; if the target vulnerability reproduction tool is a second type of vulnerability reproduction tool, the features of the vulnerability reproduction screenshot to be processed are extracted based on a pre-trained request response information recognition model and the large language model.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are connected; The memory is used to store programs; The processor is used to call the program stored in the memory to execute the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 8 is executed.

Images