Data processing method and device based on vulnerability report, medium and electronic equipment

Abstract

The invention discloses a data processing method and device based on a vulnerability report, a medium and electronic equipment. Relates to the field of artificial intelligence, and comprises the following steps: extracting vulnerability data recorded in a vulnerability report, the vulnerability data at least comprising a vulnerability verification script and a vulnerability address; the target language model is used for verifying the vulnerability verification script to obtain a first verification result, the target language model is used for verifying the vulnerability data, and the first verification result is used for indicating whether the vulnerability verification script can run or not; under the condition that the first verification result indicates that the vulnerability verification script can run, the target language model and the vulnerability verification script are utilized to verify the validity of the target vulnerability to obtain a second verification result, and the second verification result is used for indicating whether the target vulnerability is valid in the vulnerability address or not. Through the method and the device, the problem of low checking efficiency of checking the vulnerability report through a vulnerability library manager in the related technology is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to a data processing method, device, medium and electronic device based on vulnerability reports. Background Art

[0002] With the rapid development of information technology, network security issues have become increasingly prominent, and various types of vulnerabilities have been frequently exposed, posing a great threat to system security. Companies and organizations use the method of building a vulnerability library to collect and record vulnerability information. Security researchers submit the discovered security vulnerabilities through the vulnerability library. Vulnerability library managers review the submitted reports and determine the severity of the vulnerabilities, thereby notifying developers to verify and repair the vulnerabilities and improve system security.

[0003] At present, vulnerability libraries are mostly managed through manual review and verification. If there are many vulnerability reports submitted by security researchers, it will take a long time for vulnerability library managers to review (or verify) the vulnerability reports, which is inefficient. At the same time, vulnerability library managers and developers have different security awareness. Therefore, there will be many differences in the understanding and verification of vulnerabilities, resulting in a long vulnerability repair cycle and poor vulnerability repair effect. It is possible that developers think that the vulnerability has been repaired but it has not actually been repaired, which reduces the security of the system and increases the cost of verifying the vulnerability repair situation.

[0004] With regard to the problem of low verification efficiency of vulnerability reports by vulnerability library managers in related technologies, no effective solution has been proposed yet. Summary of the invention

[0005] The main purpose of the present application is to provide a data processing method, device, medium and electronic device based on vulnerability reports to solve the problem of low verification efficiency of vulnerability reports by vulnerability library managers in related technologies.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a data processing method based on vulnerability report is provided. The method comprises: extracting vulnerability data recorded in the vulnerability report, wherein the vulnerability data at least comprises: a vulnerability verification script and a vulnerability address, wherein the vulnerability address is used to record the address of the target vulnerability, and the vulnerability verification script is used to verify whether the target vulnerability exists in the vulnerability address; verifying the vulnerability verification script using a target language model to obtain a first verification result, wherein the target language model is used to verify the vulnerability data, and the first verification result is used to indicate whether the vulnerability verification script can be run; when the first verification result indicates that the vulnerability verification script can be run, verifying the validity of the target vulnerability using the target language model and the vulnerability verification script to obtain a second verification result, wherein the second verification result is used to indicate whether the target vulnerability is valid in the vulnerability address.

[0007] Furthermore, the target language model and the vulnerability verification script are used to verify the validity of the target vulnerability to obtain a second verification result, including: executing the vulnerability verification script, accessing the vulnerability address, and collecting response data returned by the vulnerability address to obtain first response data; inputting the first response data and the vulnerability data into the target language model, verifying the validity of the target vulnerability, and obtaining the second verification result.

[0008] Furthermore, the vulnerability data also includes: expected response data, the expected response data includes: response data that can be generated by accessing the vulnerability address, verifying the validity of the target vulnerability, and obtaining the second verification result, including: verifying whether the first response data and the expected response data are consistent, and obtaining a target verification result; when the target verification result indicates that the first response data is consistent with the expected response data, determining that the second verification result indicates that the target vulnerability is valid in the vulnerability address; when the target verification result indicates that the first response data is inconsistent with the expected response data, determining that the second verification result indicates that the target vulnerability is invalid in the vulnerability address.

[0009] Furthermore, after extracting the vulnerability data recorded in the vulnerability report, it also includes: classifying the vulnerability data based on the data type of the vulnerability data to obtain classified vulnerability data, wherein the data type includes at least one of the following: text, code, and picture; storing the classified vulnerability data in a vulnerability database, wherein the vulnerability database includes: a database for recording data of different vulnerabilities.

[0010] Furthermore, the classified vulnerability data includes at least one of the following: text data, code data and image data, wherein the text data includes at least one of the following: an identifier of the target vulnerability, a type of the target vulnerability and an address of the vulnerability, the code data includes: a code of the vulnerability verification script, and the image data includes: an image file in the vulnerability data. Storing the classified vulnerability data in a vulnerability database includes: converting the text data into a key-value pair to obtain a target key-value pair, and storing the target key-value pair in the vulnerability database; or, storing the code data and the image data in the vulnerability data in the form of files, and recording the file paths where the code data and the image data are stored in the vulnerability database.

[0011] Furthermore, the target language model is also used for vulnerability detection. When the first verification result indicates that the vulnerability verification script can be run, the validity of the target vulnerability is verified using the target language model and the vulnerability verification script. After obtaining the second verification result, it also includes: when the second verification result indicates that the target vulnerability is valid in the vulnerability address, determining the machine to be detected, and generating the vulnerability detection script based on the vulnerability verification script and the target address of the machine to be detected, wherein the machine to be detected includes: a machine to be detected whether the target address has the target vulnerability, and the vulnerability detection script is used to perform vulnerability detection on the machine to be detected; executing the vulnerability detection script, and collecting response data returned by the target address to obtain second response data; based on the second response data and the expected response data, detecting whether the target address of the machine to be detected has the target vulnerability, obtaining a detection result, and storing the detection result in a vulnerability database.

[0012] Furthermore, the target language model is obtained in the following manner: obtaining training samples, wherein the training samples include: vulnerability data of N types of vulnerabilities, N is a positive integer; based on the training samples, performing model training on an initial language model until the initial language model converges to obtain the target language model, wherein the initial language model includes: a language model to be trained, and the loss function used in the process of training the initial language model includes: a cross entropy loss function.

[0013] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a data processing device based on vulnerability report is provided, the device comprising: an extraction unit, used to extract vulnerability data recorded in the vulnerability report, wherein the vulnerability data at least comprises: a vulnerability verification script, a vulnerability address, the vulnerability address is used to record the address of finding the target vulnerability, and the vulnerability verification script is used to verify whether the target vulnerability exists in the vulnerability address; a first verification unit, used to verify the vulnerability verification script using a target language model to obtain a first verification result, wherein the target language model is used to verify the vulnerability data, and the first verification result is used to indicate whether the vulnerability verification script can be run; a second verification unit, used to verify the validity of the target vulnerability using the target language model and the vulnerability verification script when the first verification result indicates that the vulnerability verification script can be run, and obtain a second verification result, wherein the second verification result is used to indicate whether the target vulnerability is valid in the vulnerability address.

[0014] Furthermore, the second verification unit includes: a first processing subunit, used to execute the vulnerability verification script, access the vulnerability address, and collect response data returned by the vulnerability address to obtain first response data; a second processing subunit, used to input the first response data and the vulnerability data into the target language model, verify the validity of the target vulnerability, and obtain the second verification result.

[0015] Furthermore, the vulnerability data also includes: expected response data, the expected response data includes: response data that can be generated by accessing the vulnerability address, and the second processing subunit includes: a verification module, used to verify whether the first response data and the expected response data are consistent, and obtain a target verification result; a first determination module, used to determine that the second verification result indicates that the target vulnerability is valid in the vulnerability address when the target verification result indicates that the first response data is consistent with the expected response data; a second determination module, used to determine that the second verification result indicates that the target vulnerability is invalid in the vulnerability address when the target verification result indicates that the first response data is inconsistent with the expected response data.

[0016] Furthermore, the data processing device based on vulnerability reports also includes: a classification unit, which is used to classify the vulnerability data based on the data type of the vulnerability data after extracting the vulnerability data recorded in the vulnerability report, so as to obtain classified vulnerability data, wherein the data type includes at least one of the following: text, code and picture; a storage unit, which is used to store the classified vulnerability data in a vulnerability database, wherein the vulnerability database includes: a database for recording data of different vulnerabilities.

[0017] Furthermore, the classified vulnerability data includes at least one of the following: text data, code data and image data, wherein the text data includes at least one of the following: an identifier of the target vulnerability, a type of the target vulnerability and an address of the vulnerability, the code data includes: a code of the vulnerability verification script, the image data includes: an image file in the vulnerability data, and the storage unit includes: a conversion subunit for converting the text data into a key-value pair, obtaining a target key-value pair, and storing the target key-value pair in the vulnerability database; or, a storage subunit for storing the code data and the image data in the vulnerability data in the form of files, and recording the file paths where the code data and the image data are stored in the vulnerability database.

[0018] Furthermore, the target language model is also used for vulnerability detection, and the data processing device based on vulnerability reports also includes: a first processing unit, which is used to verify the validity of the target vulnerability using the target language model and the vulnerability verification script when the first verification result indicates that the vulnerability verification script can be run, and after obtaining a second verification result, when the second verification result indicates that the target vulnerability is valid in the vulnerability address, determine the machine to be detected, and generate the vulnerability detection script based on the vulnerability verification script and the target address of the machine to be detected, wherein the machine to be detected includes: a machine to be detected whether the target address has the target vulnerability, and the vulnerability detection script is used to perform vulnerability detection on the machine to be detected; a second processing unit, which is used to execute the vulnerability detection script and collect response data returned by the target address to obtain second response data; a detection unit, which is used to detect whether the target address of the machine to be detected has the target vulnerability based on the second response data and the expected response data, obtain a detection result, and store the detection result in a vulnerability database.

[0019] Furthermore, the target language model is obtained through the following units: an acquisition unit, used to acquire training samples, wherein the training samples include: vulnerability data of N types of vulnerabilities, N is a positive integer; a training unit, used to perform model training on an initial language model based on the training samples until the initial language model converges to obtain the target language model, wherein the initial language model includes: a language model to be trained, and the loss function used in the process of training the initial language model includes: a cross entropy loss function.

[0020] In an embodiment of the present application, vulnerability data recorded in a vulnerability report is extracted, wherein the vulnerability data includes at least: a vulnerability verification script and a vulnerability address, wherein the vulnerability address is used to record the address where a target vulnerability is found, and the vulnerability verification script is used to verify whether the target vulnerability exists at the vulnerability address; the vulnerability verification script is verified using a target language model to obtain a first verification result, wherein the target language model is used to verify the vulnerability data, and the first verification result is used to indicate whether the vulnerability verification script can be run; when the first verification result indicates that the vulnerability verification script can be run, the validity of the target vulnerability is verified using the target language model and the vulnerability verification script to obtain a second verification result, wherein the second verification result is used to indicate whether the target vulnerability is valid in the vulnerability address, thereby solving the technical problem of low verification efficiency of vulnerability reports by vulnerability library managers in the related art.

[0021] In the present invention, the target language model is used to verify the vulnerability data in the vulnerability report, thereby achieving the purpose of automatically verifying the vulnerability report and avoiding the low efficiency of manual review and verification of vulnerability reports in related technologies, thereby achieving the technical effect of improving the verification efficiency of vulnerability reports. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0023] Figure 1 A hardware structure block diagram of a computer terminal for implementing a data processing method based on vulnerability reports is shown;

[0024] Figure 2 is a flow chart of a data processing method based on vulnerability reports provided in an embodiment of the present application;

[0025] Figure 3 is a structural diagram of a data processing system based on vulnerability reporting provided according to an embodiment of the present application;

[0026] Figure 4 is a schematic diagram of a vulnerability processing module provided according to an embodiment of the present application;

[0027] Figure 5 is a schematic diagram of a large language model processing module provided according to an embodiment of the present application;

[0028] Figure 6 It is a flowchart of vulnerability verification provided according to an embodiment of the present application;

[0029] Figure 7 is a flowchart of vulnerability detection provided according to an embodiment of the present application;

[0030] Figure 8 is a schematic diagram of a data processing device based on vulnerability reporting provided according to an embodiment of the present application;

[0031] Fig. 9 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] It should be noted that the collected information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, analysis, vulnerability data, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with relevant laws, regulations and standards, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation entrances are provided for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions to provide users with corresponding operation entrances for users to choose to agree or refuse the results of automated decision-making; if the user chooses to refuse, the expert decision-making process will be entered.

[0035] Example 1

[0036] According to an embodiment of the present application, a method embodiment of a data processing method based on vulnerability reports is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal (or mobile device) for implementing a data processing method based on vulnerability reports is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0038] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0039] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the data processing method based on vulnerability reports in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned data processing method based on vulnerability reports. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0040] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0041] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0042] Under the above operating environment, this application provides Figure 2 The data processing method based on vulnerability reports is shown. Figure 2 is a flow chart of a data processing method based on vulnerability reports provided in an embodiment of the present application, such as Figure 2 As shown, the data processing method includes:

[0043] Step S201, extracting vulnerability data recorded in the vulnerability report, wherein the vulnerability data at least includes: a vulnerability verification script and a vulnerability address, the vulnerability address is used to record the address where the target vulnerability is found, and the vulnerability verification script is used to verify whether the vulnerability address has the target vulnerability.

[0044] The above-mentioned vulnerability report can be a detailed document written by a vulnerability detection system, security researchers or other personnel after detecting a security vulnerability in a system, software or network. The document can contain key information such as a description of the discovered vulnerability, the scope of impact, a severity assessment, a vulnerability verification script, the discovery address of the vulnerability (vulnerability address), reproduction steps, and possible repair suggestions. When extracting the vulnerability data recorded in the vulnerability report, key information directly related to the vulnerability can be extracted from these vulnerability reports for further analysis, verification or automated processing.

[0045] The vulnerability verification script can be an automated tool used to verify whether the target vulnerability in the vulnerability report actually exists. It can include a series of predefined test steps or codes designed to simulate the check whether the target address has the target vulnerability. By running the vulnerability verification script, it can be quickly confirmed whether the target vulnerability in the report actually exists in the target address.

[0046] The vulnerability address mentioned above refers to the specific location of the target vulnerability recorded in the vulnerability report, which can be a URL (Uniform Resource Locator), file path or network port in the target system. It is used to point to the specific page or interface where the vulnerability is found. For example, if the report describes a cross-site scripting (XSS) vulnerability, the vulnerability address may point to a web page URL that can execute malicious scripts. The vulnerability verification script can locate and test specific system parts based on the vulnerability address to verify the existence of the vulnerability.

[0047] In this embodiment, after extracting these key information (for example, vulnerability data), the extracted vulnerability data will be passed to the large language model processing module, which uses the pre-trained large language model to deeply understand and analyze the vulnerability information, determine the security of the vulnerability verification script and the rationality of the verification method (whether it can be run), greatly improve the automation and efficiency of vulnerability management, reduce the burden of manual review, and also ensure the accuracy and timely repair of vulnerabilities.

[0048] Step S202, using the target language model to verify the vulnerability verification script to obtain a first verification result, wherein the target language model is used to verify the vulnerability data, and the first verification result is used to indicate whether the vulnerability verification script can be run.

[0049] The vulnerability verification script extracted from the vulnerability report can exist in the form of code and is used to simulate the triggering conditions of the vulnerability to verify the authenticity of the vulnerability. In this embodiment, the vulnerability verification script can be passed to the target language model in the large language model processing module. The target language model can be a specially trained large language model that can understand and analyze the code structure of a specific programming language. During the model training process, a large number of security-related code samples can be input so that the model can identify security defects, potential malicious behaviors, and normal logical processes in the code.

[0050] In an optional example: the target language model can perform syntax checking on the vulnerability verification script to ensure that the vulnerability verification script has no syntax errors and can be correctly parsed and executed. In addition, the target language model can also analyze the logic of the script to determine whether it is reasonable and whether it can correctly verify the reported vulnerability. The target language model can also perform a security review of the vulnerability verification script to ensure that it does not contain any malicious code or potential attack behaviors, and avoid maliciously written vulnerability verification scripts from causing damage to the target system (i.e., the system to which the vulnerability address belongs), rather than just verifying the existence of the vulnerability.

[0051] The target language model can also check whether there are high-risk operations in the script, such as improperly processed network requests, file system access, system calls, etc. After completing the verification, the target language model can generate a first verification result. This result can be a Boolean value (such as True or False), indicating whether the vulnerability verification script has passed the verification of the target language model, that is, whether it is safe and logically executable. If the model determines that there are security issues or logical errors in the script, if the first verification result is False, it can be determined that the vulnerability verification script cannot be run directly and requires further review or correction (including but not limited to manual review and correction). If the first verification result is True, that is, the target language model believes that the vulnerability verification script is safe and reasonable, then it can be determined that the vulnerability verification script can be run.

[0052] By verifying the vulnerability verification scripts, the security and effectiveness of the vulnerability verification scripts are verified, the running of scripts with potential malicious behaviors is avoided, the automation level of vulnerability management is improved, the need for manual intervention is reduced, and the security and accuracy of the vulnerability management process are guaranteed.

[0053] Step S203, when the first verification result indicates that the vulnerability verification script can be run, the validity of the target vulnerability is verified using the target language model and the vulnerability verification script to obtain a second verification result, wherein the second verification result is used to indicate whether the target vulnerability is valid in the vulnerability address.

[0054] If the first verification result indicates that the vulnerability verification script can be run, for example, the vulnerability verification script has passed the verification in terms of syntax, logic and security and can be run safely, then the validity verification of the vulnerability can continue.

[0055] The vulnerability verification script can be combined with the target vulnerability address, and the vulnerability verification script can be executed in a controlled environment (for example, a virtual machine or a sandbox) to access the vulnerability address, and the response data generated by the vulnerability address can be collected. The response data and the expected response data recorded in the vulnerability report are analyzed and compared to verify whether the target vulnerability is valid in the vulnerability address to obtain a second verification result.

[0056] For example, after executing the vulnerability verification script, the target system's response data to the vulnerability verification script can be collected. The response data can be HTTP response code, response body content, exception information, etc. These data will be used to determine whether the target vulnerability is valid. Specifically, these response data can be compared with the vulnerability behavior described in the vulnerability report to determine whether the actual response meets the expected vulnerability performance to verify whether the target vulnerability is valid in the vulnerability address. For example, if it meets expectations, it can be determined that the target vulnerability is valid in the vulnerability address. If it does not meet expectations, it can be determined that the target vulnerability is invalid in the vulnerability address (for example, the target vulnerability does not exist or the target vulnerability has been repaired). If it is invalid, it can also be transferred to manual processing.

[0057] Combining the intelligent analysis of the large language model with the actual execution of the vulnerability verification script ensures the accuracy and efficiency of vulnerability verification. The large language model can not only review the security of the script, but also judge the effectiveness of the vulnerability by dynamically executing the script and analyzing the response, thereby providing strong automated support for vulnerability management, reducing manual intervention, shortening the vulnerability management cycle, and improving the security of the system.

[0058] Through the above steps, in this embodiment, the target language model is used to verify the vulnerability data in the vulnerability report, so as to achieve the purpose of automatically verifying the vulnerability report, avoid the low efficiency of manual review and verification of vulnerability reports in the related art, and thus achieve the technical effect of improving the verification efficiency of vulnerability reports. This solves the technical problem of low verification efficiency of vulnerability reports by vulnerability library managers in the related art.

[0059] Optionally, in the data processing method based on vulnerability report provided in the embodiment of the present application, the validity of the target vulnerability is verified using the target language model and the vulnerability verification script to obtain a second verification result, including: executing the vulnerability verification script, accessing the vulnerability address, and collecting response data returned by the vulnerability address to obtain first response data; inputting the first response data and the vulnerability data into the target language model, verifying the validity of the target vulnerability, and obtaining a second verification result.

[0060] For example, after determining that the vulnerability verification script can be run (for example, the code is compliant and there is no malicious behavior), the vulnerability verification script can be run in the virtual machine and the vulnerability address can be passed in to obtain the response of the vulnerability address. Finally, the response of the vulnerability address (corresponding to the first response data) and other information of the target vulnerability (for example, the expected response data of the target vulnerability) can be passed to the target language model for processing for verification, and it is determined whether the response of the vulnerability target address is consistent with other information descriptions of the vulnerability (that is, whether the first response data is consistent with the expected response data), thereby verifying whether the vulnerability exists and is valid, and obtaining a second verification result, and it can also update the result to the vulnerability database.

[0061] By combining vulnerability verification scripts with large language models to automatically verify the vulnerability data in vulnerability reports, the low efficiency of manual verification of vulnerability data in vulnerability reports in related technologies is avoided, thereby achieving the technical effect of improving the verification efficiency of vulnerability reports.

[0062] Optionally, in the data processing method based on vulnerability report provided in the embodiment of the present application, the vulnerability data also includes: expected response data, the expected response data includes: response data that can be generated by accessing the vulnerability address, and the validity of the target vulnerability is verified to obtain a second verification result, including: verifying whether the first response data and the expected response data are consistent, and obtaining a target verification result; when the target verification result indicates that the first response data is consistent with the expected response data, determining that the second verification result indicates that the target vulnerability is valid in the vulnerability address; when the target verification result indicates that the first response data is inconsistent with the expected response data, determining that the second verification result indicates that the target vulnerability is invalid in the vulnerability address.

[0063] The expected response data includes: the response data that is expected to be generated by accessing the vulnerability address. For example, after obtaining the first response data, the first response data can be compared with the expected response data. Specifically, the target language model (for example, the large language model in the large language model processing module) is used for comparative analysis. The target language model can analyze all features in the first response data, including but not limited to HTTP (Hypertext Transfer Protocol) status code, response content, response time, etc., and compare them with the ideal response after the vulnerability is triggered as described in the expected response data to determine whether the first response data is consistent with the expected response data. If the two are consistent, that is, the first response data presents the expected vulnerability behavior or characteristics, then a target verification result can be generated, indicating that the comparison result is consistent, which means that the vulnerability verification script successfully reproduces the vulnerability behavior in the report.

[0064] If the target verification result indicates that the first response data is consistent with the expected response data, then the second verification result will confirm that the target vulnerability is valid in the vulnerability address. Conversely, if the target verification result indicates that the first response data is inconsistent with the expected response data, the system will obtain a second verification result, indicating that the target vulnerability is invalid in the vulnerability address and may have been fixed or reported as a false positive.

[0065] The above second verification result can be recorded in the vulnerability database to update the status of the target vulnerability. At the same time, this result will also be fed back to relevant teams, such as the security team and the development team, so that they can take corresponding measures based on the verification results, whether it is vulnerability repair, further investigation or elimination of false positives.

[0066] By comparing the first response data with the expected response data, the effectiveness of the vulnerability can be accurately verified, false positives and negatives can be reduced, and the overall efficiency and accuracy of vulnerability management can be improved. The large language model plays a key role in this process. It can not only understand the complex expected response data description, but also perform accurate response data comparison, improving the ability of automated vulnerability verification.

[0067] Optionally, in the data processing method based on vulnerability reports provided in the embodiment of the present application, after extracting the vulnerability data recorded in the vulnerability report, it also includes: classifying the vulnerability data based on the data type of the vulnerability data to obtain classified vulnerability data, wherein the data type includes at least one of the following: text, code, and picture; storing the classified vulnerability data in a vulnerability database, wherein the vulnerability database includes: a database for recording data of different vulnerabilities.

[0068] In this embodiment, the vulnerability data can be classified into three categories: text, code and image. The text data may include but is not limited to: the name, type, target address and other information of the vulnerability, the code data may include but is not limited to the vulnerability verification script code, and the image data may include but is not limited to: image files in the vulnerability description, vulnerability proof, repair suggestions and other data. Then, the classified vulnerability data can also be stored in the vulnerability database.

[0069] In this embodiment, when a new vulnerability report is received, the content of the vulnerability report can be parsed to identify key information related to the vulnerability. This information may exist in different data types, including text descriptions, code snippets, and image evidence, etc., and then the parsed vulnerability data can be classified according to the data type to obtain classified vulnerability data, and the classified vulnerability data is stored in the vulnerability database.

[0070] Storing vulnerability data in vulnerability data in a classified storage manner can facilitate subsequent query of the vulnerability data.

[0071] Optionally, in the data processing method based on vulnerability reports provided in the embodiment of the present application, the classified vulnerability data includes at least one of the following: text data, code data and image data, wherein the text data includes at least one of the following: an identifier of the target vulnerability, a type of the target vulnerability and a vulnerability address, the code data includes: the code of the vulnerability verification script, the image data includes: an image file in the vulnerability data, and storing the classified vulnerability data in a vulnerability database includes: converting the text data into a key-value pair to obtain a target key-value pair, and storing the target key-value pair in the vulnerability database; or, storing the code data and the image data in the vulnerability data in the form of files, and recording the file paths where the code data and the image data are stored in the vulnerability database.

[0072] The above-mentioned text data may include but is not limited to: vulnerability description, vulnerability type, vulnerability level, affected system or version and other data; the above-mentioned code data may include but is not limited to: vulnerability verification scripts, attack codes or examples for exploiting vulnerabilities, and code snippets for fixing vulnerabilities, etc.; the above-mentioned image data may include but is not limited to: images of vulnerability descriptions, screenshots when the vulnerability is triggered, visualizations of network traffic, etc., to intuitively demonstrate the manifestation of the vulnerability or prove the existence of the vulnerability.

[0073] The above text data can be directly stored in the form of key-value pairs, such as using JSON format, where each key represents an attribute of the vulnerability data, such as "ID", "Description", "Type", etc. Since the code may be long and contain complex logic, the code data can be saved to the vulnerability database, and the path of the file can be stored in the database for subsequent analysis and reference. Similarly, image files can also be directly saved to the vulnerability data, and the database records the paths of these image files, which improves the retrieval speed of vulnerability data.

[0074] Optionally, in the data processing method based on vulnerability report provided in the embodiment of the present application, the target language model is also used for vulnerability detection. When the first verification result indicates that the vulnerability verification script can run, the validity of the target vulnerability is verified using the target language model and the vulnerability verification script. After obtaining the second verification result, it also includes: when the second verification result indicates that the target vulnerability is valid in the vulnerability address, determining the machine to be detected, and generating a vulnerability detection script based on the vulnerability verification script and the target address of the machine to be detected, wherein the machine to be detected includes: a machine whether there is a target vulnerability at the target address to be detected, and the vulnerability detection script is used to perform vulnerability detection on the machine to be detected; executing the vulnerability detection script, and collecting response data returned by the target address to obtain second response data; based on the second response data and the expected response data, detecting whether there is a target vulnerability at the target address of the machine to be detected, obtaining a detection result, and storing the detection result in a vulnerability database.

[0075] After the second verification result confirms that the target vulnerability is valid, a list of machines to be detected (used to record the machines to be detected) can be received to determine the machines to be detected. The types of machines to be detected may include but are not limited to: servers, database systems, network equipment, and any other hardware or software systems that may be affected by the vulnerability.

[0076] In order to detect whether the target vulnerability exists on these machines to be detected, a special vulnerability detection script can be generated based on the verified vulnerability verification script. The generation of these scripts may be completed by the large language model processing module. The model automatically adjusts the script to adapt to different target addresses and system environments based on the characteristics of the target vulnerability and the verification method. The generated vulnerability detection script should avoid causing actual damage to the operating environment and is only used to check the existence of vulnerabilities. Then, the generated vulnerability detection script can be executed on the machines to be detected sequentially or in parallel, and a request can be sent to each target address to check whether the target vulnerability exists at the target address of the machine to be detected. During the execution of the script, the response data returned by the machine to be detected (i.e., the second response data) can also be collected.

[0077] After collecting the second response data, the second response data can be compared with the expected response data in the vulnerability data. This comparison process can be completed by the target language model to determine whether the second response data is consistent with the expected response data. If they are consistent, it can be determined that the target vulnerability exists at the target address of the machine to be detected; if they are inconsistent, it can be determined that the target vulnerability does not exist on the machine to be detected or has been repaired, and the detection results are obtained. The detection results can be recorded in the vulnerability database. For each machine to be detected, its vulnerability status in the database can be updated, including whether it is affected by the target vulnerability, the severity of the vulnerability, whether it has been repaired, and other information.

[0078] By automatically detecting whether there are other machines in the enterprise network that are affected by the verified vulnerabilities, timely remedial measures can be taken to prevent the vulnerabilities from being maliciously exploited and protect the security of the network and system.

[0079] Optionally, in the data processing method based on vulnerability reports provided in an embodiment of the present application, the target language model is obtained in the following manner: obtaining training samples, wherein the training samples include: vulnerability data of N types of vulnerabilities, N is a positive integer; based on the training samples, performing model training on the initial language model until the initial language model converges to obtain the target language model, wherein the initial language model includes: a language model to be trained, and the loss function used in the process of training the initial language model includes: a cross entropy loss function.

[0080] For example, a large language model can be trained to obtain a target language model. During the model training process, a cross entropy loss function can be used for training. The cross entropy loss function formula Loss is:

[0081]

[0082] Where N is the number of training samples, V is the size of the vocabulary, and y i,j is the unique hot encoding of the jth index of the i-th word in the vocabulary of the training sample, representing the true label, (1≤i≤N), (1≤j≤V); is the model's predicted probability of the jth word in the vocabulary for the i-th sample.

[0083] In this embodiment, the existing large language model can be trained by collecting security blogs, security articles, CVE (Common Vulnerabilities and Exposures) information, vulnerability information and other security-related information on the Internet, and the performance and effect of the model can be optimized by continuously adjusting the hyperparameters, learning rate, etc., and finally a large language model (target language model) specifically targeting security vulnerabilities is obtained. At the same time, the large language model is updated and optimized every time new vulnerability information is collected to ensure that the large language model can be iteratively updated in a timely manner.

[0084] For example, data containing N different types of vulnerabilities can be obtained from multiple sources, such as CVE databases, security research papers, technical forums, open source security projects, etc. Ensure that the data covers multiple vulnerability types and that each vulnerability type can have a corresponding description, verification method, and repair suggestion. Clean the collected raw vulnerability data to remove irrelevant information, erroneous entries, or duplicate data. Convert text data into a format suitable for model input, and perform preprocessing such as word segmentation and encoding so that the model can understand and learn.

[0085] The processed data set (corresponding to the training samples) is divided into a training set and a test set, usually in a certain ratio (such as 80% training and 20% test) to ensure that the model can be effectively verified on unknown data and evaluate its generalization ability.

[0086] In this embodiment, a pre-trained large language model can be selected as the initial language model, and the structural parameters of the model, such as the number of layers, the number of hidden units, the learning rate, etc., can also be adjusted according to the specific vulnerability management task. During the model training process, the cross entropy loss function calculates the difference between the category probability distribution predicted by the model and the unique hot vector of the actual sample label. For each training sample, the model will output an N-dimensional vector representing the predicted probability of N types of vulnerability types. The loss function will measure the difference between this predicted vector and the actual label. It can also calculate the gradient of the loss function to the model parameters, and use the optimization algorithm to perform gradient descent to adjust the model parameters to minimize the loss function. Through multiple rounds of iterations, the model can gradually learn how to more accurately predict and understand vulnerability information. During the training process, the value of the loss function is continuously monitored. If the value of the loss function no longer decreases significantly in multiple consecutive training cycles, it can be considered that the model has converged. The model can also be verified using the previously divided test set data to evaluate the model's accuracy, recall rate and other indicators to ensure that the model not only performs well on the training set, but also maintains high performance on unseen data.

[0087] Once the model training is completed and verified by the test set, the model converges and the performance is stable. The model at this time is the target language model. This model can be saved and deployed in the vulnerability library management system to process vulnerability reports, verify vulnerability information, generate vulnerability repair suggestions, and other tasks.

[0088] It should be noted that as new vulnerabilities are discovered and the security field is updated, the target language model can be regularly fine-tuned using new vulnerability data to keep the model up-to-date and accurate.

[0089] The data processing method based on vulnerability reports provided in the embodiment of the present application uses the target language model to verify the vulnerability data in the vulnerability report, thereby achieving the purpose of automatically verifying the vulnerability report and avoiding the low efficiency of manual review and verification of vulnerability reports in the related art, thereby achieving the technical effect of improving the verification efficiency of vulnerability reports. This solves the technical problem of low verification efficiency of vulnerability reports by vulnerability library managers in the related art.

[0090] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0091] Example 2

[0092] This embodiment provides an optional vulnerability report-based data processing system, which can be used to execute the vulnerability report-based data processing method provided in Implementation 1.

[0093] Figure 3 is a structural diagram of a data processing system based on vulnerability reports provided in an embodiment of the present application. Figure 3 As shown, the vulnerability processing module 301 and the large language model processing module 302 together constitute the vulnerability library vulnerability management system based on the large language model proposed by the present invention. The vulnerability processing module 301 and the large language model processing module 302 can be deployed together in the same local environment. Figure 3 This paper introduces how to deploy the vulnerability report processing system in the Internet environment to achieve vulnerability database data sharing and collaborative construction of large language models. The following is an explanation of each module.

[0094] 1. Vulnerability processing module 301:

[0095] Figure 4 is a schematic diagram of a vulnerability processing module provided according to an embodiment of the present application, such as Figure 4 As shown. The vulnerability processing module 301 is mainly composed of three parts: a vulnerability submission unit 401, a vulnerability verification unit 402, and a vulnerability detection unit 403. The vulnerability processing module can be responsible for extracting vulnerability information from vulnerability reports, submitting the vulnerability information (or vulnerability data) to the large language model processing module to obtain a vulnerability verification and detection method, and using the method to perform vulnerability verification and detection.

[0096] (1) The vulnerability submission unit 401 can be used to extract vulnerability information from the vulnerability report and store the vulnerability information (corresponding to the vulnerability data) in the vulnerability database. For the received vulnerability report, the vulnerability submission unit 401 can classify the vulnerability data into three categories: text, code and image. The text data mainly includes the name, type, target address and other information of the vulnerability, the code data mainly includes the vulnerability verification script code, and the image data mainly includes the image file in the vulnerability description, vulnerability proof, repair suggestion and other information. The key-value pair of the database is directly stored for the text data; the overall file content is first saved for the code and image data, and then the corresponding file path is stored in the database. At the same time, the vulnerability submission unit can serialize and output the stored vulnerability information, and store it in the log file for future auditing. The serialized vulnerability information is in JSON format, such as: {ID:"11234",Type:"XSS","URL":"http: / / 1.1.1.2 / index.html","POC_PATH":" / data / poc / 11234 / poc.py"…"Time":"2024-04-10"}.

[0097] (2) The vulnerability verification unit 402 can be used to verify the extracted vulnerability information, determine whether the vulnerability actually exists, and provide corresponding repair suggestions. The vulnerability verification unit 402 can interact with the large language model processing module 302.

[0098] After obtaining the vulnerability information from the vulnerability database, the vulnerability verification unit 402 can pass the vulnerability verification script to the large language model processing module 302 for verification, determine whether the verification script is compliant and has no malicious behavior, and run the vulnerability verification script in the virtual machine after determining that there is no malicious behavior and pass in the vulnerability target address (corresponding to the vulnerability address in the first embodiment) to obtain the response of the vulnerability target address. Finally, the response of the vulnerability target address (corresponding to the first vulnerability data in the first embodiment) and other information of the vulnerability are passed to the large language model processing module 302 for verification, and determine whether the response of the vulnerability target address is consistent with other information descriptions of the vulnerability, thereby verifying whether the vulnerability exists and is valid, and updating the result to the vulnerability database.

[0099] (3) The vulnerability detection unit 403 can be used to detect confirmed vulnerabilities (verified vulnerability data) and determine whether the vulnerability exists in other machines. The vulnerability detection unit 203 can interact with the large language model processing module 302. After obtaining the vulnerability information from the vulnerability database, the vulnerability detection unit 403 can pass the verification script of the confirmed vulnerability and the target address to be detected to the large language model processing module 302 to generate the vulnerability detection script, and then run the vulnerability detection script in the virtual machine and obtain the corresponding response information. Finally, the response information and other information about the vulnerability are passed to the large language model processing module 302 for verification, to determine whether the vulnerability exists in other machines, and to update the result to the vulnerability database.

[0100] 2. Large language model processing module 302:

[0101] Figure 5 is a schematic diagram of a large language model processing module provided according to an embodiment of the present application, such as Figure 5 As shown, the large language model processing module mainly includes two parts: a model training unit 501 and a model question-answering unit 502.

[0102] (1) The model training unit 501 can be used to train the large language model. The model training unit 501 can use the cross entropy loss function for training, and the specific formula is:

[0103]

[0104] Where N is the number of training samples, V is the size of the vocabulary, and y i,j is the unique hot encoding of the jth index of the i-th word in the vocabulary of the training sample, representing the true label, (1≤i≤N), (1≤j≤V); is the model's predicted probability of the jth word in the vocabulary for the i-th sample.

[0105] By collecting security blogs, security articles, CVE information, vulnerability information and other security-related information on the Internet, we train the existing large language model, and continuously adjust the hyperparameters, learning rate, etc. to optimize the performance and effect of the model, and finally obtain a large language model specifically for security vulnerabilities. At the same time, we update and optimize the large language model every time new vulnerability information is collected to ensure that the large language model can be iterated and updated in a timely manner.

[0106] (2) The model question-answering unit 502 can be used to interact with the vulnerability processing module 301 to answer questions raised by the vulnerability processing module 101. Before each question-answering session, the model question-answering unit 502 can generate specific prefix words and send them to the large language model to guide the large language model to a special question-answering scenario, so as to answer more professional and format-compliant sentences.

[0107] 3. Operation process of vulnerability management system based on large language model:

[0108] Figure 6 is a vulnerability verification flow chart provided according to an embodiment of the present application, such as Figure 6 As shown, the specific process description includes:

[0109] Step S601, start the vulnerability verification process;

[0110] Step S602: parsing the submitted vulnerability report and extracting vulnerability information;

[0111] Step S603: the vulnerability processing module 301 transmits the vulnerability information to the large language model processing module 302 to determine whether there is malicious behavior in the vulnerability information. If so, the process ends; if not, the process continues;

[0112] Step S604: the vulnerability processing module 301 runs the vulnerability verification script to obtain the target's response;

[0113] Step S605: the vulnerability processing module 301 passes the response information and the vulnerability information to the large language model processing module 302, and determines whether the response information is consistent with the expected vulnerability response. If yes, the process continues; if not, the process ends.

[0114] Step S606: Update the vulnerability database and store vulnerability information.

[0115] Figure 7 is a flowchart of vulnerability detection provided according to an embodiment of the present application, such as Figure 7 As shown, including;

[0116] Step S701: Read vulnerability information from the vulnerability database;

[0117] Step S702: the vulnerability processing module 301 transmits the vulnerability information and the target address to be detected to the large language model processing module 302 to generate a vulnerability verification script;

[0118] Step S703: the vulnerability processing module 301 runs the vulnerability verification script to obtain the target's response;

[0119] Step S704: the vulnerability processing module 301 transmits the response information and the vulnerability information to the large language model processing module 302 to determine whether the response information is consistent with the expected vulnerability response;

[0120] Step S705, if yes, then the detection target has a vulnerability;

[0121] Step S706, if not, the detection target does not have a vulnerability;

[0122] Step S707: Record the vulnerability detection result.

[0123] In this embodiment, the cost of manual review and verification can be saved while ensuring the normal implementation of the vulnerability management cycle, helping enterprises to better manage vulnerabilities. For example, by using a large language model to optimize the steps of vulnerability report checking, vulnerability verification, and vulnerability detection, manual time can be saved, the vulnerability management cycle can be optimized, the difficulty of enterprise vulnerability management can be reduced, the basic capabilities of enterprise vulnerability management can be improved, and enterprises can be helped to discover and repair more security vulnerabilities.

[0124] Example 3

[0125] The embodiment of the present application also provides a data processing device based on vulnerability reports. It should be noted that the data processing device based on vulnerability reports in the embodiment of the present application can be used to execute the data processing method based on vulnerability reports provided in the embodiment of the present application. The data processing device based on vulnerability reports provided in the embodiment of the present application is introduced below.

[0126] According to an embodiment of the present application, a device for implementing the above-mentioned data processing method based on vulnerability reports is also provided, such as Figure 8 As shown, the device includes: an extraction unit 81, a first verification unit 82 and a second verification unit 83.

[0127] The extraction unit 81 is used to extract vulnerability data recorded in the vulnerability report, wherein the vulnerability data at least includes: a vulnerability verification script and a vulnerability address, the vulnerability address is used to record the address where the target vulnerability is found, and the vulnerability verification script is used to verify whether the vulnerability address contains the target vulnerability;

[0128] A first verification unit 82, used to verify the vulnerability verification script using a target language model to obtain a first verification result, wherein the target language model is used to verify the vulnerability data, and the first verification result is used to indicate whether the vulnerability verification script can be run;

[0129] The second verification unit 83 is used to verify the validity of the target vulnerability using the target language model and the vulnerability verification script when the first verification result indicates that the vulnerability verification script can be run, and obtain a second verification result, wherein the second verification result is used to indicate whether the target vulnerability is valid in the vulnerability address.

[0130] According to another aspect of the present application, a data processing device based on vulnerability reports is provided, the device comprising: an extraction unit 81, used to extract vulnerability data recorded in the vulnerability report, wherein the vulnerability data at least comprises: a vulnerability verification script, a vulnerability address, the vulnerability address is used to record the address where the target vulnerability is found, and the vulnerability verification script is used to verify whether the target vulnerability exists in the vulnerability address; a first verification unit 82, used to verify the vulnerability verification script using a target language model to obtain a first verification result, wherein the target language model is used to verify the vulnerability data, and the first verification result is used to indicate whether the vulnerability verification script can be run; a second verification unit 83, used to verify the validity of the target vulnerability using the target language model and the vulnerability verification script when the first verification result indicates that the vulnerability verification script can be run, and obtain a second verification result, wherein the second verification result is used to indicate whether the target vulnerability is valid in the vulnerability address.

[0131] The data processing device based on vulnerability reports provided in the embodiment of the present application uses the target language model to verify the vulnerability data in the vulnerability report, thereby achieving the purpose of automatically verifying the vulnerability report, thereby achieving the technical effect of improving the verification efficiency of the vulnerability report. This solves the technical problem of low verification efficiency of the vulnerability report by the vulnerability library manager in the related art.

[0132] Optionally, in the data processing device based on vulnerability reporting provided in an embodiment of the present application, the second verification unit includes: a first processing sub-unit, used to execute a vulnerability verification script, access the vulnerability address, and collect response data returned by the vulnerability address to obtain first response data; a second processing sub-unit, used to input the first response data and the vulnerability data into a target language model, verify the validity of the target vulnerability, and obtain a second verification result.

[0133] Optionally, in the data processing device based on vulnerability report provided in the embodiment of the present application, the vulnerability data also includes: expected response data, the expected response data includes: response data that can be generated by accessing the vulnerability address, and the second processing subunit includes: a verification module, used to verify whether the first response data and the expected response data are consistent to obtain a target verification result; a first determination module, used to determine that the second verification result indicates that the target vulnerability is valid in the vulnerability address when the target verification result indicates that the first response data is consistent with the expected response data; a second determination module, used to determine that the second verification result indicates that the target vulnerability is invalid in the vulnerability address when the target verification result indicates that the first response data is inconsistent with the expected response data.

[0134] Optionally, in the data processing device based on vulnerability reports provided in the embodiment of the present application, the data processing device based on vulnerability reports also includes: a classification unit, which is used to classify the vulnerability data based on the data type of the vulnerability data after extracting the vulnerability data recorded in the vulnerability report, so as to obtain classified vulnerability data, wherein the data type includes at least one of the following: text, code, and picture; a storage unit, which is used to store the classified vulnerability data in a vulnerability database, wherein the vulnerability database includes: a database for recording data of different vulnerabilities.

[0135] Optionally, in the data processing device based on vulnerability reports provided in the embodiment of the present application, the classified vulnerability data includes at least one of the following: text data, code data and image data, wherein the text data includes at least one of the following: an identifier of the target vulnerability, a type of the target vulnerability and a vulnerability address, the code data includes: the code of the vulnerability verification script, the image data includes: an image file in the vulnerability data, and the storage unit includes: a conversion subunit for converting the text data into a key-value pair, obtaining a target key-value pair, and storing the target key-value pair in a vulnerability database; or, a storage subunit for storing the code data and the image data in the vulnerability data in the form of files, and recording the file paths where the code data and the image data are stored in the vulnerability database.

[0136] Optionally, in the data processing device based on vulnerability reports provided in the embodiment of the present application, the target language model is also used for vulnerability detection, and the data processing device based on vulnerability reports also includes: a first processing unit, which is used to verify the validity of the target vulnerability using the target language model and the vulnerability verification script when the first verification result indicates that the vulnerability verification script can be run, and after obtaining the second verification result, when the second verification result indicates that the target vulnerability is valid in the vulnerability address, determine the machine to be detected, and generate a vulnerability detection script based on the vulnerability verification script and the target address of the machine to be detected, wherein the machine to be detected includes: a machine whether there is a target vulnerability at the target address to be detected, and the vulnerability detection script is used to perform vulnerability detection on the machine to be detected; a second processing unit, which is used to execute the vulnerability detection script and collect response data returned by the target address to obtain second response data; a detection unit, which is used to detect whether there is a target vulnerability at the target address of the machine to be detected based on the second response data and the expected response data, obtain a detection result, and store the detection result in a vulnerability database.

[0137] Optionally, in the data processing device based on vulnerability reports provided in an embodiment of the present application, the target language model is obtained through the following units: an acquisition unit, used to acquire training samples, wherein the training samples include: vulnerability data of N types of vulnerabilities, N is a positive integer; a training unit, used to perform model training on the initial language model based on the training samples until the initial language model converges to obtain the target language model, wherein the initial language model includes: a language model to be trained, and the loss function used in the process of training the initial language model includes: a cross entropy loss function.

[0138] It should be noted that the above-mentioned extraction unit 81, first verification unit 82 and second verification unit 83 correspond to steps S201 to S203 in Example 1, and the two modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules or units may be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n), and the above-mentioned modules may also be part of the device and may be run in the computer terminal 10 provided in Example 1.

[0139] Example 4

[0140] An embodiment of the present application may provide an electronic device, Fig. 9 is a structural block diagram of an electronic device according to an embodiment of the present application. Fig. 9 As shown, the electronic device may include: one or more ( Fig. 9(only one is shown) processor 902, memory 904, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0141] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0142] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: extract the vulnerability data recorded in the vulnerability report, wherein the vulnerability data at least includes: a vulnerability verification script, a vulnerability address, the vulnerability address is used to record the address of the target vulnerability, and the vulnerability verification script is used to verify whether the target vulnerability exists in the vulnerability address; use the target language model to verify the vulnerability verification script to obtain a first verification result, wherein the target language model is used to verify the vulnerability data, and the first verification result is used to indicate whether the vulnerability verification script can be run; when the first verification result indicates that the vulnerability verification script can be run, use the target language model and the vulnerability verification script to verify the validity of the target vulnerability to obtain a second verification result, wherein the second verification result is used to indicate whether the target vulnerability is valid in the vulnerability address.

[0143] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: execute the vulnerability verification script, access the vulnerability address, and collect the response data returned by the vulnerability address to obtain the first response data; input the first response data and the vulnerability data into the target language model, verify the validity of the target vulnerability, and obtain the second verification result.

[0144] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: verify whether the first response data and the expected response data are consistent to obtain a target verification result; when the target verification result indicates that the first response data is consistent with the expected response data, determine that the second verification result indicates that the target vulnerability is valid in the vulnerability address; when the target verification result indicates that the first response data is inconsistent with the expected response data, determine that the second verification result indicates that the target vulnerability is invalid in the vulnerability address.

[0145] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: after extracting the vulnerability data recorded in the vulnerability report, classify the vulnerability data based on the data type of the vulnerability data to obtain classified vulnerability data, wherein the data type includes at least one of the following: text, code, and picture; store the classified vulnerability data in a vulnerability database, wherein the vulnerability database includes: a database for recording data of different vulnerabilities.

[0146] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: convert the text data into a key-value pair, obtain the target key-value pair, and store the target key-value pair in the vulnerability database; or, store the code data and image data in the vulnerability data in the form of a file, and record the file path where the code data and image data are stored in the vulnerability database.

[0147] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: when the first verification result indicates that the vulnerability verification script can be run, use the target language model and the vulnerability verification script to verify the validity of the target vulnerability, and after obtaining the second verification result, it also includes: when the second verification result indicates that the target vulnerability is valid in the vulnerability address, determine the machine to be detected, and generate a vulnerability detection script based on the vulnerability verification script and the target address of the machine to be detected, wherein the machine to be detected includes: a machine whether there is a target vulnerability at the target address to be detected, and the vulnerability detection script is used to perform vulnerability detection on the machine to be detected; execute the vulnerability detection script, and collect response data returned by the target address to obtain second response data; based on the second response data and the expected response data, detect whether there is a target vulnerability at the target address of the machine to be detected, obtain a detection result, and store the detection result in a vulnerability database.

[0148] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: obtain training samples, wherein the training samples include: vulnerability data of N types of vulnerabilities, N is a positive integer; based on the training samples, perform model training on the initial language model until the initial language model converges to obtain the target language model, wherein the initial language model includes: the language model to be trained, and the loss function used in the process of training the initial language model includes: the cross entropy loss function.

[0149] By adopting the embodiment of the present application, a solution of using a target language model to verify the vulnerability data in the vulnerability report is adopted, so as to achieve the purpose of automatically verifying the vulnerability report, avoid the low efficiency of manual review and verification of the vulnerability report in the related art, and thus achieve the technical effect of improving the verification efficiency of the vulnerability report. In addition, the technical problem of low verification efficiency of the vulnerability report by the vulnerability library manager in the related art is solved.

[0150] It can be understood by those skilled in the art that Fig. 9 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Fig. 9 The structure of the electronic device is not limited. Fig. 9 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Fig. 9 Different configurations are shown.

[0151] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0152] Example 5

[0153] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the data processing method based on vulnerability report provided in the first embodiment.

[0154] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0155] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing the program steps of the data processing method based on the vulnerability report.

[0156] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0157] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0158] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0159] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0160] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0161] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0162] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data processing method based on vulnerability reports, characterized in that: include: Extracting vulnerability data recorded in the vulnerability report, wherein the vulnerability data at least includes: a vulnerability verification script and a vulnerability address, wherein the vulnerability address is used to record the address where the target vulnerability is found, and the vulnerability verification script is used to verify whether the target vulnerability exists in the vulnerability address; Verifying the vulnerability verification script using a target language model to obtain a first verification result, wherein the target language model is used to verify the vulnerability data, and the first verification result is used to indicate whether the vulnerability verification script can be run; When the first verification result indicates that the vulnerability verification script can be run, the validity of the target vulnerability is verified using the target language model and the vulnerability verification script to obtain a second verification result, wherein the second verification result is used to indicate whether the target vulnerability is valid in the vulnerability address.

2. The data processing method according to claim 1, characterized in that: Using the target language model and the vulnerability verification script, verifying the validity of the target vulnerability to obtain a second verification result includes: Executing the vulnerability verification script, accessing the vulnerability address, and collecting response data returned by the vulnerability address to obtain first response data; The first response data and the vulnerability data are input into the target language model, the validity of the target vulnerability is verified, and the second verification result is obtained.

3. The data processing method according to claim 2, characterized in that: The vulnerability data also includes: expected response data, the expected response data includes: response data that can be generated by accessing the vulnerability address, and the validity of the target vulnerability is verified to obtain the second verification result, including: Verify whether the first response data is consistent with the expected response data, and obtain a target verification result; In a case where the target verification result indicates that the first response data is consistent with the expected response data, determining that the second verification result indicates that the target vulnerability is valid in the vulnerability address; In a case where the target verification result indicates that the first response data is inconsistent with the expected response data, determining that the second verification result indicates that the target vulnerability is invalid in the vulnerability address.

4. The data processing method according to claim 1, characterized in that: After extracting the vulnerability data recorded in the vulnerability report, it also includes: Based on the data type of the vulnerability data, the vulnerability data is classified to obtain classified vulnerability data, wherein the data type includes at least one of the following: text, code, and picture; The classified vulnerability data is stored in a vulnerability database, wherein the vulnerability database includes: a database for recording data of different vulnerabilities.

5. The data processing method according to claim 4, characterized in that: The classified vulnerability data includes at least one of the following: text data, code data, and image data, wherein the text data includes at least one of the following: an identifier of the target vulnerability, a type of the target vulnerability, and an address of the vulnerability, the code data includes: a code of the vulnerability verification script, and the image data includes: an image file in the vulnerability data. Storing the classified vulnerability data in a vulnerability database includes: Convert the text data into a key-value pair to obtain a target key-value pair, and store the target key-value pair in the vulnerability database; or The code data and the image data are stored in the vulnerability data in the form of files, and the file paths where the code data and the image data are stored in the vulnerability database are recorded.

6. The data processing method according to claim 3, characterized in that: The target language model is also used for vulnerability detection. When the first verification result indicates that the vulnerability verification script can be run, the validity of the target vulnerability is verified using the target language model and the vulnerability verification script. After obtaining the second verification result, the method further includes: In the case where the second verification result indicates that the target vulnerability is valid in the vulnerability address, determining a machine to be detected, and generating a vulnerability detection script based on the vulnerability verification script and the target address of the machine to be detected, wherein the machine to be detected includes: a machine to be detected whether the target address has the target vulnerability, and the vulnerability detection script is used to perform vulnerability detection on the machine to be detected; Executing the vulnerability detection script and collecting response data returned by the target address to obtain second response data; Based on the second response data and the expected response data, it is detected whether the target address of the machine to be detected has the target vulnerability, a detection result is obtained, and the detection result is stored in a vulnerability database.

7. The data processing method according to claim 1, characterized in that: The target language model is obtained in the following way: Obtaining a training sample, wherein the training sample includes: vulnerability data of N types of vulnerabilities, where N is a positive integer; Based on the training samples, the initial language model is trained until the initial language model converges to obtain the target language model, wherein the initial language model includes: a language model to be trained, and the loss function used in the process of training the initial language model includes: a cross entropy loss function.

8. A data processing device based on vulnerability reports, characterized in that: include: An extraction unit, used to extract vulnerability data recorded in the vulnerability report, wherein the vulnerability data at least includes: a vulnerability verification script and a vulnerability address, wherein the vulnerability address is used to record the address where the target vulnerability is found, and the vulnerability verification script is used to verify whether the target vulnerability exists in the vulnerability address; A first verification unit, configured to verify the vulnerability verification script using a target language model to obtain a first verification result, wherein the target language model is used to verify the vulnerability data, and the first verification result is used to indicate whether the vulnerability verification script can be run; The second verification unit is used to verify the validity of the target vulnerability using the target language model and the vulnerability verification script when the first verification result indicates that the vulnerability verification script can be run, so as to obtain a second verification result, wherein the second verification result is used to indicate whether the target vulnerability is valid in the vulnerability address.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the data processing method based on vulnerability reporting according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory storing an executable program; A processor is used to run the program, wherein the program, when running, executes the data processing method based on vulnerability reports as described in any one of claims 1 to 7.

11. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the data processing method based on vulnerability reports described in any one of claims 1 to 7 are implemented.

Images