Automatic vulnerability verification method, system, equipment and medium

Abstract

The invention discloses an automatic vulnerability verification method and system. The method comprises the steps that code comparison groups are obtained and subjected to serialization processing to obtain mark sequences; matching the mark sequences to obtain a mark sequence group; screening the marker sequence group to obtain a difference fragment; obtaining the longest common subsequence in the marked sequence group to obtain a basic difference fragment which is a difference fragment meeting a preset similarity; obtaining a minimum editing operation sequence in the mark sequence group, and positioning the position of a difference fragment; determining the similarity among the difference segments based on the basic difference segments, and determining the adjacency among the difference segments based on the positions of the difference segments; combining the difference fragments meeting the preset similarity and the preset adjacency in the mark sequence group to obtain a difference block; and performing code comparison based on the difference block to generate a difference result of a code comparison group. According to the invention, the accuracy and efficiency of vulnerability verification are improved.

Description

Technical Field

[0001] The present invention relates to the field of automobile safety technology, and in particular to an automatic vulnerability verification method, system, program product and medium. Background Art

[0002] With the development of connected and intelligent vehicles, the issue of automotive network security is becoming increasingly severe. More and more companies are using vulnerability scanning to conduct security testing on codes in order to discover more potential risks.

[0003] Although existing automated vulnerability scanning tools can efficiently identify and match vulnerabilities such as CVE vulnerabilities, they still have obvious deficiencies in vulnerability verification. Traditional vulnerability scanning tools usually lack the ability to automatically verify CVE vulnerabilities, which requires R&D personnel to spend a lot of time and energy on vulnerability verification and evaluation. Although some tools provide basic automated verification functions, their verification accuracy and coverage are still limited, making it difficult to meet the complex and diverse needs of enterprises, resulting in the following problems: Limited vulnerability coverage: Existing automated verification systems have limited coverage of vulnerability verification and cannot effectively verify all types of vulnerabilities, especially complex, multi-step vulnerability verification; Insufficient vulnerability verification accuracy: Many vulnerability verification methods are prone to false positives or false negatives during the verification process, affecting the accuracy of vulnerability verification; Vulnerability verification lacks flexibility: Existing vulnerability verification methods are difficult to flexibly adapt to the code repository environment and development processes of different enterprises, and are difficult to meet the specific needs of enterprises. Summary of the invention

[0004] In response to the above technical problems, the present invention provides a vulnerability automated verification method, system, device and medium, which can realize the demand for automated verification of vulnerabilities, improve the accuracy and efficiency of vulnerability verification, reduce the handling pressure of R&D personnel, and improve software quality.

[0005] A first aspect of the present invention provides a vulnerability automated verification method, comprising: Obtain and serialize the code comparison groups to obtain each marker sequence; Match each tag sequence to obtain a tag sequence group; filter the tag sequence group to obtain a difference segment; obtain the longest common subsequence in the tag sequence group to obtain a basic difference segment, where the basic difference segment is a difference segment that meets a preset similarity; obtain the minimum editing operation sequence in the tag sequence group to locate the difference segment position; Determine the similarity between the difference segments based on the basic difference segments, and determine the proximity between the difference segments based on the difference segment positions; merge the difference segments in the marker sequence group that meet the preset similarity and preset proximity to obtain a difference block; Code comparison is performed based on the difference block to generate a difference result of the code comparison group.

[0006] As an optional implementation manner, the screening of the marker sequence group to obtain the difference fragments includes: Calculate the hash value of each tag sequence in the tag sequence group; Each tag sequence in the tag sequence group is compared based on the hash value, and a difference segment is obtained.

[0007] As an optional implementation manner, obtaining the minimum editing operation sequence in the mark sequence group and locating the difference segment position includes: Calculate the edit distance between the labeled sequences in the labeled sequence group; Based on the edit distance, a minimum edit operation sequence between the marked sequences is determined, and the content of the minimum edit operation at least includes marking the modification position and the corresponding modification operation type.

[0008] As an optional implementation, the vulnerability automated verification method further includes: Query the POC corresponding to the vulnerability; For vulnerabilities with public POC, configure the corresponding test environment URL and domain controller parameters for automated verification; For vulnerabilities with undisclosed POCs, a POC script is generated based on the vulnerability features, and automated vulnerability verification is performed based on the POC script.

[0009] As an optional implementation, before querying the POC corresponding to the vulnerability, the following is also included: Respond to the user's software bill of materials entry action and record the login session to access the code repository.

[0010] A second aspect of the present invention provides a vulnerability automated verification system, including: a first verification module, wherein the first verification module includes: An acquisition module, at least used for performing serialization processing on the code comparison groups to obtain respective tag sequences; The processing module is at least used to match each tag sequence to obtain a tag sequence group; screen the tag sequence group to obtain a difference segment; obtain the longest common subsequence in the tag sequence group to obtain a basic difference segment, wherein the basic difference segment is a difference segment that meets a preset similarity; obtain the minimum editing operation sequence in the tag sequence group to locate the difference segment position; A merging module, at least used to determine the similarity between the difference segments based on the basic difference segments, and determine the proximity between the difference segments based on the difference segment positions; merge the difference segments in the marker sequence group that meet the preset similarity and preset proximity to obtain a difference block; The comparison module is at least used to perform code comparison based on the difference block and generate a difference result of the code comparison group.

[0011] As an optional implementation, the vulnerability automated verification system further includes a second verification module; The second verification module is at least used to query the POC corresponding to the vulnerability; for vulnerabilities with public POCs, the corresponding test environment URL and domain controller parameters are configured for automated verification; for vulnerabilities with unpublic POCs, a POC script is generated based on vulnerability features, and automated vulnerability verification based on the POC script is performed.

[0012] As an optional implementation, the vulnerability automated verification system further includes a retrieval module, which is used to respond to the user's software bill of materials entry action and record a login session for accessing the code repository.

[0013] A third aspect of the present invention provides a code comparison method, comprising: Obtain and serialize the source code and the patch code to obtain a source sequence and a patch sequence respectively; The source sequence and the patch sequence are screened to obtain a difference segment; the longest common subsequence in the source sequence and the patch sequence is obtained to obtain a basic difference segment, where the basic difference segment is a difference segment that meets a preset similarity; the minimum editing operation sequence in the source sequence and the patch sequence is obtained to locate the difference segment position; Determine the similarity between the difference segments based on the basic difference segments, and determine the proximity between the difference segments based on the difference segment positions; merge the difference segments in the source sequence and the patch sequence that meet the preset similarity and preset proximity to obtain a difference block; Code comparison is performed based on the difference block.

[0014] A fourth aspect of the present invention provides a code comparison system, comprising: A serialization module, at least used to obtain and serialize the source code and the patch code to obtain a source sequence and a patch sequence respectively; The sequence processing module is at least used to screen the source sequence and the patch sequence to obtain the difference segment; obtain the longest common subsequence in the source sequence and the patch sequence to obtain the basic difference segment, wherein the basic difference segment is a difference segment that meets a preset similarity; obtain the minimum editing operation sequence in the source sequence and the patch sequence to locate the difference segment position; A sequence merging module is at least used to determine the similarity between the difference segments based on the basic difference segments, and determine the proximity between the difference segments based on the difference segment positions; merge the difference segments in the source sequence and the patch sequence that meet the preset similarity and preset proximity to obtain a difference block; The block comparison module is at least used to perform code comparison based on the difference block.

[0015] A fifth aspect of the present invention provides an electronic device, comprising: At least one processor; and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute a method as described in any one of the first aspect of the present invention or the third aspect of the present invention.

[0016] A sixth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a computer, the method as described in any one of the first aspect of the present invention or the third aspect of the present invention is executed.

[0017] The present invention serializes the codes in the code comparison group and converts the codes into marker sequences, thereby improving the operability of code comparison. By screening the marker sequence group and merging the differences, adjacent and similar difference fragments can be merged, thereby improving the readability of the comparison result. The present invention can quickly and accurately identify the differences in the code, improve the verification efficiency and accuracy, and can improve the problem in actual research and development that some environments are not connected to the enterprise intranet and POC automatic verification cannot be performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The figure is a flow chart of an automatic vulnerability verification method in an embodiment of the present invention.

[0019] Figure 2 The following is a working principle diagram of another automatic vulnerability verification method in an embodiment of the present invention.

[0020] Figure 3 The block diagram of an automatic vulnerability verification system in an embodiment of the present invention.

[0021] Figure 4 The block diagram of another automatic vulnerability verification system in an embodiment of the present invention.

[0022] Figure 5 The figure is a flowchart of a code comparison method in an embodiment of the present invention.

[0023] Figure 6 The block diagram of a code comparison system in an embodiment of the present invention.

[0024] Figure 7 FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0026] It should be understood that the terms "first", "second", and "third" etc. in the claims, specifications, and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections. It should also be understood that the terms used in this disclosure specification are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure.

[0027] With the development of connected and intelligent vehicles, the security of automotive networks is becoming increasingly serious. More and more companies are using vulnerability scanning to perform security checks on codes during the R&D process. At present, mainstream binary vulnerability scanning tools mainly rely on code analysis, syntax tree analysis, data flow analysis and other methods to identify relevant components and versions from source code repositories or artifact libraries, and match publicly disclosed vulnerabilities from public vulnerability libraries such as NVD and CNVD through version matching.

[0028] Common Vulnerabilities and Exposures (CVE) is an internationally recognized, public database that aims to provide unified identification and description of known security vulnerabilities in computer systems, network devices, and applications. Each vulnerability recorded in the CVE database is assigned a unique identifier, the CVE number, which helps to standardize the identification, sharing, and response to security vulnerabilities worldwide.

[0029] Proof of Concept (POC) usually refers to code or methods that show that a vulnerability exists and can be exploited. POC code can help security researchers and developers understand how a vulnerability works and how to prevent it from being exploited maliciously.

[0030] A software bill of materials (SBOM) is a list that describes software components and their interrelationships, including component name, version number, vendor, and other information for security management, vulnerability tracking, and license compliance checking.

[0031] A code repository (code-repo) is a place to store and manage code. It can be a simple file storage or a more complex version control system. Code repositories can be used to track code change history, support multi-person collaborative development, manage different versions of code, etc. Common code repository systems include Git, Mercurial, Subversion, etc.

[0032] The vulnerability automated verification method provided by the present invention solves the deficiencies of current automated vulnerability scanning tools in CVE vulnerability verification, improves the accuracy and efficiency of vulnerability verification, reduces the handling pressure of R&D personnel, and improves software quality.

[0033] like Figure 1 and Figure 2 As shown, the first aspect of the present invention provides a vulnerability automated verification method, comprising the following steps: Step S100: Obtain and serialize the code comparison group to obtain each mark sequence, wherein the code comparison group consists of source code and patch code; taking the source code of Version A and Version B as an example, the source code may be Version A, and the patch code may be Version B, which may also be called a code comparison pair.

[0034] Specifically, this step belongs to the preprocessing environment, and the serialization process includes converting the code into a token sequence by removing comments and blank lines. The purpose of removing comments and blank lines is to reduce unnecessary interference and improve the comparison efficiency between token sequences. Taking the source code of Version A and Version B as an example, this step converts the source code of Version A and Version B into token sequence A and token sequence B by removing comments and blank lines.

[0035] Step S200: Match each tag sequence to obtain a tag sequence group; filter the tag sequence group to obtain a difference segment; obtain the longest common subsequence in the tag sequence group to obtain a basic difference segment, where the basic difference segment is a difference segment that meets a preset similarity; obtain the minimum editing operation sequence in the tag sequence group to locate the difference segment position.

[0036] In this step, taking the source code of Version A and Version B as an example, the tag sequence A and the tag sequence B are matched to obtain a tag sequence group, which is the main body of the code comparison.

[0037] In this step, the marker sequence group is screened to obtain differential fragments, including: Calculate the hash value of each tag sequence in the tag sequence group; Each tag sequence in the tag sequence group is compared based on the hash value, and the difference fragments are obtained by screening.

[0038] In this step, taking the source code of Version A and Version B as an example, the longest common subsequence algorithm (LCS algorithm) is used for accurate comparison, the longest common subsequence in the tag sequence group is obtained, and the basic difference segment is obtained. The basic difference segment is a difference segment that meets the preset similarity. Specifically, after hash screening, the LCS algorithm is used to accurately compare the part with large differences. Since the LCS algorithm can find the longest common subsequence in the two sequences, that is, the similar part, it can be identified as the basic difference segment. And the difference segment with the preset similarity proposed in the present invention corresponds to the longest common subsequence in the tag sequence A and the tag sequence B obtained by the LCS algorithm.

[0039] In this step, taking the source code of Version A and Version B as an example, the step of obtaining the minimum editing operation sequence in the tag sequence group and locating the difference segment position includes: Calculate the edit distance between the marker sequences in the marker sequence group, such as marker sequence A and marker sequence B; Based on the edit distance, the minimum edit operation sequence between the marker sequences such as marker sequence A and marker sequence B is determined, and the difference segment position is further located, wherein the content of the minimum edit operation at least includes marking the modification position and the corresponding modification operation type.

[0040] Step S300: determining the similarity between the difference segments based on the basic difference segments, and determining the proximity between the difference segments based on the difference segment positions; merging the difference segments in the marker sequence group that meet the preset similarity and preset proximity to obtain or generate a difference block.

[0041] Specifically, taking the source code of Version A and Version B as an example, by using a merging strategy to merge the difference fragments that meet a preset similarity and a preset proximity in a tag sequence group, such as a tag sequence group composed of tag sequence A and tag sequence B, adjacent and similar difference fragments can be merged into a larger difference block to improve the readability and comprehensibility of the comparison results of tag sequence A and tag sequence B.

[0042] Step S400: Perform code comparison based on the difference block, and generate difference results of the code comparison group to realize automatic vulnerability verification. Specifically, taking the source code of Version A and Version B as an example, the difference results are used to show the differences and gaps between the codes of Version A and Version B, which are displayed with the help of visualization means, so as to help users quickly verify vulnerabilities.

[0043] Specifically, the application scenarios of the above-mentioned method based on automated vulnerability verification include but are not limited to the situation where there are official patches and the source code and patch code are available. The logic is that when an official patch is provided, code comparison can quickly verify whether the patch has been applied. For example, a company's code repository contains two versions of source code (Version A and Version B), and it is necessary to verify whether there is a CVE vulnerability in them. After multiple verification attempts, the patch code provided by the official patch is finally compared.

[0044] Also, in the application scenario of the above-mentioned method based on automated vulnerability verification, if there is no public POC for the vulnerability, a POC script is generated based on the trigger function, component version and other features of the vulnerability, and the Web Token or domain controller login credentials are called as needed to execute the POC script for verification. Since the component is indirectly introduced by other components and cannot be directly executed after packaging, custom POC verification is not applicable, and finally verification is attempted through code comparison.

[0045] Based on this, the present invention combines hash screening, LCS algorithm, Myers algorithm and difference merging strategy to design a new code comparison algorithm, which can quickly and accurately identify differences in the code and present them in a visual way, thereby improving verification efficiency and accuracy. It can improve the problem in actual research and development that some environments are not connected to the enterprise intranet and the automated verification module cannot perform POC automatic verification.

[0046] Furthermore, the vulnerability automated verification method further includes: Query the POC corresponding to the vulnerability; For vulnerabilities with public POC, configure the corresponding test environment URL and domain controller parameters for automated verification; For vulnerabilities with undisclosed POCs, a POC script is generated based on the vulnerability features, and the Web Token or domain controller login credentials are called as needed to execute the POC script for verification. The vulnerability features include the trigger function of the vulnerability, component version and other features.

[0047] The following strategies and methods are adopted for specific technical means of vulnerability points: 1. Configure the URL of the test environment for verification Environment parameterization configuration: In the automated verification module, configure the URL and related parameters of the test environment. You can flexibly set the access address and port of the test environment through configuration files or interface input.

[0048] 2. Configure the domain controller login method, commands, and parameters Domain environment login configuration: For vulnerability verification involving domain controllers, you need to configure the domain environment login method and credentials (such as user name, password, security certificate, etc.); Command execution and parameter configuration: Configure the command execution mode and parameters on the domain controller according to the specific POC script or verification requirements. You can use network protocols (such as SSH, Telnet, ADB, etc.) to establish a connection with the domain controller and perform related operations through scripts or command line tools.

[0049] 3. Verification process and result determination POC script execution: Based on the public POC of the CVE vulnerability, write or configure the automated verification module to execute the corresponding POC script to ensure that the script can be correctly executed in the specified test environment or domain controller; Result acquisition and analysis: After the automated verification module executes the POC script, it obtains the execution results and output. Information and events generated during the verification process can be obtained through logging, exception capture, and output parsing; Verify whether there is a vulnerability: Analyze the results of the POC execution to determine whether the target system is affected by the vulnerability. The results may include different verification scenarios such as successful exploitation of the vulnerability, denial of service, and information leakage.

[0050] The process is as follows: WEB vulnerability verification: configure the URL of the test environment for verification; Domain controller vulnerability verification: Configure the domain controller login method, commands, and parameters to execute POC in the corresponding environment.

[0051] In addition, based on the code comparison verification mode, POC automatic verification mode (POC not disclosed) and automated verification mode (public POC), the present invention also provides manual assisted verification, which is applicable to manual assisted verification when there is no public POC or official patch, or the environment cannot be connected to the automated verification module.

[0052] In order to better accurately and quickly verify the path vulnerability, the present invention provides a priority based on the vulnerability verification model, which mainly involves priority logic and content including: The code comparison verification mode is in high priority and is applicable to situations where official patches are available and the source code and patch code are available. The logic is that when an official patch is provided, code comparison can quickly verify whether the patch has been applied. The automatic verification mode is in medium priority and is applicable to scenarios where there is a public POC and a test environment or domain controller can be configured for verification. The logic is that when code comparison verification is not possible or patches are not available, using POC for automatic verification is the second best choice. For vulnerabilities without public POC, the priority is between the automatic verification mode and the manual assisted mode; The manual verification mode is of low priority and is applicable when there is no public POC or official patch, or when the environment cannot be connected to the automated verification module. The logic is that manual verification is the last resort when automated means cannot complete verification.

[0053] In one embodiment of the present invention, the vulnerability automatic verification method, before querying the POC corresponding to the vulnerability, further includes: Respond to the user's SBOM list entry action and record the login session for accessing the code repository.

[0054] In the relevant retrieval step, traditional vulnerability scanning tools usually only rely on manual entry or simple automated extraction when searching the code repository. The present invention not only provides an automatic retrieval function, but also supports manual entry of the SBOM list and recording of login sessions for accessing the code repository, and flexibly adapts to the environments and needs of different enterprises. The present invention supports multiple ways to obtain the SBOM list, and through advanced permission control and session recording, ensures comprehensive coverage and accurate retrieval of the code repository.

[0055] like Figure 3 and Figure 4 As shown, the second aspect of the present invention provides a vulnerability automated verification system, including: an automated verification module 50, which includes a first verification module 51: Specifically, the first verification module 51 includes: An acquisition module 511 is at least used to perform serialization processing on the code comparison groups to obtain respective tag sequences; The processing module 512 is at least used to match each tag sequence to obtain a tag sequence group; filter the tag sequence group to obtain a difference segment; obtain the longest common subsequence in the tag sequence group to obtain a basic difference segment, wherein the basic difference segment is a difference segment that meets a preset similarity; obtain the minimum editing operation sequence in the tag sequence group to locate the difference segment position; The merging module 513 is at least used to determine the similarity between the difference segments based on the basic difference segments, and determine the proximity between the difference segments based on the difference segment positions; merge the difference segments in the marker sequence group that meet the preset similarity and preset proximity to obtain a difference block; The comparison module 514 is at least used to perform code comparison based on the difference block to generate a difference result of the code comparison group.

[0056] In one embodiment of the present invention, the automated verification module further includes a second verification module 52 and a third verification module 53; The second verification module 52 is at least used to query the POC corresponding to the vulnerability; for vulnerabilities of publicly disclosed POCs, the corresponding test environment URL and domain controller parameters are configured for automated verification; for vulnerabilities of undisclosed POCs, a POC script is generated based on vulnerability features, and automated vulnerability verification based on the POC script is performed.

[0057] The third verification module 53 is at least used to provide an interface or platform for manual assisted verification when there is no public POC or official patch, or the environment cannot be connected to the automated verification module.

[0058] Before performing the above vulnerability verification, the vulnerability automated verification system further includes a crawler module 54, a data processing module 55 and a retrieval module 56; Among them: the crawler module 54 is responsible for continuously collecting open source vulnerability information and intelligence from the Internet. At present, it mainly uses authoritative vulnerability libraries such as NVD and CNVD and mainstream intelligence communities such as the Weibu X community as information sources to crawl CVE vulnerability details and related links. Other information sources can also be expanded as needed in the future, such as GitHub, Telegram, etc.

[0059] For details, please refer to Figure 2,The crawler module 54 mainly obtains data from multiple authoritative databases and intelligence communities such as NVD (National Vulnerability Database), CNVD (China National Vulnerability Database), Weibu X Community, Reddit Security Sub-forum, Telegram Vulnerability Disclosure Channel and GitHub Security Project. In order to cover more information sources, the system will also use RSS subscription technology to obtain the latest vulnerability information from major security blogs, news websites and forums, and design differentiated crawling strategies for different data sources. For authoritative databases such as NVD and CNVD, set daily / weekly scheduled crawling; for intelligence communities and social media, use WebSocket or polling mechanism for real-time monitoring; for other information sources, regularly check the updates of RSS subscriptions, use Selenium or Puppeteer to simulate user operations, and solve the content loading problem of dynamic web pages. In order to deal with the anti-crawler mechanism, IP rotation, request header camouflage and random delay technology are used to improve the crawling success rate, realize multi-channel and multi-format vulnerability information acquisition and integration, and greatly improve the coverage and timeliness of vulnerability intelligence.

[0060] The data processing module 55 is responsible for cleaning and normalizing the raw data collected by the crawler module 54, mapping it according to the CVE vulnerability number, vulnerability level, CVSS score, vulnerability details, affected component name, affected component version, vulnerability repair suggestions, related links and other fields, removing duplicate data and other information of little value, and storing it in the local database.

[0061] Specifically, the data processing module 55 cleans and normalizes the raw data collected by the crawler module 54, and the processing steps are as follows: Data cleaning: Remove duplicate data and other information of little value.

[0062] Data normalization: Map according to fields such as CVE vulnerability number, vulnerability level, CVSS score, vulnerability details, affected component name, affected component version, vulnerability repair suggestions, and related links.

[0063] Data storage: Store the cleaned and normalized data in a local database.

[0064] Data mapping: Map the cleaned data according to the CVE vulnerability number, vulnerability level, CVSS score, vulnerability details, affected component name, affected component version, vulnerability repair suggestions, related links and other fields.

[0065] Among them, in the mapping stage, the following technical processing methods are used: Entity recognition and linking: Use natural language processing (NLP) technology to identify entities and automatically extract key information such as component names, version numbers, and vulnerability descriptions. Use named entity recognition (NER) technology to extract key entities (such as software names and version numbers) from text and standardize them. For example, map "Windows Server 2016" to the standardized "Windows Server, Version 2016"; Fuzzy matching and standardization: For non-standardized data from different sources, similarity is calculated using fuzzy matching algorithms (such as Levenshtein distance), and similar entries are merged and standardized. For example, "MSWindows Server 2016" and "Microsoft Windows Server 2016" are treated as the same entry; Automatic classification and scoring: Use machine learning models to automatically classify and score vulnerabilities. By training the classifier, the vulnerability level (such as Critical, High, Medium, Low) and CVSS score are automatically predicted and filled in based on the vulnerability description and related information; Relationship graph construction: Use graph databases (such as Neo4j) to build a relationship graph between vulnerabilities and related components, versions, and repair suggestions. In the data mapping process, data nodes from different sources are connected to form a complete knowledge graph for better data query and association analysis; Duplicate data detection and deduplication: Combine hash algorithms and content similarity calculations (such as SimHash) to detect and remove duplicate data. Use data structures such as Bloom Filter to optimize the time and space complexity of the deduplication process, ensuring efficient and accurate removal of duplicate entries; Data verification and completion: During the data mapping process, the completeness and accuracy of the data are automatically verified. For missing information, try to automatically complete it through contextual information. For example, if some vulnerability entries lack repair suggestions, you can complete them by analyzing the repair suggestions of other similar vulnerabilities.

[0066] The data processing module 55 provides a relatively complete automated data processing flow, which combines machine learning algorithms to clean and normalize data to ensure the accuracy and completeness of the data.

[0067] The retrieval module 56 is used to respond to the user's SBOM list entry action and record the login session for accessing the code repository.

[0068] Specifically, traditional vulnerability scanning tools usually only rely on manual entry or simple automated extraction when searching the code repository. The retrieval module 56 provided by the present invention not only provides an automatic retrieval function, but also supports manual entry of the SBOM list and recording of login sessions for accessing the code repository, and is flexibly adapted to the environments and needs of different enterprises. The present invention supports multiple ways to obtain the SBOM list, and through advanced permission control and session recording, ensures comprehensive coverage and accurate retrieval of the code repository.

[0069] Specifically, the retrieval module 56 is responsible for searching for projects, software components and version information from the enterprise's code repository, forming an SBOM, matching it with the vulnerability entries stored in the local database, and outputting a list of vulnerabilities to be verified. Since the current mainstream R&D management tools are relatively strict in terms of authority control, the retrieval module 56 needs to provide the function of manually entering the SBOM list, and support recording the login session for accessing the code repository, or configuring valid token parameters.

[0070] Automatic retrieval includes the timing of triggering automatic retrieval and extracting project and component information. The timing of triggering automatic retrieval includes scheduled tasks, code submission triggers and manual triggers. Scheduled tasks configure scheduled tasks through task schedulers (such as Cron, Quartz, etc.) to trigger automatic retrieval regularly; code submission triggers use the Webhook mechanism of the code repository to automatically trigger retrieval when the code is submitted or merged; manual triggers provide an interface for manually triggering retrieval, allowing administrators to perform instant retrieval as needed. Extracting project and component information is mainly done through code repository API calls, parsing configuration files and static code analysis. Code repository API calls obtain projects, file directory structures and dependencies by calling the API interfaces of code repositories (such as GitHub, GitLab, Bitbucket, etc.); parsing configuration files automatically parses dependency configuration files (such as package.json, pom.xml, requirements.txt, etc.) in the project to extract software components and their version information; static code analysis performs static analysis on the code in the code repository to identify and extract hard-coded components and version information.

[0071] Exemplarily, in the manual entry of SBOM lists supporting administrators to manually enter or upload SBOM lists, the present invention supports batch import, edit and delete operations, and ensures that the entered SBOM lists conform to the expected format by performing basic format verification and validation on the manually entered data.

[0072] Exemplarily, login session recording supports recording login sessions for accessing code repositories. The session recording tool integrates third-party session recording tools (such as Selenium, Puppeteer, etc.) to automatically record the user's operation steps for logging into the code repository, including entering a user name, password, and 2FA verification. Session replay saves the recorded login session as a script, and replays the login process when access to the code repository is required to automatically obtain authorized access rights.

[0073] In addition, Token configuration supports configuring valid token parameters. The Token management interface provides an interface for administrators to configure and manage token parameters for accessing code repositories, and supports configuring different tokens for different code repositories. Secure storage encrypts the configured token to ensure the security of sensitive information. It is decrypted and used when actually called.

[0074] By comparing the components and version information in the SBOM list with the vulnerability entries in the local database, a list of vulnerabilities to be verified is generated, and the matching accuracy is improved by using hash algorithms and fuzzy matching technology. The automated verification module automatically verifies the vulnerabilities to be verified, which is also the core module that distinguishes this system from other current security tools.

[0075] In one embodiment of the present invention, the vulnerability automatic verification system further includes a design task and report module 58 (not shown in the figure), which involves the creation of vulnerability verification tasks and the output of vulnerability verification reports according to dimensions such as SBOM and projects, and supports visual display. SBOM dimension: allows users to select or import vulnerability information based on existing SBOM and create verification tasks, which can be implemented through interface input, file upload or API call.

[0076] Project dimension: Supports the creation of vulnerability verification tasks based on projects or applications. Users can select specific projects and associate related vulnerability information and verification requirements.

[0077] Task parameterization: Provides flexible task parameter configuration, including the selection of verification environment (such as test environment URL, domain controller login configuration, etc.), verification method (such as automated POC verification, manual verification, etc.), verification priority, etc.

[0078] Automatic report generation: Automatically generate vulnerability verification reports based on the execution results of verification tasks. The report content should include basic information of verification tasks, execution status, vulnerability verification results, abnormal situation reports, etc.

[0079] Report template customization: Report templates can be customized as needed, including custom report style and structure to enhance the professionalism and readability of the report.

[0080] Report output format: Provides multiple output format options, such as PDF, HTML, Excel, etc., to facilitate review and use by different roles and departments.

[0081] Vulnerability statistics and trend analysis: Displays vulnerability verification statistics and trend analysis through charts and graphical interfaces, including the number of vulnerabilities, verification pass rate, high-risk vulnerability distribution, etc.

[0082] Dynamic monitoring and real-time update: Supports real-time update of the execution status and results of vulnerability verification tasks, and dynamic monitoring of verification progress and vulnerability repair status.

[0083] Please continue reading Figure 2 , CVE-2022-12929 is a Common Vulnerability Exposure (CVE) number assigned to identify a specific security vulnerability. This number refers to a critical security vulnerability found in OpenSSL, a widely used open source software library for implementing the Secure Sockets Layer (SSL) and Transport Layer Security (TLS) protocols, as well as cryptographic algorithms for encrypting and decrypting data. The change history indicates all important changes recorded for the CVE-2022-1292 vulnerability, such as vulnerability update information, patch releases, etc.; in the announcement and resolution plan link, the announcement includes the official announcement of the software and some third-party announcements, and the resolution plan is to fix the vulnerability, such as upgrading the version or applying a patch.

[0084] The CVE vulnerability automated verification system based on open source intelligence proposed in this patent improves the efficiency and accuracy of vulnerability verification comprehensively through the collaborative work of crawler module, data processing module, retrieval module, automated verification module, task and report module. It provides more comprehensive vulnerability intelligence through multi-source data crawling and fusion, ensures high data quality through automated data cleaning and normalization, and can adapt to different enterprise environment requirements through flexible SBOM list acquisition methods. It covers a wider range of vulnerability verification needs for users through code comparison verification mode, POC verification mode (including two types: POC public and POC undisclosed) and manual assisted mode.

[0085] This system combines hash screening, LCS algorithm, Myers algorithm and difference merging strategy to design a new code comparison algorithm for automated verification. It can quickly and accurately identify differences in the code and present them in a visual way, improving verification efficiency and accuracy. Through efficient information collection and processing, precise vulnerability matching and verification, and the comprehensive application of multiple code comparison algorithms, it improves the accuracy and efficiency of CVE vulnerability verification, reduces the handling pressure of R&D personnel, and improves the security and quality of the software development process.

[0086] like Figure 5 As shown, the fourth aspect of the present invention provides a code comparison method, comprising: Step S610: Obtain and serialize the source code and the patch code to obtain a source sequence and a patch sequence respectively; Step S620: screening the source sequence and the patch sequence to obtain a difference segment; obtaining the longest common subsequence in the source sequence and the patch sequence to obtain a basic difference segment, wherein the basic difference segment is a difference segment that meets a preset similarity; obtaining the minimum editing operation sequence in the source sequence and the patch sequence to locate the difference segment position; Step S630: determining the similarity between the difference segments based on the basic difference segments, and determining the proximity between the difference segments based on the difference segment positions; merging the difference segments in the source sequence and the patch sequence that meet the preset similarity and preset proximity to obtain a difference block; Step S640: performing code comparison based on the difference block to generate a code comparison difference result.

[0087] like Figure 6 As shown, the fourth aspect of the present invention provides a code comparison system, comprising: A serialization module 71 is at least used to obtain and serialize the source code and the patch code to obtain a source sequence and a patch sequence respectively; The sequence processing module 72 is at least used to screen the source sequence and the patch sequence to obtain a difference segment; obtain the longest common subsequence in the source sequence and the patch sequence to obtain a basic difference segment, wherein the basic difference segment is a difference segment that meets a preset similarity; obtain the minimum editing operation sequence in the source sequence and the patch sequence to locate the difference segment position; The sequence merging module 73 is at least used to determine the similarity between the difference segments based on the basic difference segments, and determine the proximity between the difference segments based on the difference segment positions; merge the difference segments in the source sequence and the patch sequence that meet the preset similarity and preset proximity to obtain a difference block; The block comparison module 74 is at least used for performing code comparison based on the difference block.

[0088] like Figure 7 As shown, the fifth aspect of the present invention provides an electronic device, including: At least one processor; and at least one memory in communication with the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method described in any of the above embodiments.

[0089] A sixth aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in any one of the above embodiments is implemented.

[0090] It can be understood that computer-readable storage media may include: any entity or system capable of carrying a computer program, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc. A computer program includes a computer program code. The computer program code may be in source code form, object code form, an executable file, or some intermediate form, etc. A computer-readable storage medium may include: any entity or system capable of carrying a computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.

[0091] In certain embodiments of the present invention, the electronic device may include a controller or a processor, and the controller is a single-chip microcomputer chip that integrates a processor, a memory, a communication module, etc. The processor may refer to a processor included in the controller. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.

[0092] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0093] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0094] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vulnerability automated verification method, characterized in that: include: Obtain and serialize the code comparison groups to obtain each marker sequence; Matching each marker sequence to obtain a marker sequence group; Screening the marker sequence group to obtain a difference segment; obtaining the longest common subsequence in the marker sequence group to obtain a basic difference segment, wherein the basic difference segment is a difference segment that meets a preset similarity; obtaining the minimum editing operation sequence in the marker sequence group to locate the difference segment position; Determine the similarity between the difference segments based on the basic difference segments, and determine the proximity between the difference segments based on the difference segment positions; merge the difference segments in the marker sequence group that meet the preset similarity and preset proximity to obtain a difference block; Code comparison is performed based on the difference block to generate a difference result of the code comparison group.

2. The vulnerability automated verification method according to claim 1, characterized in that: The screening of the marker sequence group to obtain the difference fragments includes: Calculate the hash value of each tag sequence in the tag sequence group; Each tag sequence in the tag sequence group is compared based on the hash value, and a difference segment is obtained.

3. The vulnerability automated verification method according to claim 1, characterized in that: The step of obtaining the minimum editing operation sequence in the marker sequence group and locating the position of the difference segment includes: Calculate the edit distance between the labeled sequences in the labeled sequence group; Based on the edit distance, a minimum edit operation sequence between the marked sequences is determined, and the content of the minimum edit operation at least includes marking the modification position and the corresponding modification operation type.

4. The vulnerability automated verification method according to claim 1, characterized in that: Also includes: Query the vulnerability proof corresponding to the vulnerability; For vulnerabilities with public POC, configure the corresponding test environment URL and domain controller parameters for automated verification; For vulnerabilities with undisclosed POCs, a POC script is generated based on the vulnerability features, and automated vulnerability verification is performed based on the POC script.

5. The vulnerability automated verification method according to claim 4, characterized in that: Before querying the POC corresponding to the vulnerability, it also includes: Respond to the user's software bill of materials entry action and record the login session to access the code repository.

6. A vulnerability automated verification system, characterized in that: include: A first verification module, wherein the first verification module comprises: An acquisition module, at least used for performing serialization processing on the code comparison groups to obtain respective tag sequences; The processing module is at least used to match each tag sequence to obtain a tag sequence group; screen the tag sequence group to obtain a difference segment; obtain the longest common subsequence in the tag sequence group to obtain a basic difference segment, wherein the basic difference segment is a difference segment that meets a preset similarity; obtain the minimum editing operation sequence in the tag sequence group to locate the difference segment position; A merging module, at least used to determine the similarity between the difference segments based on the basic difference segments, and determine the proximity between the difference segments based on the difference segment positions; merge the difference segments in the marker sequence group that meet the preset similarity and preset proximity to obtain a difference block; The comparison module is at least used to perform code comparison based on the difference block and generate a difference result of the code comparison group.

7. The vulnerability automated verification system according to claim 6, characterized in that: Also includes a second verification module; The second verification module is at least used to query the POC corresponding to the vulnerability; for vulnerabilities with public POCs, the corresponding test environment URL and domain controller parameters are configured for automated verification; for vulnerabilities with unpublic POCs, a POC script is generated based on vulnerability features, and automated vulnerability verification based on the POC script is performed.

8. The vulnerability automated verification system according to claim 7, characterized in that: It also includes a retrieval module, which is used to respond to the user's software material list entry action and record the login session for accessing the code warehouse.

9. A code comparison method, characterized in that: include: Obtain and serialize the source code and the patch code to obtain a source sequence and a patch sequence respectively; The source sequence and the patch sequence are screened to obtain a difference segment; the longest common subsequence in the source sequence and the patch sequence is obtained to obtain a basic difference segment, where the basic difference segment is a difference segment that meets a preset similarity; the minimum editing operation sequence in the source sequence and the patch sequence is obtained to locate the difference segment position; Determine the similarity between the difference segments based on the basic difference segments, and determine the proximity between the difference segments based on the difference segment positions; merge the difference segments in the source sequence and the patch sequence that meet the preset similarity and preset proximity to obtain a difference block; Code comparison is performed based on the difference block.

10. A code comparison system, characterized in that: include: A serialization module, at least used to obtain and serialize the source code and the patch code to obtain a source sequence and a patch sequence respectively; The sequence processing module is at least used to screen the source sequence and the patch sequence to obtain the difference segment; obtain the longest common subsequence in the source sequence and the patch sequence to obtain the basic difference segment, wherein the basic difference segment is a difference segment that meets a preset similarity; obtain the minimum editing operation sequence in the source sequence and the patch sequence to locate the difference segment position; A sequence merging module is at least used to determine the similarity between the difference segments based on the basic difference segments, and determine the proximity between the difference segments based on the difference segment positions; merge the difference segments in the source sequence and the patch sequence that meet the preset similarity and preset proximity to obtain a difference block; The block comparison module is at least used to perform code comparison based on the difference block.

11. An electronic device, characterized in that: include: at least one processor; and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method as claimed in any one of claims 1 to 5 or 9.

12. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 5 or 9 is executed.

Images