Scanning platform based on software component analysis

Abstract

The invention relates to the field of software component analysis, and particularly discloses a scanning platform based on software component analysis, which comprises a knowledge base service unit, a detection engine service unit, a business platform management service unit and a data persistence service unit. An enterprise can be assisted to identify vulnerability risks and compliance risks in an open source component, analyze levels of the risks and track and position known vulnerabilities; repair suggestions can be provided for existing risks, and software is prevented from getting online with diseases. An enterprise is assisted to collect and manage software asset information, and enterprise-level, department-level and project-level software asset distribution visualization is realized; and whether the component dependency relationship or the vulnerability propagation chain can be clearly checked through the platform, so that the risk checking time cost of developers and security personnel is reduced, and the development process is accelerated.

Description

Technical Field

[0001] The invention relates to a scanning platform, in particular to a scanning platform based on software component analysis, and belongs to the field of software component analysis. Background Art

[0002] The software component analysis platforms currently on the market have many problems, such as low detection quality, failure to meet relevant management requirements, relatively low automation, and complex operation methods. These problems can be mainly summarized into the following categories:

[0003] Security risks cannot be grasped: The risk situation of the open source components used is unclear, and it is impossible to accurately determine whether there are vulnerabilities and compliance issues. They may contain undiscovered or unpatched security vulnerabilities that can be easily exploited by attackers, leading to data leakage, data tampering or other malicious activities.

[0004] Untimely vulnerability warnings: Faced with an endless stream of new vulnerability threats, enterprises are unable to obtain information feedback in the shortest possible time, and are unable to promptly learn about various security and compliance issues encountered in the process of using open source components.

[0005] Substandard component quality: The quality of open source components is unstable, which leads to unstable software operation or other problems. Since the source code of open source components is public, it may be easier for attackers to find vulnerabilities and launch attacks.

[0006] Insufficient dependency component analysis capabilities: Unable to perform dependency analysis and management on open source components, quickly locate and identify risky components, and difficult to promptly block risks through dependency analysis and impact analysis to ensure project transparency and security.

[0007] Compliance checking work is scattered: The compliance checking process is not integrated into the components, which cannot ensure that the licenses of all open source components used have been reviewed and approved, and cannot conduct regular audits to ensure continuous compliance.

[0008] Unclear problem fixes: When faced with open source component security issues, there is a lack of effective solutions and automated processing methods. Summary of the invention

[0009] In view of the problems in the prior art, the present invention provides a scanning platform based on software component analysis.

[0010] The purpose of the present invention can be achieved through the following technical solutions:

[0011] The scanning platform based on software component analysis includes knowledge base service unit, detection engine service unit, business platform management service unit, and data persistence service unit;

[0012] The business platform management service unit is used to provide application management, asset management, user role management, security management, plug-in management, detection report, statistical analysis, SBOM, private server firewall and fast scanning functions;

[0013] The detection engine service unit is used to provide detection support for the business platform management service unit through an API;

[0014] The knowledge base service unit is used to provide data to the business platform management service unit and the detection engine service unit;

[0015] The data persistence service unit is used to cache data and provide data support for statistical analysis and report export based on the cached data.

[0016] Optionally, the knowledge base service unit includes a data interface module, a data verification module, a data cleaning module, a data updating module and a data collection module;

[0017] The data collection module can be used to collect third-party software information, which includes the original version information of the third-party software, the release time of the original version, the latest version information, the release time of the latest version, and the CVE public vulnerability information contained in the current version. The data update module is used to collect third-party software information and update the data. The data cleaning module cleans the data updated by the data update module and the data verification module verifies the cleaned data. Finally, the verified data is transmitted to the business platform management service unit and the detection engine service unit through the data interface.

[0018] Optionally, the detection engine service unit includes a package manager analysis module, a source code tracing analysis module, a multi-link component dependency analysis module, a product binary component analysis module and a security vulnerability reachability analysis module, and performs detection and analysis on open source components based on the package manager analysis module, the source code tracing analysis module, the multi-link component dependency analysis module, the product binary component analysis module and the security vulnerability reachability analysis module.

[0019] Optionally, the package manager analysis module scans the binary import packages in the configuration files and application packages of the target detection object to obtain the component information directly introduced and the component information indirectly introduced, and obtains the complete component information by matching and identifying the content in the knowledge base service unit. It can realize component behaviors such as the identification of modified components, dependency relationships between components, and self-developed component settings.

[0020] Optionally, the source code tracing analysis module collects open source code and indexes the collected open source source code, so that the source file and the target file are merged, and then used to detect whether the code itself has redundant and repeated code and detect code duplication between the source code and the target code.

[0021] Optionally, the multi-link component dependency analysis module obtains complete component information based on the package manager analysis module to simulate the process tracking and positioning of component dependencies, verifies the data information of the dependency tree with the collected open source data, and supplements the missing information of the dependency tree data to obtain the final dependency data.

[0022] Optionally, the product binary component analysis module recursively extracts the content of the binary product file to be detected, identifies the format through the file format recognition engine, calls the feature extraction engine for files of different formats respectively, extracts the feature information of the binary product file, and matches it with the open source component feature library through graph similarity, function similarity or semantic similarity algorithms to determine the name and version of the introduced open source component, and obtains the license and vulnerability risk information of the open source component in combination with the data support of the knowledge base service unit.

[0023] Optionally, the security vulnerability reachability analysis module extracts AST information from the source code uploaded by the user through analysis based on AST (abstract syntax tree) and call analysis methods, obtains function call chain information of the source code, and downloads source code of various versions of open source projects from mainstream open source repositories, extracts methods in the source code, stores the extracted function information in the open source component knowledge base, and establishes an association relationship with the open source component to form an open source function library, based on the analysis of the call chain and call location, to provide more accurate repair information for the R&D and security departments.

[0024] Optionally, while extracting the binary features, sensitive information in the binary artifact file is extracted to generate a sensitive information list.

[0025] Beneficial effects of the present invention:

[0026] 1. Accurately control the risks of open source components: It can help enterprises identify vulnerability risks and compliance risks in open source components, analyze the level of risks, track and locate known vulnerabilities; for existing risks, it can provide repair suggestions to prevent software from being put online with problems.

[0027] 2. Transparent management: Help enterprises collect and manage software asset information, and visualize the distribution of software assets at the enterprise, department, and project levels. Whether it is component dependencies or vulnerability propagation chains, all are clearly visible through the platform, reducing the time cost of risk investigation for developers and security personnel and speeding up the development process.

[0028] 3. Minimize the cost of risk management: It can monitor new vulnerabilities in real time and quickly analyze the impact of new vulnerabilities, curb the attack of new vulnerabilities in the initial stage, minimize the losses caused by sudden vulnerabilities, and help enterprises reduce the cost of security risk management.

[0029] 4. Quickest way to achieve secure components: With a component knowledge base of hundreds of millions of levels, it can provide professional repair suggestions and component replacement solutions for risks in enterprise software assets; helping developers complete component selection in a shorter time.

[0030] 5. Automated integration with zero manual intervention: Supports automated testing, automatically pushes risk problem lists; integrates Cl&CD, automatically triggers testing, configures relevant bill of lading rules and project scope in the product; combines with defect or work order systems, automatically generates work orders and submits them to R&D personnel. Automated processes greatly help companies improve R&D efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0032] Figure 1 This is a system diagram of the present invention. DETAILED DESCRIPTION

[0033] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] See also Figure 1 As shown, the scanning platform based on software component analysis includes a knowledge base service unit, a detection engine service unit, a business platform management service unit, and a data persistence service unit;

[0035] The business platform management service unit is used to provide application management, asset management, user role management, security management, plug-in management, detection report, statistical analysis, SBOM, private server firewall and fast scanning functions. It is a visual management platform that provides users with comprehensive open source governance;

[0036] The detection engine service unit is used to provide detection support for the business platform management service unit through the API; by calling the knowledge base service unit to identify open source component information, it provides component composition analysis, source code analysis, binary product analysis, and SCANNER-CLI to provide enterprises with open source governance detection services. It supports local upload (compressed package, source code package, binary package and container image package), code repository upload and other methods to detect software packages, helping development teams to effectively manage the use of third-party software.

[0037] The knowledge base service unit is used to provide data for the business platform management service unit and the detection engine service unit; it collects information such as open source components and vulnerabilities through real-time data collection and data cleaning. As of now, the number of component versions included is no less than 130 million, and the number of vulnerabilities is no less than 300,000, which makes it more convenient to timely discover the risks of using open source components.

[0038] The data persistence service unit is used to cache data and provide data support for statistical analysis and report export based on the cached data. As the basic database of the business platform service, it maintains the customer's project and application asset data.

[0039] As a technical optimization solution of the present invention, the knowledge base service unit includes a data interface module, a data verification module, a data cleaning module, a data update module and a data collection module;

[0040] The data collection module can be used to collect third-party software information, including the original version information, original version release time, latest version information, latest version release time and CVE public vulnerability information contained in the current version of the third-party software. The data update module is used to collect third-party software information and update the data. The data cleaning module cleans the data updated by the data update module and the data verification module verifies the cleaned data. The final verified data is transmitted to the business platform management service unit and the detection engine service unit through the data interface. The third-party software is open source software.

[0041] As a technical optimization solution of the present invention, the detection engine service unit includes a package manager analysis module, a source code tracing analysis module, a multi-link component dependency analysis module, a product binary component analysis module and a security vulnerability reachability analysis module, and performs detection and analysis on open source components based on the package manager analysis module, the source code tracing analysis module, the multi-link component dependency analysis module, the product binary component analysis module and the security vulnerability reachability analysis module.

[0042] The detection engine service unit can achieve the following functions based on the included modules:

[0043] Open source component vulnerability security detection: supports security detection of applications that rely on third-party software, can identify CVE public vulnerabilities contained in the current version of third-party software, and can provide repair solutions for current vulnerability problems, including upgrading recommended versions, patch strategies, etc., to quickly and accurately locate risks.

[0044] Open source component license compliance detection: Supports compliance analysis of license information associated with projects and applications, and provides risk level identification definitions for licenses, including mainstream GPL, Apache, MIT, and domestic Mulan licenses. Through identification, you can quickly understand the rights and obligations of the license, as well as the precautions of the license, providing an effective basis for judging license compliance.

[0045] Component and vulnerability reachability detection: helps R&D and security personnel locate the trigger position more accurately when analyzing the results, and clearly understand the entire component and vulnerability method call chain. By presenting this complete link view, the problem-solving efficiency of developers and security personnel can be improved.

[0046] Open source component dependency detection: In the process of introducing third-party components, there are dependencies between components. In the actual security development process, it is found that there are often a large number of indirectly introduced risk components in the project. To address this problem, SourceCheck provides accurate identification of dependencies between components based on build execution, configuration analysis, header file analysis and other technologies, and correctly presents the relationship graph information between components.

[0047] Open source component tampering identification detection: In the process of using open source components, some components are tampered with by humans. These tampered components often still use the original open source component names during use, causing great security troubles to users and other security personnel. In response to the current tampering risks, SourceCheck provides tampering identification security detection, which can quickly and automatically identify tampered components and help users discover their potential risk issues.

[0048] Identification and detection of poisoned open source components: Component poisoning is a very troublesome problem for users because it is extremely secretive. It is difficult for users to discover risks from the appearance of the component and during use, which leads to the introduction of poisoned components during business development, thus bringing extremely high business use risks. SourceCheck obtains component poisoning information from the community and regulatory agencies, builds a poisoned component library, and combines it with a self-developed poisoning detection engine to accurately identify poisoned components based on their characteristics, thereby improving the business users' ability to protect against supply chain poisoning.

[0049] Source code sensitive information detection: Sensitive information leakage is the most common security issue faced by developers today. During the coding process, in order to better test the code, some parameter values ​​are often statically assigned, resulting in a large amount of static sensitive information in the code, such as URLs, email addresses, IP addresses, user names, ID cards, etc. This information often provides attackers with a large amount of attack information, causing significant losses to corporate organizations. SourceCheck supports the analysis of sensitive information in source code packages, which can quickly help customers discover the risk of sensitive information leakage and avoid significant impacts and losses.

[0050] Open source code traceability detection: In the process of introducing open source, in addition to the conventional component-level open source introduction, developers will also directly use the open source code in the open source project. In the process of introduction, they often inadvertently introduce unsafe content with vulnerability risks or license compliance risks. To address this more hidden risk introduction, SourceCheck provides code-level, file-level, and component-level code traceability analysis functions to achieve component source traceability of source code packages and self-review of knowledge products.

[0051] As a technical optimization solution of the present invention, the package manager analysis module scans the binary import packages in the configuration files and application packages of the target detection object to obtain the component information directly introduced and the component information indirectly introduced, and obtains the complete component information by matching and identifying the content in the knowledge base service unit. It can realize component behaviors such as the identification of modified components, dependency relationships between components, and self-developed component settings.

[0052] As a technical optimization solution of the present invention, the source code tracing analysis module collects open source code and indexes the collected open source source code, so that the source file and the target file are merged, and is used to detect whether there are redundant and repeated codes in the own code and to detect the code duplication between the source code and the target code. It not only supports comparison between files, but also has a complete unauthorized open source code library, and can compare the information of the application code and the open source code library, with higher coverage and higher accuracy.

[0053] As a technical optimization solution of the present invention, the multi-link component dependency analysis module obtains complete component information based on the package manager analysis module to simulate the process tracking and positioning of component dependencies, verifies the data information of the dependency tree with the collected open source data, and supplements the missing information of the dependency tree data to obtain the final dependency data.

[0054] As a technical optimization solution of the present invention, the product binary component analysis module recursively extracts the content of the binary product file to be detected, identifies the format through the file format recognition engine, calls the feature extraction engine for files of different formats respectively, extracts the feature information of the binary product file, and matches the open source component feature library through graph similarity, function similarity or semantic similarity algorithms to determine the name and version of the introduced open source component, and obtains the license and vulnerability risk information of the open source component in combination with the data support of the knowledge base service unit.

[0055] As a technical optimization solution of the present invention, the security vulnerability reachability analysis module extracts AST information from the source code uploaded by the user through analysis based on AST (abstract syntax tree) and call analysis methods, obtains the function call chain information of the source code, and downloads the source code of various versions of the open source project from the mainstream open source repository, extracts the methods in the source code, stores the extracted function information in the open source component knowledge base, and establishes an association relationship with the open source component to form an open source function library. Based on the analysis of the call chain and call location, more accurate repair information is provided to the R&D and security departments.

[0056] As a technical optimization solution of the present invention, the extraction of binary features simultaneously extracts sensitive information in the binary product file to generate a sensitive information list.

[0057] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A scanning platform based on software component analysis, characterized in that: It includes knowledge base service unit, detection engine service unit, business platform management service unit and data persistence service unit; The business platform management service unit is used to provide application management, asset management, user role management, security management, plug-in management, detection report, statistical analysis, SBOM, private server firewall and fast scanning functions; The detection engine service unit is used to provide detection support for the business platform management service unit through an API; The knowledge base service unit is used to provide data to the business platform management service unit and the detection engine service unit; The data persistence service unit is used to cache data and provide data support for statistical analysis and report export based on the cached data.

2. The scanning platform based on software component analysis according to claim 1, characterized in that: The knowledge base service unit includes a data interface module, a data verification module, a data cleaning module, a data update module and a data collection module; The data collection module can be used to collect third-party software information, which includes the original version information of the third-party software, the release time of the original version, the latest version information, the release time of the latest version, and the CVE public vulnerability information contained in the current version. The data update module is used to collect third-party software information and update the data. The data cleaning module cleans the data updated by the data update module and the data verification module verifies the cleaned data. Finally, the verified data is transmitted to the business platform management service unit and the detection engine service unit through the data interface.

3. The scanning platform based on software component analysis according to claim 1, characterized in that: The detection engine service unit includes a package manager analysis module, a source code tracing analysis module, a multi-link component dependency analysis module, a product binary component analysis module and a security vulnerability reachability analysis module, and performs detection and analysis on open source components based on the package manager analysis module, the source code tracing analysis module, the multi-link component dependency analysis module, the product binary component analysis module and the security vulnerability reachability analysis module.

4. The scanning platform based on software component analysis according to claim 3, characterized in that: The package manager analysis module scans the binary import packages in the configuration files and application packages of the target detection object to obtain the directly imported component information and the indirectly imported component information, and obtains the complete component information by matching the identification with the content in the knowledge base service unit.

5. The scanning platform based on software component analysis according to claim 4, characterized in that: The source code traceability analysis module collects open source code and indexes the collected open source source code, so that the source file and the target file are merged, and is used to detect whether there is redundant and repeated code in the own code and detect code duplication between the source code and the target code.

6. The scanning platform based on software component analysis according to claim 5, characterized in that: The multi-link component dependency analysis module obtains complete component information based on the package manager analysis module to simulate the process tracking and positioning of component dependencies, verifies the data information of the dependency tree with the collected open source data, and supplements the missing information of the dependency tree data to obtain the final dependency data.

7. The scanning platform based on software component analysis according to claim 6, characterized in that: The product binary component analysis module recursively extracts the content of the binary product file to be detected, identifies the format through the file format recognition engine, calls the feature extraction engine for files of different formats respectively, extracts the feature information of the binary product file, and matches it with the open source component feature library through graph similarity, function similarity or semantic similarity algorithms to determine the name and version of the introduced open source component, and obtains the license and vulnerability risk information of the open source component in combination with the data support of the knowledge base service unit.

8. The scanning platform based on software component analysis according to claim 7, characterized in that: The security vulnerability reachability analysis module uses AST-based and call analysis methods to extract AST information from the source code uploaded by the user, obtain the function call chain information of the source code, and download the source code of each version of the open source project from the mainstream open source repository, extract the methods in the source code, store the extracted function information in the open source component knowledge base, and establish an association relationship with the open source component to form an open source function library.

9. The scanning platform based on software component analysis according to claim 7, characterized in that: The binary features are extracted while sensitive information in the binary artifact file is extracted to generate a sensitive information list.

Images