Defect early warning method and device based on multi-source data and electronic equipment

Abstract

The invention provides a defect early warning method and device based on multi-source data and electronic device.The method comprises the steps that defect information is captured from multiple data sources, feature coding and feature fusion are conducted on the multiple pieces of defect information, multiple pieces of first defect data are obtained, and the data sources comprise suppliers, security manufacturers, open source communities and source code libraries; traversing software assets in the system, generating an asset atlas according to the software assets, and constructing an abstract syntax tree according to the asset atlas, the abstract syntax tree being used for representing a dependency relationship among components in the software assets; matching the first defect data with the software assets based on the abstract syntax tree, and calculating defect risk scores of the components according to matching results hit by matching; and generating alarm information according to the defect risk score and sending the alarm information to the target terminal. The method solves the problems that in the prior art, data elements of a single official defect library are single, updating lags behind, and the defect detection coverage rate is insufficient when the method is applied to emerging open source ecology.

Description

Technical Field

[0001] This invention relates to the field of data security technology, and more specifically, to a defect early warning method, apparatus, computer-readable storage medium, and electronic device based on multi-source data. Background Technology

[0002] In existing technologies, the open-source model has become a common approach. By reusing third-party software packages, developers can accelerate development, reduce costs, and overcome the bottleneck of traditional software development models in terms of update and iteration speed. With the accelerated digital transformation of the financial industry, the reliance of various business systems in banks, securities firms, and other financial institutions on third-party components is constantly increasing. However, while third-party components bring convenience to developers, they also introduce security risks to financial institutions—supply chain defects. These defects can spread through the supply chain network formed between the system and the third-party software packages.

[0003] To address the aforementioned issues, existing technologies typically rely on official defect databases as the single authoritative source. However, official defect data updates are delayed and lack sufficient coverage of emerging open-source ecosystems, resulting in defect warning vacuum periods and insufficient warning coverage. Summary of the Invention

[0004] The main objective of this application is to provide a defect early warning method, device, computer-readable storage medium, and electronic device based on multi-source data, so as to at least solve the problem that the existing technology uses a single official defect library with limited and outdated data elements, resulting in insufficient defect detection coverage when applied to emerging open source ecosystems.

[0005] To achieve the above objectives, according to one aspect of this application, a defect early warning method based on multi-source data is provided, comprising: extracting defect information from multiple data sources respectively, and performing feature encoding and feature fusion on the multiple defect information to obtain multiple first defect data, wherein the data sources include suppliers, security vendors, open source communities, and source code repositories; traversing the software assets within the system, generating a first asset graph based on the software assets, constructing an abstract syntax tree based on the first asset graph, and adding the abstract syntax tree to the first asset graph to obtain a second asset graph, wherein the abstract syntax tree is used to represent the dependencies between components in the software assets; matching each first defect data with the software assets based on the second asset graph, calculating a defect risk score for each component based on the matching results; generating alarm information based on the defect risk score and sending it to a target terminal.

[0006] Optionally, defect information is retrieved from multiple data sources, including: obtaining the historical update frequency of each data source and determining a corresponding update frequency score based on the historical update frequency; obtaining API restriction data of each data source and determining a corresponding response efficiency score based on the API restriction data; obtaining a reliability score and a contribution score corresponding to each data source; determining a corresponding information acquisition frequency based on the update frequency score, the response efficiency score, the reliability score, and the contribution score, wherein the information acquisition frequency is directly proportional to the update frequency score and the contribution score, and inversely proportional to the response efficiency and the reliability score; and retrieving defect information from the corresponding data source based on the information acquisition frequency.

[0007] Optionally, feature encoding and feature fusion are performed on multiple defect information to obtain first defect data, including: extracting entities from the defect information using natural language processing to obtain second defect data, wherein the entities include defect-affected components, version numbers, defect codes, and repair submission numbers; traversing the second defect data to determine whether there are duplicate entities between any two pairs of second defect data; if there are duplicate entities between any two pairs of second defect data, associating the second defect data to obtain third defect data; and encoding each second defect data and the third defect data to obtain the corresponding first defect data.

[0008] Optionally, traversing the software assets within the system, generating an asset graph based on the software assets, and constructing an abstract syntax tree based on the asset graph includes: traversing all component information of each software project or application in the system to obtain the software assets, wherein the component information includes component name, version number, and hash value; determining the application to which each component information belongs to obtain multiple target applications, determining the server to which each target application belongs to obtain target servers; constructing association links in the form of component-target application-server, and associating each association link with the software assets to obtain the asset graph; and performing a full scan of the source code of each component in the asset graph using fine-grained dependency analysis technology, and constructing the dependency relationships between the source code in units of functions based on the scan results to obtain the abstract syntax tree.

[0009] Optionally, matching each of the first defect data and the software assets based on the second asset graph includes: extracting the code path and function of each code segment from the first defect data to obtain multiple code identifiers; extracting the code pattern of each code segment from the first defect data; traversing each first node in the abstract syntax tree based on the code identifiers; determining the component corresponding to the first node as a first target component when a match is found; determining the first node as a second node when no match is found; retrieving the second node using a search algorithm based on the code pattern; determining the component corresponding to the second node as a second target component when a retrieval is found; the search algorithm includes depth-first search and breadth-first search; and generating the matching result based on the first target component and the second target component.

[0010] Optionally, generating the matching result based on the first target component and the second target component includes: determining whether the first version number of the first target component is within the range of influence version numbers corresponding to the code segment; if the first version number is not within the range of influence version numbers, deleting the first target component; determining whether the second version number of the second target component is within the range of influence version numbers corresponding to the code segment; if the second version number is not within the range of influence version numbers, deleting the second target component; determining the function call chains of the first target component and the second target component in the abstract syntax tree, respectively, and identifying the components covered by the function call chains as the third target component; performing context analysis on the first target component, the second target component, and the third target component based on the asset graph to determine the availability scores corresponding to the first target component, the second target component, and the third target component; if the availability score is less than a first threshold, deleting the corresponding first target component, the second target component, and the third target component to obtain the matching result.

[0011] Optionally, the defect risk score of each component is calculated based on the matching results, including: determining the availability weight of the component based on the exploit cases corresponding to the components that have been matched in each of the data sources; determining the asset importance weight of the component based on the application environment, application services, and network exposure surface of the components that have been matched; determining the risk threat weight of the component based on the application frequency and impact scope of the exploit cases; calculating a basic risk score through CVSS based on the matching results; and correcting the basic risk score based on the availability weight, the asset importance weight, and the risk threat weight to obtain the defect risk score.

[0012] According to another aspect of this application, a defect early warning device based on multi-source data is provided. The device includes: an acquisition unit, configured to capture defect information from multiple data sources respectively, and perform feature encoding and feature fusion on the multiple defect information to obtain multiple first defect data, wherein the data sources include suppliers, security vendors, open source communities, and source code libraries; a first processing unit, configured to traverse the software assets within the system, generate an asset map based on the software assets, and construct an abstract syntax tree based on the asset map, wherein the abstract syntax tree is used to represent the dependencies between components in the software assets; a second processing unit, configured to match each of the first defect data and the software assets based on the abstract syntax tree, and calculate a defect risk score for each component based on the matching results; and a generation unit, configured to generate alarm information based on the defect risk score and send it to a target terminal.

[0013] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform any of the methods described.

[0014] According to another aspect of this application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing any one of the methods described.

[0015] Applying the technical solution of this application, in the aforementioned defect early warning method based on multi-source data, firstly, defect information is captured from multiple data sources, and feature encoding and feature fusion are performed on these multiple defect information sources to obtain multiple first defect data. The data sources include suppliers, security vendors, open-source communities, and source code repositories. Then, the software assets within the system are traversed, and a first asset graph is generated based on the software assets. An abstract syntax tree (AST) is constructed based on the first asset graph and added to the first asset graph to obtain a second asset graph. The AST is used to represent the dependencies between components in the software assets. Next, each first defect data and software asset is matched based on the second asset graph, and the defect risk score of each component is calculated based on the matching results. Finally, an alarm message is generated based on the defect risk score and sent to the target terminal. This application introduces multi-source data fusion, especially the collection of original project data from open-source communities. Compared to the traditional single official defect library, this shortens the blind spot and time window for defect discovery and improves the real-time performance of defect perception, thus solving the problem of insufficient defect detection coverage when applied to emerging open-source ecosystems due to the single official defect library's limited data elements and delayed updates in existing technologies. Attached Figure Description

[0016] Figure 1 A hardware structure block diagram of a mobile terminal for a defect early warning method based on multi-source data provided in an embodiment of this application is shown.

[0017] Figure 2 A flowchart illustrating a defect early warning method based on multi-source data according to an embodiment of this application is shown.

[0018] Figure 3 A structural block diagram of a defect early warning device based on multi-source data provided according to an embodiment of this application is shown.

[0019] The above figures include the following reference numerals:

[0020] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] As described in the background section, existing technologies typically rely on official defect databases as a single authoritative database. However, official defect data updates are delayed and lack coverage of emerging open-source ecosystems, resulting in defect warning vacuum periods and insufficient warning coverage. To address the problem of insufficient defect detection coverage when applying to emerging open-source ecosystems due to the single and delayed update of data elements in existing technologies using a single official defect database, embodiments of this application provide a defect warning method, apparatus, computer-readable storage medium, and electronic device based on multi-source data.

[0025] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0026] The methods and embodiments provided in this application can be executed on a mobile terminal, a computer terminal, or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a defect early warning method based on multi-source data, according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0027] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the defect early warning method based on multi-source data in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0028] This embodiment provides a defect warning method based on multi-source data that runs on a mobile terminal, computer terminal or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although the logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0029] Figure 2 This is a flowchart of a defect early warning method based on multi-source data according to an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0030] Step S201: Extract defect information from multiple data sources, and perform feature encoding and feature fusion on the multiple defect information to obtain multiple first defect data. The data sources include suppliers, security vendors, open source communities and source code repositories.

[0031] Understandably, multi-source data acquisition relies on distributed processing and API call technologies to extract defect information from various sources such as vendors, security companies, open-source communities, and source code repositories. Feature encoding and feature fusion, on the other hand, use Natural Language Processing (NLP) technology to convert unstructured defect descriptions into structured feature vectors. Then, through data fusion algorithms, these feature vectors from different sources are integrated into comprehensive primary defect data, improving the accuracy and completeness of the information.

[0032] Specifically, taking open-source communities (such as GitHub) as an example, issues, pull requests, and commit information of projects are obtained through API calls. NLP technology is used to identify key information mentioned in these texts, such as component names, version numbers, defect descriptions, and fix commit IDs, and this information is encoded into feature vectors. Intelligence sources from security vendors (such as blogs) may contain more detailed defect analyses, proof-of-concept (POC), or exploit code; this information is also transformed and integrated into the primary defect data. Official defect repositories have relatively standardized data, but updates may be lagging; their information is used to verify and standardize data collected from other sources.

[0033] Through the above operations, the system can obtain more comprehensive defect information, including not only officially confirmed defects but also potential risks discovered early on by the open-source community and security researchers. Feature encoding and fusion ensure the standardization and consistency of information, facilitating subsequent processing and decision-making.

[0034] Step S202: Traverse the software assets within the system, generate a first asset graph based on the software assets, construct an abstract syntax tree based on the first asset graph, add the abstract syntax tree to the first asset graph to obtain a second asset graph. The abstract syntax tree is used to represent the dependency relationships between components in the software assets.

[0035] Understandably, asset mapping relies on automated tools and code scanning technologies. Integration with CI / CD pipelines, code repositories, artifact libraries, and cloud-native platforms allows for the automated tracking and identification of all internal software assets, including the components used, versions, and license information. The construction of the Abstract Syntax Tree (AST) involves source code analysis to understand the dependencies between code at the function level, providing a foundation for fine-grained risk assessment and decision-making.

[0036] Specifically, the system traverses the entire codebase within the enterprise, uses static code analysis tools to generate an Abstract Syntax Tree (AST), and identifies all functions called in the code and their dependencies. This information is integrated into an asset graph, forming a multi-level network of "component-application-host". For example, it identifies that a bank's transaction system depends on version 2.1 of an open-source payment processing library, and that this library is directly called in the transaction confirmation module.

[0037] The above operations not only provide a visual representation of software assets, but also clarify the usage context of components, which helps to accurately locate the scope of defect impact and assess defect risks, reducing false positives and false negatives.

[0038] Step S203: Match each first defect data and software asset based on the second asset map, and calculate the defect risk score of each component based on the matching results.

[0039] Understandably, the risk score calculation in this application is based on the CVSS baseline score and custom risk factors (availability factor α, asset importance factor β, and threat intelligence factor γ). Through fine-grained code dependency analysis, the system can more accurately assess the actual risk of defects in a specific environment, rather than just the theoretical risk.

[0040] Specifically, for each primary vulnerability, the system will match the affected components using asset graphs and ASTs. For example, if a vulnerability with a baseline CVSS score of 8.0 is found, and exploit code for that vulnerability is publicly available on GitHub, the exploitability factor α might be set to 2.5. If this vulnerability affects a bank's core transaction system, the asset importance factor β might be 4.0. If the threat intelligence platform indicates that the vulnerability has been exploited globally by APT groups, the threat intelligence factor γ might be 2.0. Based on this information, the environmental risk score calculated by the system will be significantly higher than the traditional CVSS score, thus prioritizing the handling of this vulnerability.

[0041] By implementing the above procedures, we can ensure more accurate early warning and remediation decisions regarding defects, avoid resource waste, and improve the efficiency of safe operations. Furthermore, by introducing context-sensitive risk factors, risk scoring becomes more aligned with the company's actual situation, facilitating rapid response to high-risk defects.

[0042] Step S204: Generate alarm information based on the defect risk score and send it to the target terminal.

[0043] Understandably, automated alerting and workflow management integrates internal enterprise communication software, email systems, SMS services, and project management tools to achieve unified management from defect discovery to remediation tracking. Alert information is automatically distributed based on risk scores, ensuring that critical information is delivered to relevant teams and personnel in a timely and accurate manner.

[0044] Understandably, when the system calculates that a component's environmental risk score exceeds a preset threshold, it will immediately generate an alert and send it to the security team via the company's internal instant messaging software, while simultaneously notifying the development and operations teams via email. The alert includes defect details, a list of affected assets, a risk score, and preliminary remediation suggestions. Upon receiving the alert, the security team can automatically create a work order in the project management tool, assign it to developers for remediation, and set a deadline for remediation.

[0045] These steps significantly shortened the time from defect discovery to response, improved repair efficiency, and reduced security risks caused by untimely defect handling. Simultaneously, automated workflow management reduced the need for manual collaboration, lowered management costs, and enhanced team collaboration efficiency.

[0046] In this embodiment, firstly, defect information is captured from multiple data sources, and feature encoding and feature fusion are performed on these multiple defect information sources to obtain multiple first defect data. The data sources include suppliers, security vendors, open-source communities, and source code repositories. Then, the software assets within the system are traversed, and a first asset graph is generated based on these software assets. An abstract syntax tree (AST) is constructed based on the first asset graph and added to the first asset graph to obtain a second asset graph. The AST is used to represent the dependencies between components in the software assets. Next, each first defect data and software asset is matched based on the second asset graph, and the defect risk score of each component is calculated based on the matching results. Finally, an alarm message is generated based on the defect risk score and sent to the target terminal. This application introduces multi-source data fusion, especially collecting original project data from open-source communities. Compared to the traditional single official defect library, this shortens the blind spot and time window for defect discovery, improves the real-time performance of defect awareness, and solves the problem of insufficient defect detection coverage when applied to emerging open-source ecosystems due to the single official defect library's limited data elements and delayed updates.

[0047] In order to obtain the aforementioned defect information from multiple data sources, in one optional implementation, step S201 includes:

[0048] Step S2011: Obtain the historical update frequency of each data source and determine the corresponding update frequency score based on each historical update frequency.

[0049] Understandably, the update frequency of a data source directly affects the timeliness and freshness of defect information. The update frequency score aims to quantify this characteristic so that the system can prioritize collecting information from data sources with high update frequencies and achieve real-time response to the latest defects.

[0050] Through the above embodiments, the system analyzes the update records of each data source over a period of time to calculate its average daily update frequency (this can be extended to monthly and yearly data sources with fewer than one update per day). For example, official bug repositories may have fewer average daily updates due to their strict update processes, while open-source communities like GitHub may have dozens, or even more, average daily updates due to the high frequency of new bug discoveries and reports. Based on the update frequency, the system will assign a score; the higher the frequency, the higher the score.

[0051] Through the above operations and by updating the frequency score, the system can identify which data sources provide the latest and most frequent defect information, and thus prioritize data collection from these sources to ensure the real-time nature and effectiveness of defect warnings.

[0052] Step S2012: Obtain API restriction data from each data source and determine the corresponding response efficiency score based on the API restriction data.

[0053] Understandably, API limitation data reflects the degree of restriction imposed by the data source on requests, while response efficiency scores aim to assess the efficiency constraints that a system may encounter when retrieving information from different data sources. A high response efficiency score means that a higher request frequency will be limited, requiring corresponding adjustments to the system's data collection strategy.

[0054] Specifically, the system analyzes the API documentation of each data source to extract information such as request rate limits, request quotas, and time intervals between requests. For example, GitHub's API may limit a maximum of 100 requests per minute. Based on the degree of limitation, the system assigns a response efficiency score; the stricter the limitation, the lower the score.

[0055] Through the above operations, the response efficiency score helps the system understand the efficiency obstacles that may be encountered when collecting information from different data sources, so as to take these limitations into account in the collection strategy, avoid triggering rate limiting due to frequent requests, and ensure the continuity and stability of information acquisition.

[0056] Step S2013: Obtain the reliability score and contribution score corresponding to each data source;

[0057] Understandably, the reliability score focuses on the accuracy and authority of the data source information, while the contribution score focuses on the degree to which the data source contributes to the bank's specific business scenarios. These two scores are used to determine the weight of the data source in the overall decision-making process, ensuring that information is obtained from authoritative, reliable data sources that have a significant impact on the business.

[0058] Specifically, scoring is based on the accuracy of historical data, the level of detail in the information, and the endorsement of official institutions. Scoring is also based on the degree of relevance of the data source information to the bank's internal assets and business scenarios.

[0059] Through the above operations, the system can identify which data sources are more valuable in the banking scenario by using reliability and contribution scores, and thus give them higher weight in the collection strategy to ensure that the information obtained is both accurate and relevant.

[0060] Step S2014: Determine the corresponding information acquisition frequency based on each update frequency score, response efficiency score, reliability score, and contribution score. The information acquisition frequency is directly proportional to the update frequency score and the contribution score, and inversely proportional to the response efficiency score and the reliability score.

[0061] Understandably, the information acquisition frequency takes into account update frequency score, response efficiency score, reliability score and contribution score to ensure that information is collected from different data sources in a way that is both efficient and reasonable.

[0062] The above operations ensure that the system can adjust the information collection frequency according to the characteristics of the data source, avoiding both inefficient or wasteful collection due to excessive frequency and missed key information due to insufficient collection frequency. By adjusting the collection frequency, the system can operate more efficiently while maintaining the comprehensiveness and timeliness of the information.

[0063] Step S2015: Extract defect information from the corresponding data source based on the information acquisition frequency.

[0064] Through the above embodiments, the system can intelligently acquire defect information from multiple data sources and dynamically adjust the frequency of information acquisition by quantitatively analyzing the characteristics of different data sources. This not only ensures the timeliness, comprehensiveness, and accuracy of the acquired information but also avoids efficiency issues caused by frequent requests, achieving optimal allocation of system resources. By prioritizing high-frequency and high-contribution data source information, the system can more accurately provide early warnings of open-source defects, building an efficient, real-time, and reliable full lifecycle management mechanism for open-source defects for banks, significantly improving their ability to cope with software supply chain risks.

[0065] In order to convert defect information into standardized data, in one optional implementation, step S201 above includes:

[0066] Step S2016: Natural language processing is used to extract entities from the defect information to obtain the second defect data. The entities include the defect-affected components, version number, defect code, and repair submission number.

[0067] Understandably, Natural Language Processing (NLP) techniques, particularly text information extraction, are used to automatically identify and extract key entity information from unstructured defect description text, such as the names of affected components, version numbers, defect codes (e.g., CVE numbers), and fix submission numbers. This step transforms the raw, natural language-expressed defect information into a structured, machine-readable data format, facilitating subsequent data fusion and processing.

[0068] Specifically, entity recognition is performed using NLP tools (such as spaCy or NLTK). These tools, through well-trained models, can effectively parse named entities in text, including but not limited to organization names, software names, numbers (for version identification), and dates. In detail, for each captured defect message, the NLP tool analyzes its descriptive text, identifies and tags the aforementioned entity information, and then extracts and stores this information as structured secondary defect data.

[0069] Through the above operations and NLP technology, the system can automatically process defect information from multiple data sources, identify key entities without human intervention, greatly improve the speed and efficiency of information processing, and reduce the possibility of human error.

[0070] Step S2017: Traverse the second defect data and determine whether there are duplicate entities between pairs of the second defect data;

[0071] Understandably, by comparing the entity information extracted from the second set of defect data, different data records describing the same defect can be identified, thereby reducing duplicate data and improving data quality. This is a crucial step in data cleaning and deduplication, ensuring that the data processed subsequently is consistent and unique.

[0072] Specifically, the system will perform pairwise comparisons on all the second defect data to check if there are any completely identical combinations of entities (i.e., the affected component, version number, defect code, and fix submission number are all exactly the same). If such a match is found, the system will consider that the two defect information entries describe the same defect, and thus perform data fusion.

[0073] The above operations greatly streamlined the data, removed redundancy and duplication, avoided potential confusion in subsequent processing, and improved the accuracy and efficiency of the entire system.

[0074] Step S2018: If there are duplicate entities between any two second defect data, associate the second defect data to obtain the third defect data;

[0075] Understandably, fusing second-order defect data containing duplicate entities can build a more complete defect profile, ensuring that each defect has the most detailed information possible.

[0076] Specifically, for duplicate entities found, the system will merge the relevant description information, fix submission numbers, and other details to create a third defect data set containing all relevant information. If a defect has multiple fixes, the system will also record the submission numbers of these fixes to facilitate subsequent tracking and verification of whether all fixes have been applied.

[0077] By performing the above operations and integrating duplicate defect information, a more comprehensive and accurate defect description is obtained, reducing potential information loss or inconsistency in subsequent processing and enhancing the system's ability to identify and track defects.

[0078] Step S2019: Encode each second defect data and third defect data to obtain the corresponding first defect data.

[0079] It is understandable that converting the second and third defect data into a machine-readable, standardized encoding format would facilitate subsequent automated processing and analysis.

[0080] Specifically, the system uses predefined encoding rules to transform the entity information in the second and third defect data into a unified encoding format, forming the first defect data. Encoding rules may include converting component names into internal identifiers, version number normalization, defect coding, and formatting of fix submission numbers.

[0081] Through the above operations, standardized coding not only simplifies the subsequent processing flow, but also ensures the consistency and traceability of the system when processing large amounts of data, providing a solid foundation for subsequent risk assessment and alarm generation.

[0082] Through the above embodiments, by employing NLP technology for entity extraction, data deduplication and fusion, and code standardization, the system can efficiently and accurately process defect information from multiple data sources, constructing high-quality primary defect data. This not only improves the timeliness and accuracy of defect warnings but also simplifies the data processing workflow and reduces subsequent workload. The standardized coding format enables the system to better interface with internal asset maps and other modules, achieving closed-loop management from defect perception to remediation tracking. This significantly enhances the control and response speed of open-source defects, providing enterprises with stronger security guarantees.

[0083] In order to compile the above abstract syntax tree, in an optional implementation, step S202 includes:

[0084] Step S2021: Traverse all component information of each software project or application in the system to obtain software assets. Component information includes component name, version number and hash value.

[0085] Understandably, the automated generation of the Software Bill of Materials (SBOM) involves integrating CI / CD tools, code repositories, artifact libraries, and cloud-native platforms to scan all software projects and applications within the system, identifying the components they depend on and their version information. This step forms the basis for building an asset map.

[0086] Specifically, the system obtains dependency information during the build process by calling the APIs of CI / CD tools (such as Jenkins and GitLab CI) or using plugins. Simultaneously, it reads project metadata from code repositories (such as GitHub and GitLab), including component names, version numbers, and hash values, and retrieves information about the actually deployed components from artifact repositories (such as Nexus and Artifactory). In cloud-native environments, the system also needs to call container image scanning tools, such as Clair and Trivy, to analyze the dependent components in the container images.

[0087] Through the above operations, a comprehensive identification of internal software assets was achieved, and an SBOM was generated, providing a detailed data foundation for subsequent fine-grained analysis and risk assessment.

[0088] Step S2022: Determine the application to which the component information belongs, obtain multiple target applications, determine the server to which each target application belongs, and obtain the target server;

[0089] Understandably, analyzing the SBOM and system architecture helps identify multi-level relationships between components, applications, and servers, thus constructing the relational links in the asset graph. This aids in identifying the actual location and usage of components within a specific environment.

[0090] Specifically, for each identified component, the system determines its application by analyzing the metadata of the code repository and artifact repository. Then, it obtains the server information where the application resides through the APIs of cloud-native platforms (such as Kubernetes and Docker Swarm).

[0091] By constructing the associated links through the above operations, the system can accurately locate the specific position of the component in the enterprise's technical architecture, increase the contextual information for risk assessment, and help to more accurately assess the potential impact of defects.

[0092] Step S2023: Construct the association links in the form of component-target application-server, and associate each association link with the software assets to obtain the asset map;

[0093] Understandably, an asset map is created by collecting and organizing information about the relationships between components, applications, and servers to form a visual and structured map that shows the overall picture and dependencies of an enterprise's internal software assets.

[0094] Ultimately, the system links the information of components, applications, and servers, forming a "component-application-server" relationship. The system then integrates all identified relationships to construct an asset graph. This asset graph includes not only the component's name and version number but also which applications use the component and on which servers these applications are deployed. The graph can be created and maintained using graph database technologies (such as Neo4j) or data visualization tools (such as Gephi).

[0095] Through the above operations, the generation of the asset map enables the system to intuitively display the distribution and dependencies of software components, facilitating the rapid location and analysis of potential risk points and enhancing risk perception capabilities.

[0096] Step S2024: Use fine-grained dependency analysis technology to perform a full scan of the source code of each component in the asset graph, and construct the dependency relationship between the source code in units of functions based on the scan results to obtain an abstract syntax tree.

[0097] Understandably, fine-grained dependency analysis techniques identify function-level dependencies through deep scanning of source code, thereby constructing an abstract syntax tree (AST). This is a crucial step in improving the accuracy of defect identification and risk assessment.

[0098] Specifically, the system uses static code analysis tools (such as SonarQube and FindBugs) to perform a full scan of the component's source code and generate an abstract syntax tree (AST). The AST not only reflects the call relationships between functions but also annotates detailed information such as function parameter types, return values, and dependencies between functions and external components. The system combines the AST with component information from the asset graph to form a fine-grained dependency network for the components.

[0099] Through the above operations, fine-grained dependency analysis enables the system to more accurately identify which specific functions or code paths may be affected by defects, reducing false positives and false negatives, and improving the pertinence and effectiveness of risk assessment.

[0100] Through the above embodiments, by automatically generating SBOM, constructing asset graphs, and creating fine-grained dependency abstract syntax trees, the method of this invention achieves comprehensive and detailed identification and analysis of internal software assets. This series of steps significantly enhances the system's ability to perceive, identify, and assess risks associated with open-source components, reduces resource waste, and improves the efficiency of security operations. Specifically, the generated asset graphs and abstract syntax trees can help financial institutions such as banks quickly pinpoint the scope of vulnerability impacts and assess their risk levels in specific business scenarios, thereby enabling more accurate and timely security decisions and response actions.

[0101] To automate the inspection of system software assets, in one optional implementation, step S203 includes:

[0102] Step S2031: Extract the code path and function of each code segment from the first defect data to obtain multiple code identifiers, and extract the code pattern of each code segment from the first defect data.

[0103] Understandably, for the initial defect data obtained from multiple data sources and after feature encoding and fusion, it is necessary to further extract specific code paths and function information, recording them in the form of code identifiers. Simultaneously, code patterns are extracted from the defect description, which typically involve the triggering conditions or characteristics of potential defects, such as specific function call sequences or configuration items.

[0104] Specifically, through deep learning models or rule engines, the system can identify key code paths and function names from defect descriptions, which are then converted into code identifiers. For example, if the defect description involves the Spring Security framework's permission check function "hasPermission," the model will extract this information and convert it into an internal system code identifier. Code pattern extraction involves more complex NLP processing, potentially including entity relation extraction and semantic analysis, to identify the contextual conditions that triggered the defect.

[0105] Through the above operations, by extracting code identifiers and code patterns, the system can more accurately understand and locate the details of defects, providing precise information for subsequent fine-grained analysis and matching.

[0106] Step S2032: Based on the code identifier, traverse each first node in the abstract syntax tree. If a match is found, determine the component corresponding to the first node as the first target component.

[0107] Understandably, an Abstract Syntax Tree (AST) contains structural information about the source code within a software asset. By traversing the AST, the system can find nodes that match code identifiers and determine which components may be affected by defects.

[0108] Specifically, the system traverses the AST using either DFS (Depth-First Search) or BFS (Breadth-First Search) algorithms to find nodes that match the code paths and function names in the code identifiers. If a match is found, the component corresponding to that node is identified as the first target component.

[0109] By performing the above operations, the specific functions and code segments within a component that may be affected by defects can be accurately located, greatly improving the accuracy and relevance of defect warnings.

[0110] Step S2033: If no match is found, the first node is determined as the second node;

[0111] Understandably, when a complete match is not found in the AST based solely on code identifiers, the system further searches for potentially affected components through code pattern analysis. This leverages the generalization ability of code patterns to identify issues with similar functionality but different code surface forms.

[0112] Specifically, the system uses either Depth-First Search (DFS) or Breadth-First Search (BFS) algorithms to find code segments in the Abstract Syntax Tree (AST) that match the code pattern. For example, if the code pattern involves unauthorized access, the system will search for all code paths related to permission verification, not just specific function calls.

[0113] Through the above operations and code pattern search, the system can discover components that are not directly identified but have similar functional vulnerabilities, thereby reducing false negatives and improving the comprehensive coverage of risk assessment.

[0114] Step S2034: Based on the code pattern, a search algorithm is used to retrieve the second node. If the retrieval is successful, the component corresponding to the second node is determined as the second target component. The search algorithm includes depth-first search and breadth-first search.

[0115] Understandably, when a complete match is not found in the AST based solely on code identifiers, the system further searches for potentially affected components through code pattern analysis. This leverages the generalization ability of code patterns to identify issues with similar functionality but different code surface forms.

[0116] Specifically, the system uses either Depth-First Search (DFS) or Breadth-First Search (BFS) algorithms to find code segments in the Abstract Syntax Tree (AST) that match the code pattern. For example, if the code pattern involves unauthorized access, the system will search for all code paths related to permission verification, not just specific function calls.

[0117] Through the above operations and code pattern search, the system can discover components that are not directly identified but have similar functional vulnerabilities, thereby reducing false negatives and improving the comprehensive coverage of risk assessment.

[0118] Step S2035: Generate a matching result based on the first target component and the second target component.

[0119] Understandably, by comparing the code identifiers and code patterns in the first defect data with the AST of the internal software assets, the system generates a clear matching result, namely which components are affected by the defect, as well as the specific code location and potential risk level of the affected components.

[0120] Specifically, the system lists all matching first and second target components, generating matching results. The results include detailed information for each component, such as component name, version number, specific affected code paths and functions, and a risk score calculated based on the CVSS score and custom risk factors.

[0121] The above operations generate matching results that provide security teams and developers with clear action guidelines, helping them quickly locate and assess the severity of vulnerabilities, and thus take timely remedial measures.

[0122] The above embodiments enable fine-grained identification and accurate early warning of open-source defects. By extracting code identifiers and patterns from the initial defect data and matching them with the AST of internal software assets, the system can pinpoint specific code segments within specific components, significantly improving the accuracy of defect identification and reducing false positives and false negatives. Simultaneously, based on the fine-grained matching results, the system can generate more accurate risk scores, helping banks manage security risks in the open-source supply chain more efficiently and enhancing their security during digital transformation. Overall, this method improves banks' security control capabilities over open-source components and contributes to building a real-time, comprehensive, and intelligent open-source defect lifecycle management mechanism.

[0123] To further improve the accuracy of the matching results, in one optional implementation, step S2035 includes:

[0124] Step S20351: Determine whether the first version number of the first target component is within the range of affected version numbers of the corresponding code segment. If the first version number is not within the range of affected version numbers, delete the first target component.

[0125] Understandably, by comparing the version numbers of the affected components listed in the defect information with the actual version numbers of the software assets within the system, the target components truly affected by the defect can be identified. This is a crucial step in ensuring the accuracy of alerts and avoiding false alarms.

[0126] Specifically, the system compares the version numbers of the affected components obtained from the first defect data with the version numbers of each component recorded in the asset map. If the version number of an internal component matches the version number range in the defect description, the component will be marked as a target component for further analysis; otherwise, the component will be removed from the candidate list.

[0127] The above operations effectively filter out unaffected components, reduce unnecessary analysis work in subsequent steps, and improve overall alarm accuracy and system efficiency.

[0128] Step S20352: Determine whether the second version number of the second target component is within the range of affected version numbers of the corresponding code segment. If the second version number is not within the range of affected version numbers, delete the second target component.

[0129] Understandably, once the actual version number of a component is determined, if it is found that the version number is not within the scope of the defect description, the system will automatically delete the relevant records of these components to prevent them from being mistakenly identified as components affected by the risk.

[0130] Specifically, for the first and second target components marked as to be analyzed, if the version number comparison determines that their versions are not within the scope of influence, the system will exclude these components from the subsequent processing flow.

[0131] The above operations ensured the high accuracy of alarm information, avoided unnecessary repair work on unaffected components by the development and operations teams, and saved valuable IT resources.

[0132] Step S20353: Determine the function call chains of the first target component and the second target component in the abstract syntax tree, and determine the component covered by the function call chain as the third target component;

[0133] Understandably, fine-grained dependency analysis based on abstract syntax trees (ASTs) to determine the function call chain within the target component helps to more accurately assess the likelihood and actual risk of defect propagation.

[0134] Specifically, the system traverses every node in the abstract syntax tree, identifying all possible function call paths involved in the first and second target components. This includes direct calls, indirect calls, conditional call paths, etc., ensuring coverage of all possible code execution flows.

[0135] Through the above operations and by determining the fine-grained function call chain, the system can identify the precise path that the defect may propagate, thereby improving the accuracy and relevance of risk assessment.

[0136] Step S20354: Perform contextual analysis on the first target component, the second target component, and the third target component based on the asset map to determine the availability scores corresponding to the first target component, the second target component, and the third target component;

[0137] Understandably, based on factors such as the component's usage environment, call chain complexity, code visibility, and permission level, the system performs contextual analysis on components marked as targets to quantify their actual usability in the enterprise environment, thereby further improving the accuracy and effectiveness of alerts.

[0138] Specifically, the system generates an availability score for each third-party target component, based on factors such as the component's position in the code call chain, its access permissions, and its deployment status in the production environment. The scoring rules can be flexibly adjusted to suit the specific needs and security policies of the enterprise.

[0139] Through the above operations, this mechanism can determine the level of risk of a component based on its specific usage within the enterprise, thereby prioritizing the handling of high-risk components and avoiding the waste of resources on low-risk components.

[0140] Step S20355: If the availability score is less than the first threshold, delete the corresponding first target component, second target component and third target component to obtain the matching result.

[0141] Understandably, by setting an availability score threshold, the system can automatically filter out components that pose a lower actual risk in the enterprise environment, ensuring that alerts are focused on the most urgent and threatening areas.

[0142] Specifically, the system sets a first threshold for availability scoring; for example, a score below 2.0 is considered low-risk and does not require immediate alerting or remediation. All components with scores below the first threshold will be removed from the matching results and will no longer be processed in subsequent alerting processes.

[0143] By taking the above steps, we avoid over-focusing on low-risk components, which helps security teams and developers concentrate their efforts on high-risk components that urgently need fixing, thereby improving response efficiency and resource utilization.

[0144] Through the above embodiments, the system can accurately identify and assess the potential risks of open-source components based on the actual version of the component, function call chain, and context analysis. This method effectively reduces the false alarm rate, improves the accuracy and urgency of alerts, and ensures that alert information focuses on the components that require the most attention. Simultaneously, by introducing fine-grained function call chain analysis and availability scoring, the alert mechanism can better adapt to the specific business environment and security needs of enterprises, improving the enterprise's overall control over open-source supply chain risks. Ultimately, this optimized alert process not only improves the enterprise's security response level but also saves time and resources spent processing inefficient alerts, bringing significant benefits to the enterprise's security operations.

[0145] In order to calculate the above-mentioned defect risk score, in an optional implementation, step S203 further includes:

[0146] Step S2036: Determine the availability weight of the component based on the use cases corresponding to the components that have been matched in each data source;

[0147] Understandably, by analyzing exploitation cases, the likelihood of a component being successfully utilized in reality is assessed, thereby determining its availability weight. This weight reflects the component's vulnerability level when faced with a specific defect.

[0148] Specifically, the system examines exploit cases related to the matching results (such as proof-of-concept (POC) code, exploit tools, real attack events, etc.) and assigns weights based on the specific circumstances of each case. If there is publicly available and verified exploit code, the weight will be higher; conversely, if it is only a theoretical possibility, the weight will be lower.

[0149] By implementing the above steps, companies can better understand the risks they face, prioritize addressing components that are more easily exploited, and thus effectively reduce the probability of security incidents.

[0150] Step S2037: Determine the asset importance weight of the components based on their application environment, application services, and network exposure surface.

[0151] Understandably, asset importance weights reflect the criticality of components within their business environment and their exposure risks at the network layer. This step ensures that enterprises can conduct a refined assessment of defect risks based on their specific circumstances.

[0152] Specifically, the system considers the application environment (e.g., production, testing), business importance (e.g., core transaction system, internal tools), and network exposure (e.g., public network, intranet) of a component, and calculates the asset importance weight using predefined scoring rules. For example, a core transaction system component that directly faces the public network has a much higher weight than an auxiliary tool component that is only used in the internal environment.

[0153] Through the above operations, the introduction of asset importance weights ensures that enterprises can focus on monitoring and protecting those components that are critical to business operations, effectively improving the efficiency of security resource allocation.

[0154] Step S2038: Determine the risk threat weight of the component based on the application frequency and impact scope of the use case.

[0155] Understandably, by analyzing the frequency and scope of use cases, the system can assess the potential threat level of defects and assign risk threat weights to components.

[0156] Specifically, the system examines the activity logs of each exploit case, including the number of downloads, integration within the attack framework, and whether any actual attack events have been observed. Based on this information, the system generates a risk threat weight for each vulnerability to reflect its actual threat level.

[0157] By implementing the above steps, setting risk threat weights helps companies distinguish between risks that are merely theoretical and threats that have already been translated into actual actions, enabling security teams to respond to high-risk vulnerabilities more quickly.

[0158] Step S2039: Calculate the basic risk score using CVSS based on the matching results. Correct the basic risk score based on availability weight, asset importance weight, and risk threat weight to obtain the defect risk score.

[0159] Understandably, by combining CVSS scores with custom weighting factors (availability weight, asset importance weight, and risk threat weight), the system can generate defect risk scores that are closer to the actual situation of enterprises.

[0160] Specifically, the system calculates a basic risk score based on the CVSS scoring model, and then multiplies it by various custom weighting factors to obtain the final vulnerability risk score. For example, if Spring Security's basic CVSS score is 8.0, its availability weight is 3.0, its asset importance weight is 4.0, and its risk threat weight is 3.5, then the final vulnerability risk score = 8.0. 3.0 4.0 3.5 = 336.0.

[0161] Through the above operations, by modifying the CVSS base score with custom weighting factors, the defect risk score generated by the system more accurately reflects the actual risk level of components in the enterprise environment, helping enterprises to more accurately determine the priority of risks and optimize the direction of resource investment.

[0162] Through the above embodiments, the system can comprehensively consider the exploitation risk of components, business importance, and actual threat situation to generate customized defect risk scores, making alerts and remediation decisions more accurate and avoiding the waste of resources in low-risk areas. Furthermore, this mechanism, by combining external threat intelligence with internal asset status, realizes an intelligent process from defect discovery to response, significantly improving the enterprise's response speed and capability to supply chain security risks. Ultimately, the system can help enterprises better protect critical assets, prevent supply chain attacks, and provide strong security support for enterprise digital transformation.

[0163] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0164] This application also provides a defect early warning device based on multi-source data. It should be noted that this defect early warning device based on multi-source data can be used to execute the defect early warning method based on multi-source data provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0165] The following describes the defect early warning device based on multi-source data provided in the embodiments of this application.

[0166] Figure 3 This is a structural block diagram of a defect early warning device based on multi-source data according to an embodiment of this application. Figure 3 As shown, the device includes:

[0167] The acquisition unit 10 is used to capture defect information from multiple data sources, and perform feature encoding and feature fusion on the multiple defect information to obtain multiple first defect data. The data sources include suppliers, security vendors, open source communities and source code libraries.

[0168] The first processing unit 10 is used to traverse the software assets inside the system, generate a first asset graph based on the software assets, construct an abstract syntax tree based on the first asset graph, add the abstract syntax tree to the first asset graph to obtain a second asset graph. The abstract syntax tree is used to represent the dependency relationship between components in the software assets.

[0169] The second processing unit 30 is used to match each first defect data and software asset based on the second asset map, and calculate the defect risk score of each component based on the matching results.

[0170] The generation unit 40 is used to generate alarm information based on the defect risk score and send it to the target terminal.

[0171] In this embodiment, the acquisition unit captures defect information from multiple data sources and performs feature encoding and feature fusion on the multiple defect information to obtain multiple first defect data. The data sources include suppliers, security vendors, open source communities, and source code repositories. The first processing unit traverses the software assets within the system, generates a first asset graph based on the software assets, constructs an abstract syntax tree based on the first asset graph, and adds the abstract syntax tree to the first asset graph to obtain a second asset graph. The abstract syntax tree is used to represent the dependencies between components in the software assets. The second processing unit matches each first defect data with the software assets based on the second asset graph, and calculates the defect risk score of each component based on the matching results. The generation unit generates alarm information based on the defect risk score and sends it to the target terminal. This application introduces multi-source data fusion, especially the collection of original project data from open source communities. Compared with the traditional single official defect library, it shortens the blind spot and time window for defect discovery and improves the real-time performance of defect perception, thus solving the problem that the existing technology uses a single official defect library with single data elements and delayed updates, resulting in insufficient defect detection coverage when applied to emerging open source ecosystems.

[0172] In order to obtain the aforementioned defect information from multiple data sources, in one optional implementation, the obtaining unit includes:

[0173] The first acquisition module is used to acquire the historical update frequency of each data source and determine the corresponding update frequency score based on each historical update frequency.

[0174] The second acquisition module is used to acquire API restriction data from each data source and determine the corresponding response efficiency score based on the API restriction data.

[0175] The third acquisition module is used to acquire the reliability score and contribution score corresponding to each data source;

[0176] The first determining module is used to determine the corresponding information acquisition frequency based on each update frequency score, response efficiency score, reliability score and contribution score. The information acquisition frequency is directly proportional to the update frequency score and the contribution score, and inversely proportional to the response efficiency score and the reliability score.

[0177] The fourth acquisition module is used to retrieve defect information from the corresponding data source based on the acquisition frequency of each piece of information.

[0178] In order to convert defect information into standardized data, in one optional implementation, the acquisition unit includes:

[0179] The first processing module is used to extract entities from the defect information using natural language processing to obtain the second defect data. The entities include the defect-affected components, version number, defect code, and repair submission number.

[0180] The second determination module is used to traverse the second defect data and determine whether there are duplicate entities between pairs of the second defect data.

[0181] The second processing module is used to associate the second defect data to obtain the third defect data when there are duplicate entities between any two second defect data.

[0182] The third processing module is used to encode each second defect data and the third defect data to obtain the corresponding first defect data.

[0183] In order to compile the above abstract syntax tree, in one optional implementation, the first processing unit includes:

[0184] The fourth module is used to traverse all component information of each software project or application in the system to obtain software assets. The component information includes component name, version number and hash value.

[0185] The third determination module is used to determine the application to which the component information belongs, obtain multiple target applications, determine the server to which each target application belongs, and obtain the target server.

[0186] The first building module is used to construct the associated links in the form of component-target application-server, and associate each associated link with software assets to obtain an asset map;

[0187] The second building module is used to perform a full scan of the source code of each component in the asset graph using fine-grained dependency analysis technology, and to build the dependency relationships between the source code in units of functions based on the scan results, thereby obtaining an abstract syntax tree.

[0188] To automate the inspection of system software assets, in one optional implementation, the second processing unit includes:

[0189] The fifth acquisition module is used to extract the code path and function of each code segment from the first defect data, obtain multiple code identifiers, and extract the code pattern of each code segment from the first defect data.

[0190] The fourth determination module is used to traverse each first node in the abstract syntax tree based on the code identifier, and determine the component corresponding to the first node as the first target component when a match is found.

[0191] The fifth determination module is used to determine the first node as the second node when no match is found.

[0192] The sixth determination module is used to retrieve the second node based on the code pattern using a search algorithm. If the retrieval is successful, the component corresponding to the second node is determined as the second target component. The search algorithm includes depth-first search and breadth-first search.

[0193] The first generation module is used to generate matching results based on the first target component and the second target component.

[0194] To further improve the accuracy of the matching results, in one optional implementation, the first generation module includes:

[0195] The first determining submodule is used to determine whether the first version number of the first target component is within the range of affected version numbers of the corresponding code segment. If the first version number is not within the range of affected version numbers, the first target component is deleted.

[0196] The second determining submodule is used to determine whether the second version number of the second target component is within the range of affected version numbers of the corresponding code segment. If the second version number is not within the range of affected version numbers, the second target component is deleted.

[0197] The third determination submodule is used to determine the function call chains of the first target component and the second target component in the abstract syntax tree, and to determine the component covered by the function call chain as the third target component;

[0198] The first processing submodule is used to perform contextual analysis on the first target component, the second target component, and the third target component based on the asset map, so as to determine the availability score corresponding to the first target component, the second target component, and the third target component;

[0199] The second processing submodule is used to delete the corresponding first target component, second target component and third target component when the availability score is less than the first threshold, so as to obtain the matching result.

[0200] In order to calculate the aforementioned defect risk score, in one optional implementation, the second processing unit further includes:

[0201] The seventh module is used to determine the availability weight of a component based on the use cases corresponding to the components that have been matched in each data source.

[0202] The eighth determination module is used to determine the asset importance weight of the components based on the application environment, application services, and network exposure surface of the matched components.

[0203] The ninth module is used to determine the risk threat weight of components based on the application frequency and impact scope of the use cases;

[0204] The calculation module is used to calculate the basic risk score based on the matching results using CVSS, and then correct the basic risk score based on availability weight, asset importance weight, and risk threat weight to obtain the defect risk score.

[0205] The aforementioned defect early warning device based on multi-source data includes a processor and a memory. The acquisition unit, first processing unit, second processing unit, and generation unit are all stored as program units in the memory, and the processor executes the program units stored in the memory to achieve the corresponding functions. All of the above modules are located in the same processor; alternatively, the modules may be located in different processors in any combination.

[0206] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can improve the accuracy of software defect screening.

[0207] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0208] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the aforementioned defect early warning method based on multi-source data.

[0209] Specifically, defect early warning methods based on multi-source data include:

[0210] Step S201: Extract defect information from multiple data sources, and perform feature encoding and feature fusion on the multiple defect information to obtain multiple first defect data. The data sources include suppliers, security vendors, open source communities and source code repositories.

[0211] Step S202: Traverse the software assets within the system, generate a first asset graph based on the software assets, construct an abstract syntax tree based on the first asset graph, add the abstract syntax tree to the first asset graph to obtain a second asset graph. The abstract syntax tree is used to represent the dependency relationships between components in the software assets.

[0212] Step S203: Match each first defect data and software asset based on the second asset map, and calculate the defect risk score of each component based on the matching results.

[0213] Step S204: Generate alarm information based on the defect risk score and send it to the target terminal.

[0214] This invention provides a processor for running a program, wherein the program executes the aforementioned defect warning method based on multi-source data.

[0215] Specifically, defect early warning methods based on multi-source data include:

[0216] Step S201: Extract defect information from multiple data sources, and perform feature encoding and feature fusion on the multiple defect information to obtain multiple first defect data. The data sources include suppliers, security vendors, open source communities and source code repositories.

[0217] Step S202: Traverse the software assets within the system, generate a first asset graph based on the software assets, construct an abstract syntax tree based on the first asset graph, add the abstract syntax tree to the first asset graph to obtain a second asset graph. The abstract syntax tree is used to represent the dependency relationships between components in the software assets.

[0218] Step S203: Match each first defect data and software asset based on the second asset map, and calculate the defect risk score of each component based on the matching results.

[0219] Step S204: Generate alarm information based on the defect risk score and send it to the target terminal.

[0220] This invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs at least the following steps:

[0221] Step S201: Extract defect information from multiple data sources, and perform feature encoding and feature fusion on the multiple defect information to obtain multiple first defect data. The data sources include suppliers, security vendors, open source communities and source code repositories.

[0222] Step S202: Traverse the software assets within the system, generate a first asset graph based on the software assets, construct an abstract syntax tree based on the first asset graph, add the abstract syntax tree to the first asset graph to obtain a second asset graph. The abstract syntax tree is used to represent the dependency relationships between components in the software assets.

[0223] Step S203: Match each first defect data and software asset based on the second asset map, and calculate the defect risk score of each component based on the matching results.

[0224] Step S204: Generate alarm information based on the defect risk score and send it to the target terminal.

[0225] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having at least the following method steps:

[0226] Step S201: Extract defect information from multiple data sources, and perform feature encoding and feature fusion on the multiple defect information to obtain multiple first defect data. The data sources include suppliers, security vendors, open source communities and source code repositories.

[0227] Step S202: Traverse the software assets within the system, generate a first asset graph based on the software assets, construct an abstract syntax tree based on the first asset graph, add the abstract syntax tree to the first asset graph to obtain a second asset graph. The abstract syntax tree is used to represent the dependency relationships between components in the software assets.

[0228] Step S203: Match each first defect data and software asset based on the second asset map, and calculate the defect risk score of each component based on the matching results.

[0229] Step S204: Generate alarm information based on the defect risk score and send it to the target terminal.

[0230] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0231] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0232] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0233] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0234] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0235] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0236] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0237] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0238] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0239] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0240] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0241] 1) The defect early warning method based on multi-source data in this application firstly extracts defect information from multiple data sources, and performs feature encoding and feature fusion on these multiple defect information to obtain multiple first defect data. The data sources include suppliers, security vendors, open source communities, and source code repositories. Then, it traverses the software assets within the system, generates a first asset graph based on the software assets, and constructs an abstract syntax tree based on the first asset graph. The abstract syntax tree is added to the first asset graph to obtain a second asset graph. The abstract syntax tree is used to represent the dependencies between components in the software assets. Afterward, it matches each first defect data with the software assets based on the second asset graph, and calculates the defect risk score of each component based on the matching results. Finally, it generates alarm information based on the defect risk score and sends it to the target terminal. This application introduces multi-source data fusion, especially collecting original project data from open source communities. Compared with the traditional single official defect library, this shortens the blind spot and time window for defect discovery and improves the real-time performance of defect perception, thus solving the problem that the existing technology uses a single official defect library with limited data elements and delayed updates, resulting in insufficient defect detection coverage when applied to emerging open source ecosystems.

[0242] 2) The defect early warning device based on multi-source data in this application involves an acquisition unit that captures defect information from multiple data sources and performs feature encoding and feature fusion on the multiple defect information to obtain multiple first defect data. The data sources include suppliers, security vendors, open-source communities, and source code repositories. A first processing unit traverses the software assets within the system, generates a first asset map based on the software assets, constructs an abstract syntax tree based on the first asset map, and adds the abstract syntax tree to the first asset map to obtain a second asset map. The abstract syntax tree is used to represent the dependencies between components in the software assets. A second processing unit matches each first defect data with the software assets based on the second asset map, and calculates the defect risk score of each component based on the matching results. A generation unit generates alarm information based on the defect risk score and sends it to the target terminal. This application introduces multi-source data fusion, especially the collection of original project data from open-source communities. Compared with the traditional single official defect library, this shortens the blind spot and time window for defect discovery and improves the real-time performance of defect perception, thus solving the problem of insufficient defect detection coverage when applied to emerging open-source ecosystems due to the single official defect library's limited data elements and delayed updates in existing technologies.

[0243] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A defect early warning method based on multi-source data, characterized in that, include: Defect information is captured from multiple data sources, and feature encoding and feature fusion are performed on the multiple defect information to obtain multiple first defect data. The data sources include suppliers, security vendors, open source communities and source code libraries. The system traverses the software assets within the system, generates a first asset graph based on the software assets, constructs an abstract syntax tree based on the first asset graph, and adds the abstract syntax tree to the first asset graph to obtain a second asset graph. The abstract syntax tree is used to represent the dependency relationships between components in the software assets. Based on the second asset map, each of the first defect data and the software assets are matched, and the defect risk score of each component is calculated according to the matching results. An alarm message is generated based on the defect risk score and sent to the target terminal.

2. The method according to claim 1, characterized in that, Defect information is retrieved from multiple data sources, including: Obtain the historical update frequency of each of the data sources, and determine the corresponding update frequency score based on each of the historical update frequencies; Obtain API restriction data for each of the aforementioned data sources, and determine the corresponding response efficiency score based on the API restriction data for each of the aforementioned data sources; Obtain the reliability score and contribution score corresponding to each of the aforementioned data sources; The corresponding information acquisition frequency is determined based on the update frequency score, the response efficiency score, the reliability score, and the contribution score, wherein the information acquisition frequency is directly proportional to the update frequency score and the contribution score, and the information acquisition frequency is inversely proportional to the response efficiency score and the reliability score. Defect information is retrieved from the corresponding data source based on the frequency of each information acquisition.

3. The method according to claim 2, characterized in that, Multiple defect information items are feature-encoded and feature-fused to obtain the first defect data, including: Natural language processing is used to extract entities from defect information to obtain second defect data. The entities include defect-affected components, version number, defect code, and repair submission number. Traverse the second defect data to determine whether there are duplicate entities between any pairs of the second defect data; If there are duplicate entities between any two of the second defect data, the second defect data are associated to obtain the third defect data; Each of the second defect data and the third defect data is encoded to obtain the corresponding first defect data.

4. The method according to claim 1, characterized in that, Traverse the software assets within the system, generate an asset graph based on the software assets, and construct an abstract syntax tree based on the asset graph, including: The software assets are obtained by traversing all component information of each software project or application in the system. The component information includes the component name, version number, and hash value. Determine the application to which the component information belongs, thereby obtaining multiple target applications; determine the server to which each target application belongs, thereby obtaining the target server. The associated links are constructed in the form of component-target application-server, and each of the associated links is associated with the software assets to obtain the asset map; Fine-grained dependency analysis technology is used to perform a full scan of the source code of each component in the asset graph, and the dependency relationships between the source code are constructed on a function-by-function basis based on the scan results to obtain the abstract syntax tree.

5. The method according to claim 1, characterized in that, Matching each of the first defect data and the software assets based on the second asset map includes: Extract the code path and function of each code segment from the first defect data to obtain multiple code identifiers, and extract the code pattern of each code segment from the first defect data; Based on the code identifier, each first node in the abstract syntax tree is traversed, and if a match is found, the component corresponding to the first node is determined as the first target component. If no match is found, the first node is identified as the second node; Based on the code pattern, a search algorithm is used to retrieve the second node. If the retrieval is successful, the component corresponding to the second node is determined as the second target component. The search algorithm includes depth-first search and breadth-first search. The matching result is generated based on the first target component and the second target component.

6. The method according to claim 5, characterized in that, Generating the matching result based on the first target component and the second target component includes: Determine whether the first version number of the first target component is within the range of affected version numbers corresponding to the code segment. If the first version number is not within the range of affected version numbers, delete the first target component. Determine whether the second version number of the second target component is within the range of the affected version numbers corresponding to the code segment. If the second version number is not within the range of the affected version numbers, delete the second target component. The function call chains of the first target component and the second target component in the abstract syntax tree are determined respectively, and the components covered by the function call chains are determined as the third target component; Contextual analysis is performed on the first target component, the second target component, and the third target component based on the asset map to determine the availability scores corresponding to the first target component, the second target component, and the third target component; If the availability score is less than a first threshold, the corresponding first target component, second target component, and third target component are deleted to obtain the matching result.

7. The method according to claim 1, characterized in that, Based on the matching results, a defect risk score is calculated for each component, including: Based on the use cases corresponding to the components that have been matched in each of the aforementioned data sources, determine the availability weight of the components; The asset importance weight of the components is determined based on their application environment, application services, and network exposure surface. The risk threat weight of the component is determined based on the application frequency and impact scope of the aforementioned use cases; Based on the matching results, a basic risk score is calculated using CVSS. The basic risk score is then corrected based on the availability weight, the asset importance weight, and the risk threat weight to obtain the defect risk score.

8. A defect early warning device based on multi-source data, characterized in that, The device includes: The acquisition unit is used to capture defect information from multiple data sources, and perform feature encoding and feature fusion on the multiple defect information to obtain multiple first defect data. The data sources include suppliers, security vendors, open source communities and source code libraries. The first processing unit is used to traverse the software assets within the system, generate a first asset graph based on the software assets, construct an abstract syntax tree based on the first asset graph, add the abstract syntax tree to the first asset graph to obtain a second asset graph, wherein the abstract syntax tree is used to represent the dependency relationships between components in the software assets. The second processing unit is used to match each of the first defect data and the software assets based on the second asset map, and to calculate the defect risk score of each component based on the matching results. The generation unit is used to generate alarm information based on the defect risk score and send it to the target terminal.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising methods for performing any one of claims 1 to 7.

Images