A component matching method, device, computer equipment and storage medium

Abstract

The present disclosure provides a component matching method, apparatus, computer equipment and storage medium, wherein the method comprises: extracting original code features of each function and class in the component to be identified based on code information of the component to be identified obtained from a code platform; constructing a directed graph network based on the original code features using each function and class as a network node; inputting the directed graph network into a pre-trained first neural network to obtain an embedded feature vector of each network node and a relationship feature vector between each network node and other network nodes; using a pre-trained multi-layer autoencoder to perform feature encoding on the embedded feature vector and the relationship feature vector of each network node in turn to obtain a target feature vector of each network node; and determining a matching component that matches the component to be identified from each identified component in a pre-built component library based on the target feature vector of each network node of the component to be identified.

Description

Technical Field

[0001] The present disclosure relates to the field of software security technology, and in particular to a component matching method, apparatus, computer equipment and storage medium. Background Art

[0002] With the continuous development of information technology, various code platforms provide more and more open source components, which have gradually become an important part of software development systems. Using open source components for software development can effectively shorten the development cycle and reduce development costs. However, since the code of open source components is public, there is a risk of malicious modification and attack, which leads to potential security risks in software developed using open source components.

[0003] In order to avoid the impact of maliciously modified open source components on the development software as much as possible, the open source components can be identified, matched and security tested in advance. However, the conventional component identification and matching method has the problem of inaccurate matching results, which in turn reduces the accuracy of the security test results. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a component matching method, apparatus, computer device and storage medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a component matching method, including:

[0006] Extracting original code features of each function and class in the component to be identified based on the code information of the component to be identified obtained from the code platform; the original code features include code attribute features, call relationship features and dependency relationship features;

[0007] Using each function and class as a network node, constructing a directed graph network according to the original code features;

[0008] Inputting the directed graph network into a pre-trained first neural network to obtain an embedded feature vector of each of the network nodes and a relationship feature vector between each of the network nodes and other network nodes;

[0009] Using a pre-trained multi-layer autoencoder, feature encoding is performed on the embedded feature vector and the relationship feature vector of each network node in turn to obtain a target feature vector of each network node;

[0010] According to the target feature vector of each of the network nodes of the component to be identified, a matching component matching the component to be identified is determined from each identified component in a pre-built component library.

[0011] In a possible implementation, obtaining code information of a component to be identified from a code platform includes:

[0012] Acquire, from the code platform, code information of the component to be identified and feature information of the component to be identified in multiple feature dimensions according to the latest acquisition frequency;

[0013] According to the latest weight coefficient corresponding to each of the feature dimensions, weighted processing is performed on the feature information under the multiple feature dimensions to obtain the first platform activity corresponding to the component to be identified;

[0014] According to the activity of the first platform, the latest acquisition frequency is adjusted to obtain an adjusted acquisition frequency; the adjusted acquisition frequency is used for obtaining the component to be identified from the code platform next time.

[0015] In a possible implementation manner, after determining the activity of the first platform, the method further includes:

[0016] Determine the activity prediction loss according to the first platform activity corresponding to the component to be identified, the historical platform activity of each historical component collected before the component to be identified, and the second platform activity of each historical component under the latest weight coefficient;

[0017] Using the activity prediction loss, each of the latest weight coefficients is updated to obtain an updated weight coefficient; the updated weight coefficient is used to determine the first platform activity of the component to be identified that is collected next time.

[0018] In a possible implementation, the extracting original code features of each function and class in the component to be identified based on the code information of the component to be identified obtained from the code platform includes:

[0019] Performing data cleaning and standardization processing on the code information of the component to be identified to obtain standard code information;

[0020] The original code features of each function and class are extracted from the standard code information using a code analysis tool.

[0021] In a possible implementation, the method of using each function and class as a network node and constructing a directed graph network according to the original code features includes:

[0022] Determine the directed edges between the network nodes according to the call relationship features in the original code features;

[0023] Determining an edge weight of the directed edge according to a function call frequency, a call path depth, and a call parameter complexity between two network nodes corresponding to the directed edge;

[0024] Determining a characteristic attribute matrix of each network node according to the original code characteristics of each network node;

[0025] The directed graph network is constructed according to the directed edges between the network nodes, the edge weights of the directed edges, and the characteristic attribute matrix of each network node.

[0026] In a possible implementation, the first neural network includes multiple network layers;

[0027] The step of inputting the directed graph network into a pre-trained first neural network to obtain an embedded feature vector of each of the network nodes and a relationship feature vector between each of the network nodes and other network nodes includes:

[0028] For any of the network layers, obtain the first feature vector of each network node output by the previous network layer and the node relationship feature vector of each of the network nodes; wherein, when the network layer is the first layer, the first feature vector of each network node output by the previous network layer is the feature attribute matrix of each of the network nodes, and the node relationship feature vector of each of the network nodes is the edge weight of the directed edge of each of the network nodes;

[0029] Determining a second feature vector of the network node according to first feature vectors of each neighboring node of the network node and a node relationship feature vector of the network node;

[0030] Determine, according to the weight matrix of each attention head in the network layer, the second eigenvector of the network node, and the second eigenvector of each neighboring node of the network node, the aggregation weight of the network node and each neighboring node corresponding to each attention head;

[0031] Determine a node relationship feature vector output by the network node at the network layer according to the aggregation weights corresponding to the network node and each adjacent node at each attention head;

[0032] According to the aggregation weights corresponding to the network node and each adjacent node in each of the attention heads, the second feature vectors of each adjacent node of the network node are aggregated to obtain the first feature vector output by the network node in the network layer; wherein the first feature vector of each network node output by the last network layer is used as the embedded feature vector of each network node; and the node relationship feature vector of each network node output by the last network layer is used as the relationship feature vector of each network node.

[0033] In a possible implementation manner, determining the characteristic attribute matrix of each network node according to the original code feature of each network node includes:

[0034] Determine the correlation between every two of the network nodes according to the original code features of each of the network nodes;

[0035] Using a feature selection matrix composed of correlations between every two of the network nodes, the original code features of each of the network nodes are screened to obtain a screened code feature of each of the network nodes;

[0036] According to the filtered code features of each of the network nodes, a feature attribute matrix of the network nodes is determined.

[0037] In a possible implementation, a pre-trained multi-layer autoencoder is used to perform feature encoding on the embedded feature vector and the relationship feature vector of each network node in turn to obtain a target feature vector of each network node, including:

[0038] Inputting the embedded feature vector and the relationship feature vector of each of the network nodes into the first layer of the autoencoder in the multi-layer autoencoder to obtain the intermediate feature vector and the intermediate relationship feature vector of each of the network nodes;

[0039] Inputting the intermediate feature vector and the intermediate relationship feature vector of each of the network nodes into the next layer of autoencoder to obtain a new intermediate feature vector and a new intermediate relationship feature vector of each of the network nodes;

[0040] Return to the step of inputting the intermediate feature vector and the intermediate relationship feature vector of each of the network nodes to the next layer of autoencoder until the target feature vector of each of the network nodes output by the last layer of autoencoder is obtained.

[0041] In a possible implementation, determining, according to the target feature vector of each of the network nodes of the component to be identified, from each identified component in a pre-built component library, a matching component that matches the component to be identified, includes:

[0042] Determine a first function list corresponding to the component to be identified according to each function and class of the component to be identified, and determine a second function list corresponding to each identified component according to each function and class in each identified component;

[0043] Determining, according to the first function list and each of the second function lists, a component similarity between each of the identified components and the component to be identified;

[0044] According to the component similarity, selecting candidate components similar to the component to be identified from the identified components;

[0045] The matching component is determined according to the target feature vector of each of the network nodes in the component to be identified and the target feature vector of each of the network nodes in each of the candidate components.

[0046] In a possible implementation manner, determining the matching component according to the target feature vector of each of the network nodes in the component to be identified and the target feature vector of each of the network nodes in each of the candidate components includes:

[0047] Determine a first similarity between the component to be identified and each of the candidate components according to the original code feature of each of the network nodes in the component to be identified and the original code feature of each of the network nodes in each of the candidate components;

[0048] Determine a second similarity between the component to be identified and each of the candidate components according to the target feature vector of each of the network nodes in the component to be identified, the target feature vector of each of the network nodes in each of the candidate components, and the edge weight of the directed edge between every two network nodes;

[0049] Determining a third similarity between the component to be identified and each of the candidate components according to the identification time of the component to be identified and the identification time of each of the candidate components;

[0050] According to the first similarity, the second similarity and the third similarity, a target similarity between the component to be identified and each of the candidate components is determined, and the matching component is determined according to the target similarity.

[0051] In a possible implementation, the method further includes:

[0052] In the case where there is no matching component, the target feature vector and the original code feature of each network node of the component to be identified are added to the component library.

[0053] In a possible implementation, after determining a matching component that matches the component to be identified, the method further includes:

[0054] The matching components in the component library are updated using the target feature vector and the original code feature of each network node of the component to be identified.

[0055] In a possible implementation manner, the first neural network is trained according to the following steps:

[0056] Obtaining a sample directed graph network corresponding to the first sample component; the sample directed graph network includes a plurality of sample network nodes;

[0057] Inputting the sample directed graph network into a first neural network to be trained, obtaining a predicted second feature vector and a predicted node relationship feature vector of each sample network node output by a first network layer of the first neural network to be trained, and a predicted embedding feature vector and a predicted relationship feature vector of each sample network node output by a last network layer of the first neural network to be trained;

[0058] Determining a first prediction loss of the first neural network to be trained according to the predicted second feature vector, the predicted node relationship feature vector, the predicted embedding feature vector and the predicted relationship feature vector;

[0059] The first prediction loss is used to iteratively train the first neural network to obtain a trained first neural network.

[0060] In a possible implementation manner, the multi-layer autoencoder is trained according to the following steps:

[0061] Obtaining a sample embedding feature vector and a sample relationship feature vector of each sample network node in the sample directed graph network corresponding to the second sample component;

[0062] Input the sample embedding feature vector and the sample relationship feature vector of each sample network node into the multi-layer autoencoder to be trained, and obtain the predicted target feature vector and the predicted sample relationship feature vector of each network node output by the last layer of autoencoder;

[0063] Using the decoder in the multi-layer autoencoder to be trained, the predicted target feature vector and the predicted sample relationship feature vector of each network node are decoded to obtain a decoded feature vector and a decoded relationship feature vector;

[0064] Determining a second prediction loss of the multi-layer autoencoder to be trained according to the decoded feature vector, the predicted target feature vector, the predicted sample relationship feature vector, and the decoded relationship feature vector;

[0065] The multi-layer autoencoder to be trained is iteratively trained using the second prediction loss to obtain a trained multi-layer autoencoder.

[0066] In a second aspect, an embodiment of the present disclosure further provides a component matching device, including:

[0067] The first extraction module is used to extract the original code features of each function and class in the component to be identified based on the code information of the component to be identified obtained from the code platform; the original code features include code attribute features, call relationship features and dependency relationship features;

[0068] A construction module, used to construct a directed graph network according to the original code features by using each function and class as a network node;

[0069] A second extraction module is used to input the directed graph network into a pre-trained first neural network to obtain an embedded feature vector of each of the network nodes and a relationship feature vector between each of the network nodes and other network nodes;

[0070] An encoding module, used to use a pre-trained multi-layer autoencoder to perform feature encoding on the embedded feature vector and the relationship feature vector of each network node in turn to obtain a target feature vector of each network node;

[0071] The determination module is used to determine a matching component that matches the component to be identified from each identified component in a pre-built component library according to a target feature vector of each network node of the component to be identified.

[0072] In a third aspect, an optional implementation of the present disclosure further provides a computer device, a processor, and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are performed.

[0073] In a fourth aspect, an optional implementation of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are executed.

[0074] For a description of the effects of the above-mentioned component matching device, computer equipment, and computer-readable storage medium, please refer to the description of the above-mentioned component matching method, which will not be repeated here.

[0075] The component matching method, device, computer equipment and storage medium provided by the embodiments of the present disclosure input the original code features of the component to be identified into the first neural network in the form of a directed graph network, which can fully explore the code structure features, deep function call relationships, dependency relationships and other features of the component to be identified, and obtain embedded feature vectors that can accurately characterize each function and class, as well as relationship feature vectors that characterize the association relationship between functions and classes. Then, the embedded feature vector and the relationship feature vector are input into a multi-layer autoencoder, which can realize multi-level nonlinear dimensionality reduction of the embedded feature vector on the basis of retaining the association relationship between functions and classes, reduce feature redundancy and computational overhead, and obtain a target feature vector that can accurately express functions and classes. Finally, the target feature vector is used to identify and match the component to be identified in the component library, which can improve the accuracy of component identification and matching, and thus improve the accuracy of security detection.

[0076] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0078] Figure 1 A flow chart of a component matching method provided by an embodiment of the present disclosure is shown;

[0079] Figure 2 A schematic diagram showing a specific process of determining a matching component from a component library provided by an embodiment of the present disclosure is shown;

[0080] Figure 3 The specific implementation process of a component matching method provided by an embodiment of the present disclosure is shown;

[0081] Figure 4 A schematic diagram of a component matching device provided by an embodiment of the present disclosure is shown;

[0082] Figure 5 A schematic diagram of the structure of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0083] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0084] In addition, the terms "first", "second", etc. in the description and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0085] The "multiple or several" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0086] Research has found that most of the current open source component identification and matching methods are based on static code features extracted from components by static code analysis tools or single-dimensional feature matching of components. However, for methods based on static code analysis tools, static analysis tools can only extract simple function calls and dependencies in components, but cannot extract complex function call links and dynamic call relationships in components, resulting in poor accuracy in component identification. For methods that match single-dimensional features of components, it is difficult to fully consider the feature relevance of components in multiple aspects (such as function calls, contextual relationships, component updates, and component structures), which affects the accuracy of component identification. Therefore, how to improve the recognition accuracy of open source components has become a technical issue worthy of attention.

[0087] Based on the above research, the present disclosure provides a component matching method, device, computer equipment and storage medium. The original code features of the component to be identified are input into the first neural network in the form of a directed graph network, which can fully explore the code structure features, deep function call relationships, dependency relationships and other features of the component to be identified, and obtain embedded feature vectors that can accurately characterize each function and class, as well as relationship feature vectors that characterize the association relationship between functions and classes. Then, the embedded feature vector and the relationship feature vector are input into a multi-layer autoencoder, which can realize multi-level nonlinear dimensionality reduction of the embedded feature vector on the basis of retaining the association relationship between functions and classes, reduce feature redundancy and computational overhead, and obtain a target feature vector that can accurately express functions and classes. Finally, the target feature vector is used to identify and match the component to be identified in the component library, which can improve the accuracy of component identification and matching, thereby improving the accuracy of security detection.

[0088] The defects existing in the above solutions are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0089] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0090] It should be noted that the specific terms mentioned in the embodiments of the present disclosure include:

[0091] Z-score standardization: also known as standard score standardization or Z-score normalization, is a statistical method that adjusts data according to its mean and standard deviation. This method converts the data into a standard normal distribution with a mean of 0 and a standard deviation of 1;

[0092] Gumbel-Softmax feature selection: is a technique used in machine learning for differentiable sampling from discrete distributions. This method combines the Gumbel distribution and the normalization (Softmax) function to make the sampling process from the discrete distribution differentiable, allowing backpropagation.

[0093] To facilitate understanding of this embodiment, a component matching method disclosed in an embodiment of the present disclosure is first introduced in detail. The executor of the component matching method provided in the embodiment of the present disclosure is generally a terminal device or other processing device with certain computing capabilities, wherein the terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a personal digital assistant (PDA), a handheld device, a computer device, etc.; in some possible implementations, the component matching method may be implemented by a processor calling computer-readable instructions stored in a memory.

[0094] The component matching method provided in the embodiment of the present disclosure is described below by taking a computer device as an example of an execution subject.

[0095] like Figure 1 As shown, it is a flowchart of a component matching method provided by an embodiment of the present disclosure, which may include the following steps:

[0096] S101: extracting original code features of each function and class in the component to be identified based on code information of the component to be identified obtained from the code platform; the original code features include code attribute features, call relationship features and dependency relationship features.

[0097] Here, the code platform can be any open source code platform or code community, and the component to be identified is an open source component collected from the code platform. The code information is component information related to the component to be identified, and may specifically include component code, component documentation, version update log, community discussion data, etc. of the component to be identified. The component code of the component to be identified may include at least one function and / or at least one class defined by the developer.

[0098] The original code features are component features extracted from the code information of the component to be identified, which include but are not limited to the component's code attribute features, call relationship features, and dependency relationship features. Among them, the code attribute features are used to characterize the basic attribute information of the component code, which may specifically include code structure features including component constant data, variable names, function names, function parameters, number of classes, number of methods, total number of lines of code, and other code structure features. The call relationship features are used to characterize features such as simple internal call relationships, call patterns, degree of coupling, and inheritance relationships between functions or classes. The dependency relationship features are used to characterize information such as the Application Programming Interface (API) and external data references that functions and classes need to rely on.

[0099] Among them, the original code features of each function and class in the component to be identified can form an original code feature set X.

[0100] In specific implementation, a multi-threaded acquisition tool can be used to collect code information of each component to be identified from multiple source code platforms in real time. For each component to be identified, a static code analysis tool (such as IDA pro tool) can be used to analyze the code information of the component to be identified to obtain the original code features of the component to be identified.

[0101] It is understandable that for any component to be identified collected from a multi-source code platform, the component matching method provided by the embodiment of the present disclosure can be used to identify and match the component, and then security detection can be performed on the component to be identified based on the matching result.

[0102] In one embodiment, in order to improve data quality, S101 may also be implemented according to the following steps:

[0103] S101 - 1 : Clean and standardize the code information of the component to be identified to obtain standard code information.

[0104] Here, data cleaning can specifically include but is not limited to removing noise information and outliers in code information, unifying data formats, filling missing data, deduplication of data, and data type conversion. Standardization processing is used to process code information into a standardized format. Specifically, the formatting processing method can be a Z-score standardization method to ensure that the correlation between multimodal data is maintained during the data standardization process. Among them, multimodal data refers to data in the code information including component code, component documentation, version update logs, community discussion data, etc.

[0105] Exemplarily, after obtaining the code information of the component to be identified, multimodal data cleaning and standardization processing can be performed. Specifically, the collected multimodal data can be cleaned to remove noise and outliers in the data, unify the processing format and missing data, etc. Then, the Z-score standardization method is used to standardize the cleaned data to obtain standard code information.

[0106] S101-2: Using code analysis tools, extract original code features of each function and class from standard code information.

[0107] Here, the code analysis tool may specifically be a static code analysis tool, such as IDA pro, HOPPER, and the like.

[0108] For example, IDA pro can be used to disassemble the component code in the standard code information, and then analyze and process the disassembly results to obtain the original code features of each function and class in the component to be identified.

[0109] In this way, a multimodal data cleaning and standardization method based on deep fusion is adopted to process multiple data through improved Z-score standardization, improve the correlation and consistency of multimodal features, ensure the quality of data processing, and provide a more reliable data foundation for subsequent feature extraction. It effectively solves the problem of poor data processing effect in the data processing link of traditional technologies due to the complexity of open source component data sources, noise, outliers, and different data formats.

[0110] S102: Using each function and class as a network node, a directed graph network is constructed according to the original code features.

[0111] Here, a directed graph may include multiple network nodes, one network node corresponds to a function or a class of the component to be identified, and the number of network nodes is determined by the total number of functions and classes in the component to be identified. Whether there are directed edges with directions between different network nodes is determined based on the call relationship characteristics between the functions or classes corresponding to the network nodes. For example, when function A calls function B, there is a directed edge between the network node corresponding to function A and the network node corresponding to function B.

[0112] Exemplarily, each function and class in the component to be identified can be used as a network node, and the connection relationship and directed edges between the network nodes can be determined based on the calling relationship characteristics between the functions and classes. The node attribute information of each network node can be determined based on the original code characteristics of each function and class. The attribute information can specifically be the feature attribute matrix described later.

[0113] In one embodiment, the above S102 may be implemented according to the following steps:

[0114] S102-1: Determine directed edges between various network nodes according to call relationship features in the original code features.

[0115] Exemplarily, based on the calling relationship characteristics of each function and class, it can be determined that there are directed edges between network nodes corresponding to functions or classes with calling relationships, the calling relationship or inheritance relationship is used as a directed edge, and the direction of the directed edge is determined based on the calling relationship or inheritance relationship.

[0116] S102-2: Determine the edge weight of the directed edge according to the function call frequency, call path depth, and call parameter complexity between two network nodes corresponding to the directed edge.

[0117] Here, the function call frequency It is used to characterize the calling frequency between functions or classes corresponding to two network nodes with a directed edge. It is used to characterize the call hierarchy between functions or classes corresponding to two network nodes with a directed edge. For example, if function A calls function D through function B and function C, the call path depth between function A and function D is 3. Call parameter complexity It is used to characterize the complexity of the calling parameters between the functions or classes corresponding to two network nodes with a directed edge. Among them, the function calling frequency, calling path depth and calling parameter complexity can be extracted separately from the code information or determined based on the original code features.

[0118] In specific implementation, the edge weight of the directed edge can be determined using the following formula 1:

[0119] ; (Formula 1)

[0120] in, represents the edge weight, α, β, and γ represent coefficients, which can be set adaptively based on experience and the importance of function call frequency, call path depth, and call parameter complexity.

[0121] Exemplarily, for each directed edge, the edge weight of the directed edge can be determined according to the function call frequency, call path depth, and call parameter complexity corresponding to the directed edge, according to the above formula 1.

[0122] S102-3: Determine a characteristic attribute matrix of each network node according to the original code characteristics of each network node.

[0123] In specific implementation, the characteristic attribute matrix of each network node can be constructed according to the original code features of each network node, or the characteristic attribute matrix of each network node can be constructed according to the code attribute features and dependency features in the original code features of each network node.

[0124] S102-4: Construct a directed graph network according to the directed edges between the network nodes, the edge weights of the directed edges, and the characteristic attribute matrix of the network nodes.

[0125] Exemplarily, the connection relationship between each network node can be constructed based on the directed edges between each network node, and the edge weights of the directed edges are used as attributes of the directed edges, and the characteristic attribute matrix of the network nodes is used as the attribute of the network nodes to construct a directed graph network.

[0126] In one embodiment, in order to reduce the computational complexity of the first neural network, the Gumbel-Softmax feature selection method can also be applied to filter out the most important function call graph features from the original code features of the network nodes to construct the feature attribute matrix. Therefore, the above S102-3 can also be implemented according to the following steps:

[0127] S102-3-1: Determine the correlation between every two network nodes based on the original code features of each network node.

[0128] Here, for any two network nodes (network node i and network node j), the correlation between network node i and network node j can be determined according to the original code features corresponding to each network node.

[0129] In specific implementation, the correlation between every two network nodes can be determined using the following formula 2:

[0130] ; (Formula 2)

[0131] in, Represents a network node i and network nodes j The correlation between Represents a network node i The corresponding adaptive coefficient can be adaptively adjusted according to the importance of the data features corresponding to the network nodes; X represents the feature set X composed of the original code features of each network node.

[0132] S102-3-2: Using the feature selection matrix composed of the correlation between every two network nodes, the original code features of each network node are screened to obtain the screened code features of each network node.

[0133] In specific implementation, the correlation between every two network nodes can be used , construct a feature selection matrix S, and then use the feature selection matrix to perform feature selection on the original code features of each network node, so as to select the features that need to be retained, and use the features that need to be retained as the screened code features.

[0134] It is understandable that for a certain network node, its corresponding filtered code features may be all or part of the original code features, or may be empty. In the case where the filtered code features are empty, it can be explained that the network node is an unimportant node and can be removed, and the structure of the directed network graph is updated.

[0135] S102-3-3: Determine a characteristic attribute matrix of each network node based on the filtered code features of each network node.

[0136] In specific implementation, the filtered code features of each network node can be used to construct a feature attribute matrix of each network node.

[0137] In this way, by screening the original code features of the network nodes through the feature selection matrix, the dimension of the model input can be reduced and the key features can be retained, thereby optimizing the computational cost and the robustness of the model.

[0138] S103: Input the directed graph network into a pre-trained first neural network to obtain an embedded feature vector of each network node and a relationship feature vector between each network node and other network nodes.

[0139] Here, the embedded feature vector is used to characterize the node features of the network node. The relationship feature vector is used to characterize the deep call relationship and deep dependency relationship between network nodes. Based on the relationship feature vector, the complex function call links and dynamic call relationships in the component can be reflected. Compared with the call relationship features and dependency features extracted by static code analysis tools, the call relationship and dependency relationship represented by the relationship feature vector will be more complex, more accurate, and more comprehensive.

[0140] The first neural network may be a Multi-Label Adaptive Graph Convolutional Network (ML-AGCN for short).

[0141] For example, a directed graph network can be input into a pre-trained ML-AGCN, and deep function calls and deep dependency features can be extracted by ML-AGCN under a multi-layer adaptive propagation mechanism. A weighted adaptive update method between network nodes is used to achieve cross-layer feature enhancement, so as to deeply explore the complex structure of dependency links between components and obtain the embedded feature vector and relationship feature vector of each network node. During the ML-AGCN processing, by dynamically updating the node and edge weights in the graph structure, a higher-precision feature capture capability and a comprehensive representation of dependencies at different levels can be achieved.

[0142] S104: Using a pre-trained multi-layer autoencoder, feature encoding is performed on the embedded feature vector and the relationship feature vector of each network node in turn to obtain a target feature vector of each network node.

[0143] Here, Multilayer Stacked Autoencoder (MRASE) is a deep learning model that achieves unsupervised learning of data by stacking multiple autoencoder layers.

[0144] In specific implementation, the embedded feature vectors and relationship feature vectors of each network node output by ML-AGCN can be input into a pre-trained multi-layer autoencoder in the form of vector matrices. Each vector matrix is ​​sequentially passed through each autoencoder layer of the multi-layer autoencoder for feature extraction, screening and dimensionality reduction to obtain the target feature vector of each network node output by the last autoencoder layer. In the multi-layer autoencoder, the output of the previous autoencoder layer is used as the input of the next autoencoder layer, and the data is gradually compressed and reconstructed in this way. Each layer of the autoencoder will learn the data representation of the previous layer, thereby gradually extracting more abstract features.

[0145] In this way, the multi-layer autoencoder extracts and screens the key features of the network nodes in a multi-level stacking manner with relationship preservation, ensuring that the features in the low-dimensional space have the highest information density and relationship integrity, significantly improving the accuracy and computing performance of feature extraction, and providing efficient and accurate feature input for component identification. Each layer of the encoder can retain the relationship information between data samples through the relationship feature vector while extracting low-dimensional features, thereby enhancing the semantic consistency and expression ability of the reduced dimensionality features.

[0146] S105: According to the target feature vector of each network node of the component to be identified, a matching component matching the component to be identified is determined from each identified component in the pre-built component library.

[0147] Here, the component library is a pre-built component feature library, which includes the target feature vectors of each identified component that has been identified in history. The matching component is a component that is similar to the component to be identified among the identified components. The target feature vector corresponding to each identified component in the component library can be dynamically and incrementally updated according to the identification results of the identified components at different times, so that the feature library has real-time response capabilities and ensures the sensitivity and adaptability of the component library to newly identified components to be identified and data changes.

[0148] Specifically, the above S101~S104 can be used to identify each identified component from the multi-source platform, and the target feature vector and code information of the identified component can be used to adaptively construct and dynamically update the component library. For a certain identified component, if the component is identified in different times, the feature data of the component in the component library can be adaptively updated according to the target feature vector of the component determined most recently, the feature change rate and the importance of the new data. After the component library is constructed, for any identified component to be identified, the identified component in the component library can be used to match the component to be identified, and a matching component that matches the component to be identified can be obtained.

[0149] Exemplarily, after obtaining the target feature vector of the component to be identified, the target feature vector and the code information of the component to be identified, as well as the target feature vectors and code information of each identified component already in the component library, can be used to calculate the similarity between the component to be identified and each identified component, and based on the similarity, a target component with the highest similarity to the component to be identified is screened out from the identified components, and the target component is used as the matching component.

[0150] Optionally, if the similarity between the target component and the component to be identified is not less than a preset similarity, the target component is determined to be a matching component that matches the component to be identified. If the similarity between the target component and the component to be identified is less than a preset similarity, it is determined that the matching component that matches the component to be identified does not exist in the component library.

[0151] Furthermore, after determining that there is a matching component, the target feature vector and original code feature of each network node of the component to be identified can be used to update the matching component in the component library. Exemplarily, the feature change rate of each network node can be determined based on the target feature vector and original code feature of each network node in the component to be identified, and the target feature vector and original code feature of each network node in the matching component; based on the feature change rate of each network node and the importance of the target feature vector corresponding to each network node in the component to be identified, the feature vector of the matching component in the component library is updated, thereby achieving real-time update of the component library.

[0152] In another embodiment, when there is no matching component, the component to be identified can be treated as a newly identified component, and the target feature vector and original code feature of each network node of the component to be identified can be added to the component library, thereby enriching the number of components and the number of features in the component library.

[0153] In addition, after determining the matching component of the component to be identified, the component to be identified can be security checked based on the source of the matching component, the code vulnerability library corresponding to the matching component, and other information to determine whether the component to be identified has security risks.

[0154] In the disclosed embodiment, the original code features of the component to be identified are input into the first neural network in the form of a directed graph network, which can fully explore the code structure features, deep function call relationships, dependency relationships and other features of the component to be identified, and obtain embedded feature vectors that can accurately characterize each function and class, as well as relationship feature vectors that characterize the association relationship between functions and classes. The embedded feature vector and the relationship feature vector are then input into a multi-layer autoencoder, which can achieve multi-level nonlinear dimensionality reduction of the embedded feature vector on the basis of retaining the association relationship between functions and classes, reduce feature redundancy and computational overhead, and obtain a target feature vector that can accurately express functions and classes. Finally, the target feature vector is used to identify and match the component to be identified in the component library, which can improve the accuracy of component identification and matching, and thus improve the accuracy of security detection.

[0155] In one embodiment, the step of obtaining the code information of the component to be identified from the code platform can be implemented according to the following steps 1 to 3:

[0156] Step 1: According to the latest collection frequency, obtain the code information of the component to be identified and the feature information of the component to be identified in multiple feature dimensions from the code platform.

[0157] Here, the collection frequency is used to indicate the frequency of collecting components to be identified from the code platform. Since the collection frequency is constantly updated, the latest collection frequency can be the latest determined collection frequency. Feature information under multiple feature dimensions can include submission frequency C under code submission frequency, modification number B under vulnerability (bug) fix number, interaction number I under developer interaction number, number M under merge request number, and review frequency R under code review frequency.

[0158] In specific implementation, the disclosed embodiment provides an automatic code collection method for multi-dimensional real-time monitoring. Based on this method, code information of each component to be identified and feature information corresponding to each component to be identified under multiple feature dimensions can be collected from a multi-source platform according to the latest collection frequency.

[0159] Step 2: According to the latest weight coefficient corresponding to each feature dimension, weighted processing is performed on the feature information under multiple feature dimensions to obtain the first platform activity corresponding to the component to be identified.

[0160] Here, the weight coefficient of the feature dimension may also be continuously updated, and the latest weight coefficient may be the weight coefficient most recently determined for the feature dimension.

[0161] The first platform activity is used to characterize the platform activity of the code platform to which the component to be identified belongs during the current collection.

[0162] Exemplarily, for each component to be identified in any code platform, the feature information of the component to be identified in each feature dimension can be weighted according to the latest weight coefficient corresponding to each feature dimension to obtain the activity of the component to be identified. If the code platform only collects one component to be identified during the current collection, the activity of the component to be identified can be used as the first platform activity of the code platform. If the code platform collects multiple components to be identified during the current collection, the first platform activity of the code platform can be determined based on the average of the activities of the multiple components to be identified.

[0163] For example, the activity of the first platform may be determined according to the following formula 3:

[0164] A=a1·C+a2·B+a3·I+a4·M+a5·R; (Formula 3)

[0165] Among them, A represents the platform activity, and a1~a5 are the latest weight coefficients under each feature dimension.

[0166] Step 3: According to the activity of the first platform, the latest collection frequency is adjusted to obtain an adjusted collection frequency; the adjusted collection frequency is used to obtain the component to be identified from the code platform next time.

[0167] In specific implementation, after determining the activity of the first platform corresponding to the current acquisition based on the component to be identified obtained in the current acquisition, the latest acquisition frequency can be adjusted according to the frequency range to which the activity of the first platform belongs to obtain the adjusted acquisition frequency. The adjusted acquisition frequency will be used as the latest acquisition frequency and is used to indicate the frequency of obtaining the component to be identified from the code platform next time.

[0168] Specifically, the acquisition frequency can be adjusted according to the following formula 4:

[0169] ; (Formula 4)

[0170] Where F represents the adjusted acquisition frequency, , and They are the pre-set maximum collection frequency, medium collection frequency and minimum collection frequency respectively, and A represents the activity of the first platform. and Indicates the pre-set activity high and low thresholds, which are used to control the collection frequency in a hierarchical manner, thereby optimizing the frequency of data collection and resource utilization efficiency.

[0171] Exemplarily, after obtaining the first platform activity corresponding to the current collection, the adjusted collection frequency corresponding to the first platform activity can be determined according to the relationship between the first platform activity and the activity high and low thresholds.

[0172] Furthermore, after obtaining the activity of the first platform, the latest weight coefficient used in calculating the activity of the first platform can be updated so as to calculate the new activity of the first platform after the next collection. Specifically, the latest weight coefficient can be updated according to the following steps 3 to 4:

[0173] Step three: determine the activity prediction loss based on the first platform activity corresponding to the component to be identified, the historical platform activity of each historical component collected before the component to be identified, and the second platform activity of each historical component under the latest weight coefficient.

[0174] Here, the historical components can be the various components to be identified that were collected before the components to be identified were collected, that is, the components obtained in each historical collection after the current collection. The historical platform activity is the platform activity determined according to the weight coefficients corresponding to the historical collections when the historical components are collected. When the weight coefficients of any code platform are updated, the components to be identified and the historical components used are all components collected from the platform. The activity prediction loss is used to measure the loss between the activities predicted using different weight coefficients. The second platform activity is used to characterize the platform activity obtained after weighted processing of the feature information under each feature dimension obtained when collecting historical components using the latest weight coefficient corresponding to the current collection.

[0175] In specific implementation, the activity prediction loss can be determined according to the following formula 5:

[0176] ; (Formula 5)

[0177] in, represents the activity prediction loss; n represents the number of historical components, where the components to be identified obtained in the current collection can also be used as historical components; represents the historical platform activity corresponding to the j-th historical component, represents the activity of the second platform of the jth historical component under the latest weight coefficient; when the component to be identified obtained in the current collection is also used as a historical component, the corresponding and They can be the same, both are the activity levels on the first platform.

[0178] In specific implementation, for any code platform, the first platform activity corresponding to the component to be identified collected in the code platform in the current collection, the historical platform activity corresponding to each historical component collected in the code platform in historical times, and the second platform activity corresponding to each historical component under the latest weight coefficient under the current collection can be substituted into the above formula 5 to obtain the predicted activity loss of the code platform.

[0179] Step 4: Use the activity prediction loss to update each latest weight coefficient to obtain an updated weight coefficient; the updated weight coefficient is used to determine the first platform activity of the component to be identified that is collected next time.

[0180] In specific implementation, the updated weight coefficient can be determined according to the following formula 6:

[0181] ; (Formula 6)

[0182] in, Indicates the time of t+1 acquisition The corresponding weight coefficient is the updated weight coefficient. The i in can be 1~5, corresponding to a1~a5 in the above text. Indicates the t-th acquisition time The corresponding weight coefficient is the latest weight coefficient corresponding to the current acquisition. B represents the learning rate.

[0183] In specific implementation, for the weight coefficient corresponding to any feature dimension, the updated weight coefficient corresponding to the feature dimension can be determined according to the above formula 6, based on the latest weight coefficient corresponding to the feature dimension at the time of the current collection, and the activity prediction loss corresponding to the current collection and the partial derivative of the latest weight coefficient. The updated weight coefficient corresponding to each feature dimension can be used to determine the first platform activity corresponding to the code platform of the component to be identified according to the above formula 3 when the component to be identified is collected next time.

[0184] In this way, by introducing automated source code collection and community activity monitoring mechanisms, the latest developments of open source components, including code submissions, bug fixes, and developer interaction information, can be obtained in real time, ensuring the timeliness and integrity of component data, and effectively avoiding the problem of data lag in traditional methods. In addition, by using code submissions, bug fixes, and developer interaction information to adjust the collection frequency in real time, the frequency of data collection and resource utilization efficiency can be optimized.

[0185] In one embodiment, the first neural network may be an ML-AGCN including multiple network layers. The above S103 may be implemented according to the following steps:

[0186] S103-1: For any network layer, obtain the first feature vector of each network node output by the previous network layer and the node relationship feature vector of each network node; wherein, when the network layer is the first layer, the first feature vector of each network node output by the previous network layer is the feature attribute matrix of each network node, and the node relationship feature vector of each network node is the edge weight of the directed edge of each network node.

[0187] Here, the first feature vector is used to represent the final prediction vector extracted by the network layer for the network node; that is, the first feature vector is the final prediction vector output by the network layer for the network node, and one network layer outputs one first feature vector for the network node. The node relationship feature vector is used to represent the calling relationship of the network layer for the network node, and one network layer outputs one node relationship feature vector for the network node.

[0188] Exemplarily, when the network layer is the first network layer, the first feature vector of each network node output by the previous network layer can be the feature attribute matrix determined for each network node using the above S102-3; the node relationship feature vector of each network node is the edge weight determined for the directed edge of each network node using the above formula 1.

[0189] When the network layer is not the first network layer, the first feature vector of each network node output by the previous network layer is the first feature vector output by the previous network layer of the current network layer; the node relationship feature vector of each network node is the node relationship feature vector of each network node output by the previous network layer of the current network layer.

[0190] S103 - 2 : Determine a second feature vector of the network node according to the first feature vectors of each neighboring node of the network node and the node relationship feature vector of the network node.

[0191] Exemplarily, for each network node, the second feature vector of the network node may be determined according to the first feature vectors of each neighboring node of the network node and the node relationship feature vector between the network node and each neighboring node.

[0192] Exemplarily, the second eigenvector of each network node may be determined using the following formula 7:

[0193] ; (Formula 7)

[0194] in, Represents a network node i exist l The second eigenvector of the +1 layer, σ represents the activation function, Represents a network node i The adjacent node set composed of each adjacent node. Represents a network node i and network nodes j The edge weight between them, when the network layer is a non-first-layer network, Represents a network node i and network nodes j The node relationship feature vector between them. and Represents network nodes i and network nodes j The degree of Indicates l The network layer weight matrix of the network layer, which is fixed after the network training is completed. Represents a network node i Neighboring nodes j exist l The first eigenvector of the layer.

[0195] In this way, based on the above formula 7, the second eigenvector corresponding to each network node in the network layer can be determined.

[0196] S103-3: Determine the aggregation weights of the network node and each adjacent node in each attention head according to the weight matrix of each attention head in the network layer, the second eigenvector of the network node, and the second eigenvector of each adjacent node of the network node.

[0197] Here, the ML-AGCN provided in the embodiment of the present disclosure also uses a multi-head attention mechanism to aggregate node features from multiple perspectives, so as to better step the complex dependencies and call relationships between nodes. Therefore, multiple attention heads can also be set in each network layer, and the number of attention heads can be K, the number of K is greater than 1 and the value of K can be set according to experience, which is not specifically limited in the embodiment of the present disclosure.

[0198] Each attention head has a weight matrix, which is fixed after the network training is completed.

[0199] In specific implementation, for any network node, the aggregation weights of the network node and each adjacent node corresponding to each attention head can be determined according to the following formula 8:

[0200] ; (Formula 8)

[0201] in, Represents a network node i and network nodes j (i.e. network node i The aggregate weight of the neighboring nodes of the kth attention head. z is a learnable weight vector, which is a fixed value after the network is trained. represents the transpose of z, LeakyReLU represents the ReLU function with a negative slope, and ReLU represents the activation function. represents the weight matrix of the kth attention head, Represents a network node i The second eigenvector of Represents a network node j The second eigenvector of ; || represents the concatenation operation in the multi-head attention mechanism, which is used to indicate that the output results of different attention heads are concatenated.

[0202] S103-4: Determine the node relationship feature vector output by the network node at the network layer according to the aggregation weights corresponding to the network node and each adjacent node in each attention head.

[0203] Exemplarily, for any network node, the mean / variance value of the aggregation weights corresponding to the network node and each adjacent node in each attention head can be used as the node relationship feature vector output by the network node at the network layer.

[0204] S103-5: According to the aggregation weights corresponding to the network node and each adjacent node in each attention head, the second feature vectors of each adjacent node of the network node are aggregated to obtain the first feature vector output by the network node in the network layer; wherein the first feature vector of each network node output by the last network layer is used as the embedded feature vector of each network node; and the node relationship feature vector of each network node output by the last network layer is used as the relationship feature vector of each network node.

[0205] In specific implementation, the first feature vector output by each network node in the current network layer may be determined according to the following formula 9:

[0206] ; (Formula 9)

[0207] in, Represents a network node i exist l The first eigenvector output by the +1 layer, k ranges from 1 to K, where K is the number of attention heads. Represents a network node i The adjacent node set composed of each adjacent node. Represents a network node j exist l +1 layer corresponding to the second eigenvector; Represents a network node i and network nodes j (i.e. network node i The aggregate weights of the neighboring nodes of the k-th attention head.

[0208] Among them, when the current network layer is not the last network layer, the first feature vector of each network node output by the current network layer and the node relationship feature vector of each network node can be input into the next network layer for further feature extraction; if the current network layer is the last network layer, the first feature vector of each network node output by the last network layer can be used as the embedded feature vector of each network node output by ML-AGCN, and the node relationship feature vector of each network node output by the last network layer can be used as the relationship feature vector of each network node output by ML-AGCN.

[0209] In this way, by combining the original code features obtained from static code analysis and the multi-layer adaptive graph neural network (ML-AGNN) technology, the code structure features, deep function call relationships, and deep function dependencies of the components to be identified can be fully extracted. ML-AGNN uses a multi-layer adaptive propagation mechanism in the extraction process, which improves the accuracy of feature capture and cross-layer feature expression capabilities while retaining the complex dependencies of the components to be identified, thereby significantly improving the accuracy and efficiency of component identification.

[0210] In one embodiment, the above S104 may be implemented according to the following steps:

[0211] S104-1: Input the embedded feature vector and the relationship feature vector of each network node into the first layer autoencoder in the multi-layer autoencoder to obtain the intermediate feature vector and the intermediate relationship feature vector of each network node.

[0212] Exemplarily, the embedded feature vector and the relationship feature vector of each network node can be input into the first-layer autoencoder in the multi-layer autoencoder in the form of vector matrices. The first-layer autoencoder retains the relationship information between network nodes through the vector matrix corresponding to the relationship feature vector, and performs feature dimension reduction and extraction on the vector matrix corresponding to the embedded feature vector and the vector matrix corresponding to the relationship feature vector, respectively, to obtain the intermediate feature vector and intermediate relationship feature vector of each network node output by the first-layer autoencoder.

[0213] The autoencoder can determine the intermediate feature vector and the intermediate relationship feature vector of each network node according to the following formula 10 and formula 11 respectively:

[0214] ; (Formula 10)

[0215] ; (Formula 11)

[0216] in, A feature vector matrix consisting of the intermediate feature vectors of each network node output by the autoencoder, including the intermediate feature vectors of each network node; Represents the feature vector matrix input to the autoencoder. When the autoencoder is the first layer, Includes the embedded feature vector of each network node. When the autoencoder is not the first layer, is a matrix composed of each intermediate feature vector output by the previous layer of autoencoder. θ is the preset angle parameter. is the feature vector matrix mapping function corresponding to the autoencoder.

[0217] A relation vector matrix consisting of the intermediate relation feature vectors of each network node output by the autoencoder, including the intermediate relation feature vectors of each network node; Represents the relational vector matrix input to the autoencoder. When the autoencoder is the first layer, Includes the relational feature vector of each network node. When the autoencoder is not the first layer, A matrix consisting of each intermediate relation feature vector output by the previous layer of autoencoder. Represents the relationship vector matrix mapping relationship corresponding to the autoencoder.

[0218] S104-2: Input the intermediate feature vector and intermediate relationship feature vector of each network node to the next layer of autoencoder to obtain a new intermediate feature vector and a new intermediate relationship feature vector of each network node.

[0219] Exemplarily, the intermediate feature vectors and intermediate relationship feature vectors of each network node output by the first layer autoencoder are still input to the next layer autoencoder in the form of vector matrices. The next layer autoencoder determines the new intermediate feature vectors of each network node according to the following formula 12:

[0220] ; (Formula 12)

[0221] in, Indicates l The feature vector matrix output by the layer autoencoder, Indicates l The feature vector matrix mapping function of the layer autoencoder, Indicates l -1 layer of autoencoder output feature vector matrix, Indicates l The preset angle parameters corresponding to the layer autoencoder.

[0222] Similarly, the next layer of autoencoder can also determine the new intermediate relationship feature vector of each network node according to the formula related to the relationship vector matrix (similar to the principle of formula 11 above, which will not be repeated here).

[0223] S104-3: Return to the step of inputting the intermediate feature vector and the intermediate relationship feature vector of each network node into the next layer of autoencoder, until the target feature vector of each network node output by the last layer of autoencoder is obtained.

[0224] In specific implementation, after obtaining the intermediate feature vector and intermediate relationship feature vector output by any layer of autoencoder, the vector matrix of the intermediate feature vector and the intermediate relationship feature vector can be input into the next layer of autoencoder for further feature extraction, until the intermediate feature vector of each network node output by the last layer of autoencoder is obtained, and the intermediate feature vector is used as the target feature vector of each network node. Among them, by performing feature dimensionality reduction through the autoencoder, the target feature vector after dimensionality reduction can not only retain the relationship information between network nodes, but also have a high degree of nonlinear expression ability.

[0225] Among them, regarding the last layer of autoencoder, it can output the intermediate relationship feature vector of each network node, but the intermediate relationship feature vector will be ignored during the application process.

[0226] Optionally, after obtaining the target feature vector of each network node output by the last layer of autoencoder, the target feature vector of each network node can also be mapped to the final feature space to ensure that the component library can be updated using the target feature vector in a unified format.

[0227] In this way, a multi-layer relational stacked autoencoder (MRSAE) is used to perform multi-level nonlinear dimensionality reduction on the high-dimensional features output by ML-AGNN, retaining the relationship information between component data and effectively reducing feature redundancy and computational overhead. Using multi-layer autoencoders for feature extraction can improve the density and information integrity of low-dimensional feature representation, significantly improving the effect of subsequent recognition.

[0228] In one embodiment, S105 may be implemented according to the following steps:

[0229] S105-1: Determine a first function list corresponding to the component to be identified according to each function and class of the component to be identified, and determine a second function list corresponding to each identified component according to each function and class in each identified component.

[0230] Here, the function list of the component can be used as a matching feature in the coarse-grained matching stage to filter out each candidate component from multiple identified components included in the component library. The first function list is a list of functions and classes in the component to be identified, and the second function list is a list of functions and classes in the identified component.

[0231] In specific implementation, for the component to be identified, each function and class determined from the code information of the component to be identified can be used to form a first function list of the component to be identified. In addition, each function and class determined from the code information of the identified component can be used to form a second function list of the identified component.

[0232] S105 - 2 : Determine the component similarity between each identified component and the component to be identified according to the first function list and each second function list.

[0233] S105-3: According to the component similarity, select candidate components similar to the component to be identified from the identified components.

[0234] Exemplarily, the candidate components may be determined according to the following formula 13:

[0235] ; (Formula 13)

[0236] in, EFPools Represents the function pool of each identified component and class component, and Candidate represents the candidate component; represents the component similarity between the candidate component and the component to be identified, a second list of functions representing the nth candidate component, represents the first function list of the component to be identified, M is used to indicate whether the component to be identified is similar to the identified component, and then determine whether the identified component can be used as a candidate component. ∩ represents the intersection symbol.

[0237] S105-4: Determine a matching component according to the target feature vector of each network node in the component to be identified and the target feature vector of each network node in each candidate component.

[0238] In specific implementation, the matching degree between each first network node and the matching second network node can be determined according to the target feature vector of each first network node in each component to be identified and the target feature vector of the second network node matching each first network node in the candidate component, and the similarity between the component to be identified and the candidate component can be determined according to the matching degree corresponding to each first network node. The candidate component with the largest similarity and not less than the preset similarity is used as the matching component.

[0239] In one embodiment, S105-4 may be implemented according to the following steps:

[0240] S105-4-1: Determine a first similarity between the component to be identified and each candidate component according to the original code feature of each network node in the component to be identified and the original code feature of each network node in each candidate component.

[0241] Here, the first similarity is used to characterize the structural similarity between the component to be identified and the candidate component. For each candidate component, a multi-scale feature measurement model is constructed, and multi-layer nested feature descriptors are introduced in terms of structural similarity to capture fine-grained and coarse-grained structural features. For the feature set X of the component to be identified, a multi-scale similarity measurement method is used to calculate the structural similarity between the target component and the candidate component.

[0242] Since the component to be identified and the candidate components are similar components determined based on the function list, the component to be identified and the candidate components usually include the same network nodes, so the feature set X composed of the original code features of each network node of the candidate component and the feature set X composed of the original code features of each network node of the component to be identified include the same number of features.

[0243] In specific implementation, the first similarity between the component to be identified and each candidate component may be determined according to the following formula 14:

[0244] ; (Formula 14)

[0245] in, represents the first similarity, and the number of i is the total number of features in the feature set X composed of the original code features of each network node in the component to be identified.

[0246] q i It represents the weight corresponding to the i-th original code feature in the feature set X, which can be a preset weight. represents the i-th original code feature in the feature set X of the component to be identified, represents the i-th original code feature in the feature set X of the candidate component, Represents the similarity measure between the i-th original code feature of the component to be identified and the i-th original code feature of the candidate component.

[0247] Exemplarily, the original code features of each network node in the component to be identified can be combined into a feature set X corresponding to the component to be identified, and the original code features of each network node in each candidate component can be combined into a feature set X corresponding to the candidate component. Then, for the component to be identified and any candidate component, the above formula 14 can be used to determine the first similarity between the component to be identified and the candidate component.

[0248] S105-4-2: Determine the second similarity between the component to be identified and each candidate component based on the target feature vector of each network node in the component to be identified, the target feature vector of each network node in each candidate component, and the edge weight of the directed edge between every two network nodes.

[0249] Here, since the component to be identified and the candidate component are similar components determined based on the function list, the functions and classes included in the component to be identified may be consistent with the functions and classes included in the candidate component, so the network nodes in the directed network graph of the component to be identified can also correspond one-to-one with the network nodes in the candidate component. The second similarity is used to characterize the functional similarity between the component to be identified and the candidate component.

[0250] In specific implementation, the following formula 15 may be used to determine the second similarity between the component to be identified and each candidate component;

[0251] ; (Formula 15)

[0252] in, represents the second similarity, Represents the network node in the component to be identified i and network nodes j The edge weights of the directed edges between i and j The number of is related to the number of network nodes in the component to be identified. i and network nodes j It exists in both the components to be identified and the candidate components.

[0253] Represents the network node in the component to be identified i The corresponding target feature vector; Represents a network node in a candidate component j The corresponding target feature vector.

[0254] Exemplarily, the above formula fifteen can be used to determine the second similarity between the component to be identified and each candidate component based on the target feature vector of each network node in the component to be identified, the target feature vector of each network node in each candidate component, and the edge weight of the directed edge between each two network nodes in the component to be identified.

[0255] S105-4-3: Determine a third similarity between the component to be identified and each candidate component according to the identification time of the component to be identified and the identification time of each candidate component.

[0256] Here, the recognition time of the component to be identified is the time when the component to be identified is collected from the code platform during the current collection. The recognition time of the candidate component is the time when the candidate component is collected from the code platform. If the candidate component does not have an update in the component library, the recognition time is the time when the candidate component is first identified; if the candidate component has an update in the component library, the recognition time is the time when the candidate component is collected when the candidate component is updated.

[0257] The third similarity is used to characterize the time similarity between the component to be identified and the candidate component. In the time similarity matching, a time decay function is introduced to adapt to the update frequency and version changes of the component. For the component with a recognition time difference of △t, the third similarity can be determined according to the following formula 16:

[0258] ; (Formula 16)

[0259] in, represents the third similarity, λ is the pre-determined time decay parameter, and △t represents the time difference between the recognition time of the component to be recognized and the recognition time of the candidate component.

[0260] S105-4-4: Determine a target similarity between the component to be identified and each candidate component according to the first similarity, the second similarity, and the third similarity, and determine a matching component according to the target similarity.

[0261] Here, the target similarity is used to characterize the overall similarity between the component to be identified and the candidate component.

[0262] Exemplarily, the target similarity between the component to be identified and the candidate component can be determined using the following formula 17:

[0263] ; (Formula 17)

[0264] Among them, S represents the target similarity between the component to be identified and the candidate component, They represent weights set for the first similarity, the second similarity and the third similarity respectively, and each weight can be adaptively adjusted based on experience.

[0265] Furthermore, after determining the target similarity between the component to be identified and each candidate component using the above formula 17, the maximum target similarity can be determined. It is determined whether the maximum target similarity is not less than the preset similarity. If so, the candidate component corresponding to the maximum target similarity is used as the matching component; if not, it is determined that there is no matching component for the component to be identified.

[0266] In this way, the multi-dimensional similarity matching method based on structural similarity, functional similarity and temporal similarity can identify the target component more comprehensively and accurately through a context-sensitive matching mechanism. This matching method not only improves the recognition accuracy, but also effectively reduces the false alarm rate, which is significantly better than the existing single-dimensional matching technology.

[0267] like Figure 2 As shown, a specific process diagram of determining a matching component from a component library provided by an embodiment of the present disclosure may include the following steps:

[0268] Extract key features of the component to be identified; wherein the key features may specifically be target feature vectors of each network node in the component to be identified.

[0269] Coarse-grained matching: using the derived function list to filter and obtain a candidate component set; wherein, this step can refer to the above S105-1 to S105-3, and will not be repeated here. The candidate component set includes at least one candidate component.

[0270] Multi-dimensional similarity calculation: Structural similarity calculation is performed through cosine similarity matching to obtain a structural similarity score (i.e., a first similarity score); functional similarity calculation is performed by comparing function calls and dependencies to obtain a functional similarity score (i.e., a second similarity score); temporal similarity calculation is performed by comparing component development and update time to obtain a temporal similarity score (i.e., a third similarity score). The component development and update information can be the identification time mentioned above.

[0271] The similarity scores of each dimension are weighted and summed to obtain the target similarity.

[0272] Filter out candidate components with the highest target similarity.

[0273] For the specific implementation process of the above steps, please refer to the above embodiments and will not be repeated here.

[0274] To facilitate understanding of the embodiments of the present disclosure, the embodiments of the present disclosure provide Figure 3 The specific implementation process of the component matching method shown may include the following steps:

[0275] Code platform collection components and platform activity monitoring: The purpose of this step is to collect components to be identified from the code platform and update the collection frequency according to the platform activity.

[0276] Multimodal data cleaning and standardization.

[0277] Multi-dimensional basic feature extraction, this step is to extract the original code features of each function and class from the standard code information.

[0278] Feature extraction based on a multi-layer adaptive graph neural network. This step specifically includes taking each function and class as a network node, constructing a directed graph network based on the original code features. The directed graph network is a multi-level function call graph used to characterize at least one level of calling relationships of functions and classes in the component to be identified; applying Gumbel-Softmax feature selection; and using a multi-head attention mechanism to capture complex dependencies. This step is to use a multi-layer attention mechanism for deep feature extraction.

[0279] A multi-layer autoencoder is used to extract the key features of the component to be identified and construct a feature library. The purpose of this step is to extract the target feature vector of each network node in the component to be identified and use the target feature vector to construct the feature library.

[0280] Multi-dimensional similarity matching; this step specifically includes performing coarse-grained matching to determine a set of candidate components; calculating structural similarity, functional similarity, and temporal similarity; and identifying the matching components with the highest similarity.

[0281] For the specific implementation process of the above steps, please refer to the above embodiments and will not be repeated here.

[0282] In the above process, open source component data is obtained from multiple source platforms through multi-threaded data collection tools, combined with an adaptive community activity monitoring model to ensure the comprehensiveness and timeliness of the data. Multimodal data is cleaned and standardized, and multi-dimensional basic features are extracted; the improved multi-layer adaptive graph neural network (ML-AGNN) is used to extract deep dependency features, and the multi-layer relational stacking autoencoder (MRSAE) is combined for nonlinear dimensionality reduction and key feature extraction. Finally, a multi-dimensional similarity matching method is used to identify components from three dimensions: structure, function, and time similarity, which significantly improves the accuracy, real-time, and intelligence of recognition.

[0283] Based on the above embodiments, it can be known that the embodiments of the present disclosure use a first neural network, so the present disclosure also provides the following steps of training the first neural network as shown in S1 to S4:

[0284] S1: Obtain a sample directed graph network corresponding to a first sample component; the sample directed graph network includes a plurality of sample network nodes.

[0285] Here, the first sample component is a component used for network training. The sample directed graph network includes sample network nodes corresponding to each function and class in the first sample component. Each sample network node includes information such as the edge weight of the directed edge corresponding to the node, the feature attribute matrix, etc.

[0286] Exemplarily, a large amount of sample data may be collected, and one piece of sample data is a sample directed graph network corresponding to a first sample component.

[0287] S2: Input the sample directed graph network into the first neural network to be trained, and obtain the predicted second feature vector and predicted node relationship feature vector of each sample network node output by the first network layer of the first neural network to be trained, and the predicted embedding feature vector and predicted relationship feature vector of each sample network node output by the last network layer of the first neural network to be trained.

[0288] Here, the predicted second feature vector is the node relationship feature vector predicted for the sample network node by the network layer in the first neural network to be trained; the predicted second feature vector is the second feature vector predicted for the sample network node by the network layer in the first neural network to be trained. The predicted embedded feature vector is the embedded feature vector predicted for the sample network node by the first neural network to be trained; the predicted relationship feature vector is the relationship feature vector predicted for the sample network node by the first neural network to be trained.

[0289] Exemplarily, the sample directed graph network can be input into the first neural network to be trained to obtain the predicted second feature vector and predicted node relationship feature vector of each sample network node output by the first network layer, and the predicted embedding feature vector and predicted relationship feature vector of each sample network node output by the last network layer.

[0290] S3: Determine a first prediction loss of the first neural network to be trained according to the predicted second feature vector, the predicted node relationship feature vector, the predicted embedding feature vector and the predicted relationship feature vector.

[0291] Exemplarily, the prediction loss of the first neural network to be trained can be determined based on the error between a vector matrix composed of the predicted second feature vectors of each sample network node and a vector matrix composed of the predicted embedded feature vectors of each sample network node, and the error between a vector matrix composed of the predicted node relationship feature vectors of each sample network node and a vector matrix composed of the predicted relationship feature vectors of each sample network node, and this loss is defined as the first loss.

[0292] Specifically, a cross entropy loss function or a negative log-likelihood loss (Negative Log-Likelihood Loss, NLLLoss for short) may be used to determine a first prediction loss of a first neural network to be trained.

[0293] S4: Iteratively train the first neural network to be trained using the first prediction loss to obtain a trained first neural network.

[0294] In specific implementation, the determined first prediction loss can be used to iteratively train the first neural network to be trained until a training cutoff condition is met to obtain a trained first neural network; wherein the training cutoff condition is that the prediction accuracy of the trained network reaches a preset accuracy and / or the number of iterative training rounds reaches a preset number of rounds.

[0295] For example, the parameters of the first neural network to be trained can be updated based on the first prediction loss through an adaptive moment estimation (Adam) optimizer. The training process lasts for several epochs. In each epoch, the model continuously updates parameters such as the network layer weight matrix of each network layer and the weight matrix of the attention head in each network layer to maximize the prediction accuracy of the network.

[0296] In another embodiment, the present disclosure also provides the following steps of training a multi-layer autoencoder as shown in P1 to P5:

[0297] P1: Obtain the sample embedding feature vector and sample relationship feature vector of each sample network node in the sample directed graph network corresponding to the second sample component.

[0298] Here, the second sample component may include the first sample component and / or other sample components different from the first sample component.

[0299] In the case where the second sample component includes the first sample component, the sample embedding feature vector and the sample relationship feature vector may be the predicted embedding feature vector and the predicted relationship feature vector output by the first neural network for the first sample component during the training of the first neural network, or may be the predicted embedding feature vector and the predicted relationship feature vector output by the trained first neural network for the first sample component.

[0300] In the case where the second sample component includes other sample components different from the first sample component, the sample embedding feature vector and the sample relationship feature vector may be the predicted embedding feature vector and the predicted relationship feature vector output by the trained first neural network for the second sample component.

[0301] P2: Input the sample embedding feature vector and sample relationship feature vector of each sample network node into the multi-layer autoencoder to be trained, and obtain the predicted target feature vector and predicted sample relationship feature vector of each network node output by the last layer of autoencoder.

[0302] Here, the predicted target feature vector is the intermediate feature vector output by the last layer of the autoencoder, and the predicted sample relationship feature vector is the intermediate relationship feature vector output by the last layer of the autoencoder.

[0303] P3: Use the decoder in the multi-layer autoencoder to be trained to decode the predicted target feature vector and the predicted sample relationship feature vector of each network node to obtain a decoded feature vector and a decoded relationship feature vector.

[0304] In specific implementation, the decoder in the multi-layer autoencoder to be trained and the decoder located after the last layer of autoencoder can be used to decode the predicted feature vector matrix composed of the predicted target feature vectors of each network node to obtain a decoded feature vector matrix, which includes each decoded feature vector; at the same time, the above-mentioned decoder can be used to decode the predicted relationship vector matrix composed of the predicted sample relationship feature vectors of each network node to obtain a decoded relationship vector matrix, which includes each decoded relationship feature vector.

[0305] P4: Determine the second prediction loss of the multi-layer autoencoder to be trained based on the decoded feature vector, the predicted target feature vector, the predicted sample relationship feature vector and the decoded relationship feature vector.

[0306] Exemplarily, the second prediction loss may be determined according to the following formula 18:

[0307] ; (Formula 18)

[0308] in, Represents the second prediction loss, which is used to characterize the loss between the feature vector matrix and the relationship vector matrix reconstructed by the decoder and the feature vector matrix and the relationship vector matrix corresponding to the output of the last layer of the autoencoder. represents the predicted feature vector matrix, Represents the decoded feature vector matrix. c is a trade-off coefficient for balancing the errors of data reconstruction and relationship reconstruction, and is a value adaptively adjusted based on experience. represents the predicted relationship vector matrix, Represents the decoded relation vector matrix.

[0309] Optionally, in the above formula 18 It can also be a matrix constructed for the edge weights of each sample network node corresponding to the second sample component. It may also be a matrix composed of predicted target feature vectors output by each sample network node corresponding to the second sample component for the first layer autoencoder of the multi-layer autoencoder to be trained.

[0310] P5: Use the second prediction loss to iteratively train the multi-layer autoencoder to obtain a trained multi-layer autoencoder.

[0311] In specific implementation, the determined second prediction loss can be used to iteratively train the multi-layer autoencoder to be trained until the training cutoff condition is met to obtain a trained multi-layer autoencoder; wherein the training cutoff condition is that the prediction accuracy of the trained network reaches a preset accuracy and / or the number of iterative training rounds reaches a preset number of rounds.

[0312] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0313] Based on the same inventive concept, a component matching device corresponding to the component matching method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned component matching method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0314] like Figure 4 FIG. 1 is a schematic diagram of a component matching device provided by an embodiment of the present disclosure, comprising:

[0315] The first extraction module 401 is used to extract the original code features of each function and class in the component to be identified based on the code information of the component to be identified obtained from the code platform; the original code features include code attribute features, call relationship features and dependency relationship features;

[0316] A construction module 402 is used to construct a directed graph network according to the original code features by using each function and class as a network node;

[0317] A second extraction module 403 is used to input the directed graph network into a pre-trained first neural network to obtain an embedded feature vector of each of the network nodes and a relationship feature vector between each of the network nodes and other network nodes;

[0318] The encoding module 404 is used to use a pre-trained multi-layer autoencoder to perform feature encoding on the embedded feature vector and the relationship feature vector of each network node in turn to obtain a target feature vector of each network node;

[0319] The determination module 405 is used to determine a matching component that matches the component to be identified from each identified component in a pre-built component library according to the target feature vector of each network node of the component to be identified.

[0320] In a possible implementation, the device further includes an acquisition module 406, configured to acquire code information of the component to be identified from the code platform according to the following steps:

[0321] Acquire, from the code platform, code information of the component to be identified and feature information of the component to be identified in multiple feature dimensions according to the latest acquisition frequency;

[0322] According to the latest weight coefficient corresponding to each of the feature dimensions, weighted processing is performed on the feature information under the multiple feature dimensions to obtain the first platform activity corresponding to the component to be identified;

[0323] According to the activity of the first platform, the latest acquisition frequency is adjusted to obtain an adjusted acquisition frequency; the adjusted acquisition frequency is used for obtaining the component to be identified from the code platform next time.

[0324] In a possible implementation, after determining the activity of the first platform, the acquisition module 406 is further configured to:

[0325] Determine the activity prediction loss according to the first platform activity corresponding to the component to be identified, the historical platform activity of each historical component collected before the component to be identified, and the second platform activity of each historical component under the latest weight coefficient;

[0326] Using the activity prediction loss, each of the latest weight coefficients is updated to obtain an updated weight coefficient; the updated weight coefficient is used to determine the first platform activity of the component to be identified that is collected next time.

[0327] In a possible implementation manner, the first extraction module 401, when extracting original code features of each function and class in the component to be identified based on the code information of the component to be identified obtained from the code platform, is used to:

[0328] Performing data cleaning and standardization processing on the code information of the component to be identified to obtain standard code information;

[0329] The original code features of each function and class are extracted from the standard code information using a code analysis tool.

[0330] In a possible implementation, the construction module 402, when constructing a directed graph network based on the original code features using each function and class as a network node, is used to:

[0331] Determine the directed edges between the network nodes according to the call relationship features in the original code features;

[0332] Determining an edge weight of the directed edge according to a function call frequency, a call path depth, and a call parameter complexity between two network nodes corresponding to the directed edge;

[0333] Determining a characteristic attribute matrix of each network node according to the original code characteristics of each network node;

[0334] The directed graph network is constructed according to the directed edges between the network nodes, the edge weights of the directed edges, and the characteristic attribute matrix of each network node.

[0335] In a possible implementation, the first neural network includes multiple network layers;

[0336] The second extraction module 403, when inputting the directed graph network into the pre-trained first neural network to obtain the embedded feature vector of each network node and the relationship feature vector between each network node and other network nodes, is used to:

[0337] For any of the network layers, obtain the first feature vector of each network node output by the previous network layer and the node relationship feature vector of each of the network nodes; wherein, when the network layer is the first layer, the first feature vector of each network node output by the previous network layer is the feature attribute matrix of each of the network nodes, and the node relationship feature vector of each of the network nodes is the edge weight of the directed edge of each of the network nodes;

[0338] Determining a second feature vector of the network node according to first feature vectors of each neighboring node of the network node and a node relationship feature vector of the network node;

[0339] Determine, according to the weight matrix of each attention head in the network layer, the second eigenvector of the network node, and the second eigenvector of each neighboring node of the network node, the aggregation weight of the network node and each neighboring node corresponding to each attention head;

[0340] Determine a node relationship feature vector output by the network node at the network layer according to the aggregation weights corresponding to the network node and each adjacent node at each attention head;

[0341] According to the aggregation weights corresponding to the network node and each adjacent node in each of the attention heads, the second feature vectors of each adjacent node of the network node are aggregated to obtain the first feature vector output by the network node in the network layer; wherein the first feature vector of each network node output by the last network layer is used as the embedded feature vector of each network node; and the node relationship feature vector of each network node output by the last network layer is used as the relationship feature vector of each network node.

[0342] In a possible implementation manner, the construction module 402, when determining the characteristic attribute matrix of the network node according to the original code feature of each of the network nodes, is used to:

[0343] Determine the correlation between every two of the network nodes according to the original code features of each of the network nodes;

[0344] Using a feature selection matrix composed of correlations between every two of the network nodes, the original code features of each of the network nodes are screened to obtain a screened code feature of each of the network nodes;

[0345] According to the filtered code features of each of the network nodes, a feature attribute matrix of the network nodes is determined.

[0346] In a possible implementation, the encoding module 404, when using a pre-trained multi-layer autoencoder to sequentially perform feature encoding on the embedded feature vector and the relationship feature vector of each of the network nodes to obtain the target feature vector of each of the network nodes, is used to:

[0347] Inputting the embedded feature vector and the relationship feature vector of each of the network nodes into the first layer of the autoencoder in the multi-layer autoencoder to obtain the intermediate feature vector and the intermediate relationship feature vector of each of the network nodes;

[0348] Inputting the intermediate feature vector and the intermediate relationship feature vector of each of the network nodes into the next layer of autoencoder to obtain a new intermediate feature vector and a new intermediate relationship feature vector of each of the network nodes;

[0349] Return to the step of inputting the intermediate feature vector and the intermediate relationship feature vector of each of the network nodes to the next layer of autoencoder until the target feature vector of each of the network nodes output by the last layer of autoencoder is obtained.

[0350] In a possible implementation manner, the determination module 405, when determining, according to the target feature vector of each of the network nodes of the component to be identified, from the identified components in the pre-built component library, a matching component that matches the component to be identified, is configured to:

[0351] Determine a first function list corresponding to the component to be identified according to each function and class of the component to be identified, and determine a second function list corresponding to each identified component according to each function and class in each identified component;

[0352] Determining, according to the first function list and each of the second function lists, a component similarity between each of the identified components and the component to be identified;

[0353] According to the component similarity, selecting candidate components similar to the component to be identified from the identified components;

[0354] The matching component is determined according to the target feature vector of each of the network nodes in the component to be identified and the target feature vector of each of the network nodes in each of the candidate components.

[0355] In a possible implementation manner, the determination module 405, when determining the matching component according to the target feature vector of each of the network nodes in the component to be identified and the target feature vector of each of the network nodes in each of the candidate components, is configured to:

[0356] Determine a first similarity between the component to be identified and each of the candidate components according to the original code feature of each of the network nodes in the component to be identified and the original code feature of each of the network nodes in each of the candidate components;

[0357] Determine a second similarity between the component to be identified and each of the candidate components according to the target feature vector of each of the network nodes in the component to be identified, the target feature vector of each of the network nodes in each of the candidate components, and the edge weight of the directed edge between every two network nodes;

[0358] Determining a third similarity between the component to be identified and each of the candidate components according to the identification time of the component to be identified and the identification time of each of the candidate components;

[0359] According to the first similarity, the second similarity and the third similarity, a target similarity between the component to be identified and each of the candidate components is determined, and the matching component is determined according to the target similarity.

[0360] In a possible implementation manner, the determining module 405 is further configured to:

[0361] In the case where there is no matching component, the target feature vector and the original code feature of each network node of the component to be identified are added to the component library.

[0362] In a possible implementation manner, after determining the matching component that matches the component to be identified, the determination module 405 is further configured to:

[0363] The matching components in the component library are updated using the target feature vector and the original code feature of each network node of the component to be identified.

[0364] In a possible implementation, the device further includes a first training module 407, configured to train the first neural network according to the following steps:

[0365] Obtaining a sample directed graph network corresponding to the first sample component; the sample directed graph network includes a plurality of sample network nodes;

[0366] Inputting the sample directed graph network into a first neural network to be trained, obtaining a predicted second feature vector and a predicted node relationship feature vector of each sample network node output by a first network layer of the first neural network to be trained, and a predicted embedding feature vector and a predicted relationship feature vector of each sample network node output by a last network layer of the first neural network to be trained;

[0367] Determining a first prediction loss of the first neural network to be trained according to the predicted second feature vector, the predicted node relationship feature vector, the predicted embedding feature vector and the predicted relationship feature vector;

[0368] The first prediction loss is used to iteratively train the first neural network to obtain a trained first neural network.

[0369] In a possible implementation, the apparatus further includes a second training module 408, configured to train the multi-layer autoencoder according to the following steps:

[0370] Obtaining a sample embedding feature vector and a sample relationship feature vector of each sample network node in the sample directed graph network corresponding to the second sample component;

[0371] Input the sample embedding feature vector and the sample relationship feature vector of each sample network node into the multi-layer autoencoder to be trained, and obtain the predicted target feature vector and the predicted sample relationship feature vector of each network node output by the last layer of autoencoder;

[0372] Using the decoder in the multi-layer autoencoder to be trained, the predicted target feature vector and the predicted sample relationship feature vector of each network node are decoded to obtain a decoded feature vector and a decoded relationship feature vector;

[0373] Determining a second prediction loss of the multi-layer autoencoder to be trained according to the decoded feature vector, the predicted target feature vector, the predicted sample relationship feature vector, and the decoded relationship feature vector;

[0374] The multi-layer autoencoder to be trained is iteratively trained using the second prediction loss to obtain a trained multi-layer autoencoder.

[0375] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0376] Based on the same technical concept, the embodiment of the present application also provides a computer device. Figure 5 FIG. 1 is a schematic diagram of a computer device provided in an embodiment of the present application, including:

[0377] Processor 501 , memory 502 and bus 503 . Among them, the memory 502 stores machine-readable instructions executable by the processor 501, and the processor 501 is used to execute the machine-readable instructions stored in the memory 502. When the machine-readable instructions are executed by the processor 501, the processor 501 performs the following steps: S101: according to the code information of the component to be identified obtained from the code platform, the original code features of each function and class in the component to be identified are extracted; the original code features include code attribute features, call relationship features and dependency relationship features; S102: using each function and class as a network node, a directed graph network is constructed according to the original code features; S103: the directed graph network is input into a pre-trained first neural network to obtain the embedded feature vector of each network node, and the relationship feature vector between each network node and other network nodes; S104: using a pre-trained multi-layer autoencoder, feature encoding is performed on the embedded feature vector and the relationship feature vector of each network node in turn to obtain the target feature vector of each network node; and S105: according to the target feature vector of each network node of the component to be identified, a matching component that matches the component to be identified is determined from each identified component in the pre-built component library.

[0378] The above-mentioned memory 502 includes a memory 5021 and an external memory 5022; the memory 5021 here is also called an internal memory, which is used to temporarily store the calculation data in the processor 501, as well as the data exchanged with the external memory 5022 such as a hard disk. The processor 501 exchanges data with the external memory 5022 through the memory 5021. When the computer device is running, the processor 501 communicates with the memory 502 through the bus 503, so that the processor 501 executes the execution instructions mentioned in the above method embodiment.

[0379] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the component matching method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0380] The computer program product of the component matching method provided in the embodiments of the present disclosure includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the steps of the component matching method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0381] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK).

[0382] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0383] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0384] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0385] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0386] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0387] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the aforementioned embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A component matching method, characterized in that: include: Extracting original code features of each function and class in the component to be identified based on the code information of the component to be identified obtained from the code platform; the original code features include code attribute features, call relationship features and dependency relationship features; Taking each function and class as a network node, and determining the directed edges between each of the network nodes according to the calling relationship features in the original code features; Determining an edge weight of the directed edge according to a function call frequency, a call path depth, and a call parameter complexity between two network nodes corresponding to the directed edge; Determine the correlation between every two of the network nodes according to the original code features of each of the network nodes; Using a feature selection matrix composed of correlations between every two of the network nodes, the original code features of each of the network nodes are screened to obtain a screened code feature of each of the network nodes; Determining a characteristic attribute matrix of each network node according to the filtered code features of each network node; Constructing a directed graph network according to the directed edges between the network nodes, the edge weights of the directed edges, and the characteristic attribute matrix of each network node; Inputting the directed graph network into a pre-trained first neural network to obtain an embedded feature vector of each of the network nodes and a relationship feature vector between each of the network nodes and other network nodes; Using a pre-trained multi-layer autoencoder, feature encoding is performed on the embedded feature vector and the relationship feature vector of each network node in turn to obtain a target feature vector of each network node; According to the target feature vector of each of the network nodes of the component to be identified, a matching component matching the component to be identified is determined from each identified component in a pre-built component library.

2. The method according to claim 1, characterized in that Obtain code information of the component to be identified from the code platform, including: Acquire, from the code platform, code information of the component to be identified and feature information of the component to be identified in multiple feature dimensions according to the latest acquisition frequency; According to the latest weight coefficient corresponding to each of the feature dimensions, weighted processing is performed on the feature information under the multiple feature dimensions to obtain the first platform activity corresponding to the component to be identified; According to the activity of the first platform, the latest acquisition frequency is adjusted to obtain an adjusted acquisition frequency; the adjusted acquisition frequency is used for obtaining the component to be identified from the code platform next time.

3. The method according to claim 2, characterized in that After determining the activity of the first platform, the method further includes: Determine the activity prediction loss according to the first platform activity corresponding to the component to be identified, the historical platform activity of each historical component collected before the component to be identified, and the second platform activity of each historical component under the latest weight coefficient; Using the activity prediction loss, each of the latest weight coefficients is updated to obtain an updated weight coefficient; the updated weight coefficient is used to determine the first platform activity of the component to be identified that is collected next time.

4. The method according to claim 1, characterized in that The extracting original code features of each function and class in the component to be identified based on the code information of the component to be identified obtained from the code platform includes: Performing data cleaning and standardization processing on the code information of the component to be identified to obtain standard code information; The original code features of each function and class are extracted from the standard code information using a code analysis tool.

5. The method according to claim 1, characterized in that The first neural network includes multiple network layers; The step of inputting the directed graph network into a pre-trained first neural network to obtain an embedded feature vector of each of the network nodes and a relationship feature vector between each of the network nodes and other network nodes includes: For any of the network layers, obtain the first feature vector of each network node output by the previous network layer and the node relationship feature vector of each of the network nodes; wherein, when the network layer is the first layer, the first feature vector of each network node output by the previous network layer is the feature attribute matrix of each of the network nodes, and the node relationship feature vector of each of the network nodes is the edge weight of the directed edge of each of the network nodes; Determining a second feature vector of the network node according to first feature vectors of each neighboring node of the network node and a node relationship feature vector of the network node; Determine, according to the weight matrix of each attention head in the network layer, the second eigenvector of the network node, and the second eigenvector of each neighboring node of the network node, the aggregation weight of the network node and each neighboring node corresponding to each attention head; Determine a node relationship feature vector output by the network node at the network layer according to the aggregation weights corresponding to the network node and each adjacent node at each attention head; According to the aggregation weights corresponding to the network node and each adjacent node in each of the attention heads, the second feature vectors of each adjacent node of the network node are aggregated to obtain the first feature vector output by the network node in the network layer; wherein the first feature vector of each network node output by the last network layer is used as the embedded feature vector of each network node; and the node relationship feature vector of each network node output by the last network layer is used as the relationship feature vector of each network node.

6. The method according to claim 1, characterized in that Using a pre-trained multi-layer autoencoder, feature encoding is performed on the embedded feature vector and the relationship feature vector of each network node in turn to obtain a target feature vector of each network node, including: Inputting the embedded feature vector and the relationship feature vector of each of the network nodes into the first layer of the autoencoder in the multi-layer autoencoder to obtain the intermediate feature vector and the intermediate relationship feature vector of each of the network nodes; Inputting the intermediate feature vector and the intermediate relationship feature vector of each of the network nodes into the next layer of autoencoder to obtain a new intermediate feature vector and a new intermediate relationship feature vector of each of the network nodes; Return to the step of inputting the intermediate feature vector and the intermediate relationship feature vector of each of the network nodes to the next layer of autoencoder until the target feature vector of each of the network nodes output by the last layer of autoencoder is obtained.

7. The method according to claim 1, characterized in that The step of determining, according to the target feature vector of each of the network nodes of the component to be identified, a matching component that matches the component to be identified from each identified component in a pre-built component library comprises: Determine a first function list corresponding to the component to be identified according to each function and class of the component to be identified, and determine a second function list corresponding to each identified component according to each function and class in each identified component; Determining, according to the first function list and each of the second function lists, a component similarity between each of the identified components and the component to be identified; According to the component similarity, selecting candidate components similar to the component to be identified from the identified components; The matching component is determined according to the target feature vector of each of the network nodes in the component to be identified and the target feature vector of each of the network nodes in each of the candidate components.

8. The method according to claim 7, characterized in that The determining the matching component according to the target feature vector of each of the network nodes in the component to be identified and the target feature vector of each of the network nodes in each of the candidate components includes: Determine a first similarity between the component to be identified and each of the candidate components according to the original code feature of each of the network nodes in the component to be identified and the original code feature of each of the network nodes in each of the candidate components; Determine a second similarity between the component to be identified and each of the candidate components according to the target feature vector of each of the network nodes in the component to be identified, the target feature vector of each of the network nodes in each of the candidate components, and the edge weight of the directed edge between every two network nodes; Determining a third similarity between the component to be identified and each of the candidate components according to the identification time of the component to be identified and the identification time of each of the candidate components; According to the first similarity, the second similarity and the third similarity, a target similarity between the component to be identified and each of the candidate components is determined, and the matching component is determined according to the target similarity.

9. The method according to claim 1, characterized in that: The method further comprises: In the case where there is no matching component, the target feature vector and the original code feature of each network node of the component to be identified are added to the component library.

10. The method according to claim 1, characterized in that After determining a matching component that matches the component to be identified, the method further includes: The matching components in the component library are updated using the target feature vector and the original code feature of each network node of the component to be identified.

11. The method according to claim 1, characterized in that: The first neural network is trained according to the following steps: Obtaining a sample directed graph network corresponding to the first sample component; the sample directed graph network includes a plurality of sample network nodes; Inputting the sample directed graph network into a first neural network to be trained, obtaining a predicted second feature vector and a predicted node relationship feature vector of each sample network node output by a first network layer of the first neural network to be trained, and a predicted embedding feature vector and a predicted relationship feature vector of each sample network node output by a last network layer of the first neural network to be trained; Determining a first prediction loss of the first neural network to be trained according to the predicted second feature vector, the predicted node relationship feature vector, the predicted embedding feature vector and the predicted relationship feature vector; The first prediction loss is used to iteratively train the first neural network to obtain a trained first neural network.

12. The method according to claim 1, characterized in that The multi-layer autoencoder is trained according to the following steps: Obtaining a sample embedding feature vector and a sample relationship feature vector of each sample network node in the sample directed graph network corresponding to the second sample component; Input the sample embedding feature vector and the sample relationship feature vector of each sample network node into the multi-layer autoencoder to be trained, and obtain the predicted target feature vector and the predicted sample relationship feature vector of each network node output by the last layer of autoencoder; Using the decoder in the multi-layer autoencoder to be trained, the predicted target feature vector and the predicted sample relationship feature vector of each network node are decoded to obtain a decoded feature vector and a decoded relationship feature vector; Determining a second prediction loss of the multi-layer autoencoder to be trained according to the decoded feature vector, the predicted target feature vector, the predicted sample relationship feature vector, and the decoded relationship feature vector; The multi-layer autoencoder to be trained is iteratively trained using the second prediction loss to obtain a trained multi-layer autoencoder.

13. A component matching device, characterized in that: include: The first extraction module is used to extract the original code features of each function and class in the component to be identified based on the code information of the component to be identified obtained from the code platform; the original code features include code attribute features, call relationship features and dependency relationship features; A construction module is used to determine the directed edges between each of the network nodes based on the call relationship features in the original code features, using each function and class as a network node; determine the edge weight of the directed edge based on the function call frequency, call path depth and call parameter complexity between two network nodes corresponding to the directed edge; and determine the correlation between every two network nodes based on the original code features of each of the network nodes; Using a feature selection matrix composed of the correlation between every two network nodes, the original code features of each network node are screened to obtain the screened code features of each network node; according to the screened code features of each network node, a feature attribute matrix of the network node is determined; according to the directed edges between each network node, the edge weights of the directed edges, and the feature attribute matrix of each network node, a directed graph network is constructed; A second extraction module is used to input the directed graph network into a pre-trained first neural network to obtain an embedded feature vector of each of the network nodes and a relationship feature vector between each of the network nodes and other network nodes; An encoding module, used to use a pre-trained multi-layer autoencoder to perform feature encoding on the embedded feature vector and the relationship feature vector of each network node in turn to obtain a target feature vector of each network node; The determination module is used to determine a matching component that matches the component to be identified from each identified component in a pre-built component library according to a target feature vector of each network node of the component to be identified.

14. A computer device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the processor executes the steps of the component matching method as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program. When the computer program is executed by a computer device, the computer device executes the steps of the component matching method according to any one of claims 1 to 12.

Images