Product warehouse safety verification method, device and equipment and storage medium

Abstract

The invention relates to the technical field of products, and discloses a product library safety verification method, device and equipment and a storage medium, and the method comprises the steps: obtaining different types of to-be-processed products from a product library, and determining a product analysis strategy according to the types of the to-be-processed products; analyzing the to-be-processed product through the product analysis strategy to obtain component dependency information of the to-be-processed product; and performing risk matching on the component dependency information and a preset vulnerability risk library to obtain a security verification result. According to the method, the component dependency information of the to-be-processed product is analyzed, and the component dependency information is subjected to risk matching with the preset vulnerability risk library, so that compared with the prior art that whether vulnerabilities occur in dependent components in a product library or not is not detected, and whether vulnerabilities are introduced into the product library or not is not clear; according to the product library safety verification method, whether loopholes are introduced into the product library or not can be accurately known through the safety verification result, and potential safety hazards are effectively reduced.

Description

Technical Field

[0001] The present invention relates to the field of product technology, and in particular to a product library security verification method, device, equipment and storage medium. Background Art

[0002] Artifacts refer to binary files produced by compiling and packaging source code. Different development languages ​​correspond to binary files of different formats, which are usually used to run on servers or as compilation dependencies. In DevSecOps practice, in order to uniformly manage software artifacts of different formats, after the artifacts are built, R&D personnel will store the artifacts in the artifact library for unified management.

[0003] At present, due to technical requirements, products rely on a large number of external dependent components. However, the existing product management method does not detect whether there are vulnerabilities in the dependent components in the product library. It is not clear whether the product library introduces vulnerabilities and poses security risks.

[0004] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present invention is to provide a product library security verification method, device, equipment and storage medium, aiming to solve the technical problem that the prior art does not detect whether there are vulnerabilities in the dependent components in the product library, and is not clear whether vulnerabilities are introduced in the product library, resulting in security risks.

[0006] To achieve the above object, the present invention provides a product library security verification method, the method comprising the following steps:

[0007] Acquire different types of products to be processed from the product library, and determine a product parsing strategy according to the product types of the products to be processed;

[0008] Parsing the product to be processed by using the product parsing strategy to obtain component dependency information of the product to be processed;

[0009] The component dependency information is risk matched with a preset vulnerability risk library to obtain a security verification result.

[0010] Optionally, the step of determining the product parsing strategy according to the product type of the product to be processed includes:

[0011] Determining whether the product type of the product to be processed is a mirror image product type or a non-mirror image product type;

[0012] If it is the image product type, then obtain the product resolution strategy for resolving the image product.

[0013] Correspondingly, the step of parsing the artifact to be processed by the artifact parsing strategy to obtain component dependency information of the artifact to be processed includes:

[0014] Parsing the image configuration file of the product to be processed to determine the image parameter information of the product to be processed;

[0015] Each image layer of the product to be processed is parsed according to the image parameter information to obtain component dependency information of the product to be processed.

[0016] Optionally, after the step of determining whether the product type of the product to be processed is a mirror image product type or a non-mirror image product type, the step further includes:

[0017] If it is the non-image product type, determining whether the product type is a compressed file type or a binary file type;

[0018] If it is the compressed file type, then obtain the artifact parsing strategy for parsing the compressed file artifact.

[0019] Correspondingly, the step of parsing each image layer of the product to be processed by the product parsing strategy to obtain component dependency information of the product to be processed includes:

[0020] Determine a decompression tool according to the compressed file suffix of the product to be processed;

[0021] Decompressing the product to be processed by the decompression tool to obtain a decompressed file;

[0022] The configuration file of the decompressed file is parsed to obtain component dependency information of the product to be processed.

[0023] Optionally, if the product type is the non-image product type, after the step of determining whether the product type is a compressed file type or a binary file type, the method further includes:

[0024] If it is the binary file type, then obtain an artifact parsing strategy for parsing the binary file artifact.

[0025] Correspondingly, the step of parsing the artifact to be processed by using the artifact parsing strategy to obtain component dependency information of the artifact to be processed includes:

[0026] Obtain a binary parsing tool, and extract binary file information in the product to be processed through the binary parsing tool;

[0027] Parsing the binary file information;

[0028] If the analysis is successful, the analysis result is used as the component dependency information of the product to be processed.

[0029] Optionally, after the step of parsing the binary file information, the method further includes:

[0030] If the parsing fails, a neural network product parsing strategy for performing product parsing based on a neural network model is obtained;

[0031] The product to be processed is parsed using the neural network product parsing strategy to obtain component dependency information of the product to be processed.

[0032] Accordingly, the step of parsing the artifact to be processed by the neural network artifact parsing strategy to obtain component dependency information of the artifact to be processed includes:

[0033] Extracting coarse-grained features and fine-grained features of the product to be processed, and constructing a function content database according to the coarse-grained features and the fine-grained features;

[0034] Acquire a basic feature database, and perform similarity matching between the function content database and the basic feature database;

[0035] The component dependency information of the product to be processed is determined according to the matching result.

[0036] Optionally, the step of determining the component dependency information of the product to be processed according to the matching result includes:

[0037] When there is a target function content in the function content database whose similarity with the preset function content in the basic database is greater than the preset similarity, it is determined that the function content database is successfully matched with the basic feature database;

[0038] The successfully matched preset function content is used as the target function feature of the product to be processed, and the component dependency information of the product to be processed is determined according to the target function feature.

[0039] Optionally, the step of acquiring different types of products to be processed from the product library includes:

[0040] Pull different types of initial products from the Kafka message queue by consuming Kafka messages. The initial products are produced by the product library and transmitted through the real-time Kafka message queue;

[0041] The initial product is verified to obtain a product to be processed.

[0042] In addition, to achieve the above-mentioned purpose, the present invention also proposes a product library security verification device, the device comprising:

[0043] A parsing strategy module, used to obtain different types of products to be processed from the product library, and determine a product parsing strategy according to the product type of the product to be processed;

[0044] A product parsing module, used to parse the product to be processed by using the product parsing strategy to obtain component dependency information of the product to be processed;

[0045] The risk matching module is used to perform risk matching between the component dependency information and a preset vulnerability risk library to obtain a security verification result.

[0046] In addition, to achieve the above-mentioned purpose, the present invention also proposes a product library security verification device, which includes: a memory, a processor, and a product library security verification program stored on the memory and executable on the processor, and the product library security verification program is configured to implement the steps of the product library security verification method described above.

[0047] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a product library security verification program is stored, and when the product library security verification program is executed by a processor, the steps of the product library security verification method described above are implemented.

[0048] The present invention provides a product library security verification method, device, equipment and storage medium. The method obtains different types of products to be processed from the product library, and determines the product parsing strategy according to the product type of the product to be processed; then the product to be processed is parsed through the product parsing strategy to obtain the component dependency information of the product to be processed; finally, the component dependency information is risk matched with the preset vulnerability risk library to obtain the security verification result. The present invention parses the component dependency information of the product to be processed, and risk matches the component dependency information with the preset vulnerability risk library. Compared with the prior art, whether the dependent components in the product library are not detected, and whether the vulnerability is introduced into the product library is unclear, the product library security verification method of the present invention performs security verification on the component dependency information of the product through the preset vulnerability risk library. The security verification result obtained through the security verification can accurately know whether the vulnerability is introduced into the product library, which effectively reduces the safety hazard. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of the structure of a product library security verification device in a hardware operating environment involved in an embodiment of the present invention;

[0050] Figure 2 This is a flow chart of a first embodiment of a product library security verification method of the present invention;

[0051] Figure 3This is a flow chart of a second embodiment of the product library security verification method of the present invention;

[0052] Figure 4 This is a schematic diagram of parsing a compressed file in the second embodiment of the product library security verification method of the present invention;

[0053] Figure 5 This is a schematic diagram of parsing a binary file in the second embodiment of the product library security verification method of the present invention;

[0054] Figure 6 This is a schematic diagram of constructing a function content database in the second embodiment of the product library security verification method of the present invention;

[0055] Figure 7 This is a flow chart of a third embodiment of the product library security verification method of the present invention;

[0056] Figure 8 It is a schematic diagram of the overall process of security verification of different product libraries in the third embodiment of the product library security verification method of the present invention;

[0057] Fig. 9 A schematic diagram of verification of different product types in the third embodiment of the product library security verification method of the present invention;

[0058] Fig.10 This is a structural block diagram of the first embodiment of the product library security verification device of the present invention.

[0059] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0060] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0061] Reference Figure 1 , Figure 1 A schematic diagram of the structure of a product library security verification device in the hardware operating environment involved in an embodiment of the present invention.

[0062] like Figure 1As shown, the product library security verification device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0063] Those skilled in the art will understand that Figure 1 The structure shown in does not constitute a limitation on the product library security verification device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0064] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a product library security verification program.

[0065] exist Figure 1 In the product library security verification device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the product library security verification device of the present invention can be set in the product library security verification device, and the product library security verification device calls the product library security verification program stored in the memory 1005 through the processor 1001, and executes the product library security verification method provided by the embodiment of the present invention.

[0066] The embodiment of the present invention provides a product library security verification method, referring to Figure 2 , Figure 2 It is a flowchart of the first embodiment of the product library security verification method of the present invention.

[0067] In this embodiment, the product library security verification method includes the following steps:

[0068] Step S10: Acquire different types of products to be processed from the product library, and determine a product parsing strategy according to the product types of the products to be processed.

[0069] It should be noted that the execution subject of the method of this embodiment can be a computing service device with product library security verification, network communication and program running functions, such as a mobile phone, a tablet computer, a personal computer, etc., and can also be other electronic devices that achieve the same or similar functions. The following is an explanation of this embodiment and the following embodiments using the above-mentioned product library security verification device (referred to as the verification device).

[0070] It is understandable that the above-mentioned artifacts to be processed may be artifacts that have been stored in the artifact library but have not yet been security-verified, such as image artifacts, deployment file artifacts, and component dependency artifacts. In addition, the above-mentioned artifacts may be binary files produced by compiling and packaging source code, where different development languages ​​correspond to binary files of different formats, and these binary files are usually used to run on servers or as compilation dependencies.

[0071] It should be noted that the above-mentioned artifact library can be a artifact warehouse for storing the above-mentioned artifacts to be processed. By setting up a artifact library to store different types of artifacts to be processed; or setting up multiple different artifact libraries, different artifact libraries store different types of artifacts, and the artifact types in the same artifact library are the same, for example, an image artifact warehouse for storing image artifacts, a deployment file artifact warehouse for storing deployment file artifacts, and a component dependency artifact warehouse for storing component dependency artifacts.

[0072] It should be noted that the above-mentioned product parsing strategy may be a parsing strategy applicable to parsing a certain type of product to be processed, such as a mirror product parsing strategy for parsing mirror products (mirror type products), a compressed file product parsing strategy for parsing compressed file products (compressed file type products), and a binary product parsing strategy for directly parsing binary products (binary file type products).

[0073] In a specific implementation, the above-mentioned verification device can obtain a type of unprocessed products from the product library according to the target type. If there is only one product library, the unprocessed products of the target type are pulled from the product library so that the parsed products are products of the target type. If there are multiple product libraries, the unprocessed products are pulled from the product library storing the target type products so that the parsed products are products of the target type. Then, according to the target type, a product parsing strategy suitable for parsing the target type is determined.

[0074] It should be understood that the above-mentioned target type may be a product type that needs to be safety-verified. The target type may be input by a technician on a user panel of the verification device or may be pre-written into the verification device by a technician, and this embodiment does not limit this.

[0075] Step S20: parsing the artifact to be processed using the artifact parsing strategy to obtain component dependency information of the artifact to be processed.

[0076] It should be noted that the above-mentioned component dependency information may be the component information on which the above-mentioned product to be processed depends, and the component information includes the component name, component version and component type of the dependent component.

[0077] In a specific implementation, the verification device may parse the artifact to be processed of the target type through the determined artifact parsing strategy, and extract the component name, component version and component type of the component on which the artifact to be processed depends.

[0078] Step S30: Risk matching the component dependency information with a preset vulnerability risk library to obtain a security verification result.

[0079] It should be noted that the above-mentioned preset vulnerability risk library may be a vulnerability risk library storing component vulnerability information, where the component vulnerability information includes a number of component vulnerabilities and the component name, component version and component type of each component vulnerability.

[0080] In a specific implementation, the component name, component version and component type in each component dependency in the above component dependency information are matched with the component name, component version and component type of each component vulnerability in the above preset vulnerability risk library. If there is a component dependency that matches the component vulnerability, the component dependency can be determined to be a component vulnerability, that is, there is a vulnerability in the product to be processed. Correspondingly, if each component dependency fails to match the component vulnerability, it can be determined that there is a vulnerability in the product to be processed. Finally, the matching result of the risk matching is fed back as a security verification result. If there is a vulnerability in the product to be processed, the component dependency that matches the component vulnerability is marked in the security verification result, and the successfully matched component vulnerability information is added to the security verification result, so that the technician can accurately repair the vulnerability in the product to be processed.

[0081] Further, in this embodiment, the preset vulnerability risk library includes a preset component vulnerability library and a preset vulnerability blacklist, and the step S30 includes:

[0082] Step S301: Match each component dependency in the component dependency information with each preset component vulnerability in the preset component vulnerability library to obtain a matching result.

[0083] It should be noted that the above-mentioned preset component vulnerability library can be a risk library that includes various types of vulnerabilities. The vulnerabilities stored in the preset component vulnerability library can be real-time preset component vulnerabilities, such as NVD, REDHAT, various OS official vulnerabilities and several open source community vulnerabilities. The vulnerability types included in the preset component vulnerability library are relatively comprehensive.

[0084] Accordingly, in this embodiment, the component dependency information includes: the component name, component version and component type of each component, and the step S301 includes:

[0085] Step S3011: Match the component name, the component version and the component type with the component version, component type and component type of each preset component vulnerability in the preset component vulnerability library to obtain a matching result.

[0086] In a specific implementation, the verification device may match the component name, component version and component type of each component in the component dependency information with the component name, component version and component type of each preset component vulnerability recorded in the preset component vulnerability library. Since the vulnerability types in the preset component vulnerability library are relatively comprehensive, the accuracy of the matching result obtained is higher and it is easier to reflect whether there are vulnerabilities in the product to be processed.

[0087] Step S302: performing a secondary match between the component dependencies in the component dependency information that fail to match the preset component vulnerabilities once and the preset vulnerability blacklist to obtain a secondary matching result.

[0088] It should be noted that the above-mentioned preset vulnerability blacklist can be a risk library that includes vulnerabilities recorded locally, and can be updated in real time by technical personnel. The vulnerabilities included in the preset vulnerability blacklist can be vulnerabilities that have appeared in previous products, or can be vulnerabilities manually added by technical personnel.

[0089] Accordingly, in this embodiment, step S302 includes:

[0090] Step S3021: perform a secondary match between the component name, component version and component type in the component dependency information that fails to match the preset component vulnerability library once and the preset vulnerability blacklist to obtain a secondary matching result.

[0091] In a specific implementation, the verification device may match the component dependencies in the component dependency information that have not been successfully matched with the above-mentioned preset component vulnerabilities with the above-mentioned preset vulnerability blacklist again to avoid the situation where the component dependency is a vulnerability but has not been detected. The component name, component version and component type in the component dependency information that failed to match the above-mentioned preset component vulnerability library once are matched with the component name, component version and component type of the component vulnerability recorded in the above-mentioned preset vulnerability blacklist for a second time to obtain a second matching result, so as to improve the accuracy of vulnerability detection through two matches of different risk libraries.

[0092] Step S303: taking the first matching result and the second matching result as security verification results.

[0093] In a specific implementation, if the above verification fails, the above-mentioned primary matching result and secondary matching result can be used together as the safety verification result, so as to perform safety verification on the processed products through a wider range of risk libraries, thereby improving the accuracy of safety verification.

[0094] This embodiment obtains different types of products to be processed from the product library, and determines the product parsing strategy according to the product type of the product to be processed; then the product to be processed is parsed through the product parsing strategy to obtain the component dependency information of the product to be processed; finally, the component dependency information is risk matched with the preset vulnerability risk library to obtain the security verification result. This embodiment parses the component dependency information of the product to be processed, and risk matches the component dependency information with the preset vulnerability risk library. Compared with the prior art, which lacks the detection of whether the dependent components in the product library have vulnerabilities, and it is unclear whether vulnerabilities are introduced into the product library, the above-mentioned product library security verification method of this embodiment performs security verification on the component dependency information of the product through the preset vulnerability risk library. The security verification result obtained through the security verification can accurately know whether vulnerabilities are introduced into the product library, which effectively reduces security risks.

[0095] refer to Figure 3 , Figure 3 It is a flowchart of the second embodiment of the product library security verification method of the present invention.

[0096] Based on the above first embodiment, in this embodiment, the step of determining the product analysis strategy according to the product type of the product to be processed includes:

[0097] Step S101: determining whether the product type of the product to be processed is a mirror image product type or a non-mirror image product type.

[0098] It should be noted that the above-mentioned image product type can be the type of image product, which can be a packaged executable file or code library created by developers during the software development process. The image product can be used directly by other developers without having to compile or build it again. Common image products include Docker images, Maven dependency packages, NuGet packages, Ruby Gems, etc. These image products are usually stored in public or private image repositories and managed and updated through version control systems.

[0099] It is understandable that the above-mentioned non-mirror product type may be any type other than the mirror product type, and the product of the non-mirror product type may be a compressed file product or a binary file product, etc.

[0100] In the specific implementation, since the file types of the products to be processed of the mirror product type and the products to be processed of the non-mirror product type are different, the above-mentioned verification device needs to detect whether the product type of the above-mentioned product to be processed is a mirror product type or a non-mirror product type, so as to adopt the corresponding product parsing strategy to successfully parse out the components on which the above-mentioned product to be processed depends.

[0101] Step S102': If it is the image product type, then obtain a product resolution strategy for resolving the image product.

[0102] In a specific implementation, if the verification device detects that the product type of the product to be processed is the mirror product type, that is, the product to be processed is a mirror product, a product parsing strategy suitable for parsing the mirror product can be obtained to accurately extract the components on which the mirror product depends.

[0103] Accordingly, in this embodiment, step S20 includes:

[0104] Step S201 ′: parsing the image configuration file of the product to be processed, and determining the image parameter information of the product to be processed.

[0105] It should be noted that the above image configuration file can be a text file used to describe the image building process, usually named Dockerfile. It contains a series of instructions for instructing the Docker engine on how to operate when building an image. Through the image configuration file, developers can define the required software environment, code library, operating parameters and other information, as well as how to package these components into an executable image.

[0106] In a specific implementation, the verification device can parse different types of products to be processed, that is, different types of image products, such as alpine:latest, archive (tar.gz), etc., analyze each image layer of the image product, and obtain the overall image information of each layer. The overall image information can be the above-mentioned image parameter information, which may include basic parameters, image description, image structure and other information. For example, the manifest file is used to describe the image config and layer, which can complete the description of the image configuration as a content file; such as the config file, which is a complete description of the image, including all configurations such as the build environment, build history, environment variables and startup commands; layer (changeset) is a file system that describes the image hierarchy. Each layer saves the difference between the upper layers, including the representation of new, modified and deleted files.

[0107] Step S202 ′: parsing each image layer of the product to be processed according to the image parameter information to obtain component dependency information of the product to be processed.

[0108] In a specific implementation, the above-mentioned verification device can parse the image parameter information of each layer of the image, such as manifest, config, release files and layer, so as to extract the component dependency information of the image product, that is, the component name, component version and component type of the component on which the product to be processed depends.

[0109] Furthermore, in this embodiment, after step S101, the following steps are further included:

[0110] Step S102: If it is the non-image product type, determine whether the product type is a compressed file type or a binary file type.

[0111] In a specific implementation, if the above-mentioned product to be processed is not a mirror product, then the product to be processed may be a compressed file product or a binary file product. Therefore, when the above-mentioned verification device detects that the product type of the above-mentioned product to be processed is a non-mirror product type, that is, it is not the above-mentioned mirror product type, it can further determine whether the non-mirror product type is a compressed file type or a binary file type, so as to adopt the corresponding product parsing strategy for parsing according to the compressed file type or the binary file type.

[0112] Step S103": If it is the compressed file type, obtain the product parsing strategy for parsing the compressed file product.

[0113] In a specific implementation, when the verification device detects that the product type of the product to be processed is the compressed file type, that is, when the product to be processed is a compressed file product, it can obtain a product parsing strategy for parsing the compressed file product to accurately extract the components on which the compressed file product depends.

[0114] Accordingly, in this embodiment, step S20 includes:

[0115] Step S201": Determine a decompression tool according to the compressed file suffix of the product to be processed.

[0116] In the specific implementation, for ease of understanding, refer to Figure 4 This is for illustration only, but does not limit this solution. Figure 4 This is a schematic diagram of parsing a compressed file in the second embodiment of the product library security verification method of the present invention. Figure 4 In the , for compressed files with suffixes such as java, war, ear, par, and nuget, you can use the upzip decompression tool to decompress them, that is, decompress the product through upzip. Furthermore, for compressed files with suffixes such as tar.gz and tgz, you can use the untar decompression tool to decompress them.

[0117] Step S202": decompress the product to be processed by the decompression tool to obtain a decompressed file.

[0118] In a specific implementation, the verification device can obtain decompressed files after decompressing the corresponding compressed files through the decompression tool, and can extract component dependency information by parsing the configuration files of the decompressed files.

[0119] Step S203": Parse the configuration file of the decompressed file to obtain component dependency information of the product to be processed.

[0120] In a specific implementation, for the decompressed file obtained by decompressing with the unzip decompression tool, the configuration files of the decompressed file, such as pom.properties, MANIFEST.MF and other files, can be parsed to obtain the component dependency information of the decompressed file. For the decompressed file obtained by decompressing with the untar decompression tool, the parsing tools of the relevant configuration files, such as json, xml, yaml, jfather, textprot and toml, can be obtained, and the configuration files of the decompressed file, such as json or yaml, can be directly parsed by the parsing tools of the relevant configuration files to obtain the component dependency information of the decompressed file, wherein the component dependency information of the decompressed file can be the component dependency information of the above-mentioned product to be processed.

[0121] Furthermore, in this embodiment, after step S102, the following steps are further included:

[0122] Step S103: If it is the binary file type, then obtain an artifact parsing strategy for parsing the binary file artifact.

[0123] In a specific implementation, when the verification device detects that the product type of the product to be processed is neither the mirror product type nor the compressed file type, it can determine that the product to be processed is a binary file product in a specific language, that is, the type of the product to be processed is a binary file type, and then obtain the product parsing strategy for the binary file product to accurately extract the components on which the binary file product depends, wherein the binary file product in the specific language can be a product in the product library that is neither a mirror product nor a compressed file product.

[0124] Accordingly, in this embodiment, step S20 includes:

[0125] Step S201: Obtain a binary parsing tool, and extract binary file information in the product to be processed through the binary parsing tool.

[0126] In the specific implementation, for ease of understanding, refer to Figure 5 This is for illustration only, but does not limit this solution. Figure 5 This is a schematic diagram of parsing a binary file in the second embodiment of the product library security verification method of the present invention. Figure 5 In , for binary files with the suffix go, you can obtain the buildinfo parsing tool, use buildinfo to decompress the artifact, and extract the above artifact to be processed, that is, the binary file information in the binary file artifact. Correspondingly, for binary files with the suffix rust, you can obtain the rust-audit parsing tool, use rust-audit to decompress the artifact, and extract the binary file information in the above artifact to be processed.

[0127] Step S202: Parse the binary file information.

[0128] In a specific implementation, the binary file information of the above-mentioned product to be processed is parsed to extract the component dependency information on which the product to be processed depends.

[0129] Step S203: If the analysis is successful, the analysis result is used as the component dependency information of the product to be processed.

[0130] In a specific implementation, the decompression device may parse the binary file information. If the parsing is successful and it is determined that the component dependency information of the binary file has been extracted, the parsing result may be used as the component dependency information of the product to be processed.

[0131] Furthermore, in this embodiment, after step S202, the following steps are further included:

[0132] Step S203'": if the analysis fails, a neural network product analysis strategy for product analysis based on a neural network model is obtained.

[0133] It should be noted that the above-mentioned neural network artifact parsing strategy can be a strategy for detecting component dependencies in binary file artifacts through a neural network model.

[0134] In a specific implementation, when the above-mentioned verification device fails to parse the above-mentioned binary file information, that is, fails to successfully extract the component dependency information of the binary file product, it determines that the binary file product that failed to parse cannot be parsed by the above-mentioned parsing tool. Therefore, a neural network product parsing strategy based on a neural network model can be obtained to ensure the successful extraction of the component dependency information in the above-mentioned binary file product.

[0135] Step S204'': parse the product to be processed by the neural network product parsing strategy to obtain component dependency information of the product to be processed.

[0136] In a specific implementation, the above-mentioned verification failure can be solved by extracting the basic features of the binary file artifact through a neural network model, and then introducing the function content. The component dependency information in the binary file artifact is detected by combining the basic features with the function content to identify the binary file artifact, that is, the component dependency information of the component on which the policy artifact depends, that is, the component name, component version and component type of the component dependency.

[0137] Furthermore, in this embodiment, the step S204'' includes:

[0138] Step S2041'': extracting the coarse-grained features and the fine-grained features of the product to be processed, and constructing a function content database according to the coarse-grained features and the fine-grained features.

[0139] It should be noted that the above-mentioned coarse-grained features may be string literals and exported function names of binary file products; the above-mentioned fine-grained features may be the function contents introduced on the basis of the coarse-grained features, that is, the fine-grained features may be string files, exported function names and function contents of binary file products.

[0140] In the specific implementation, for ease of understanding, refer to Figure 6 This is for illustration only, but does not limit this solution. Figure 6 This is a schematic diagram of constructing a function content database in the second embodiment of the product library security verification method of the present invention. Figure 6In the process, the binary file is first input into the neural network model, which can be a model obtained by fine-tuning a large language model through deep learning. The neural network model extracts features after disassembling the binary file. The extracted features are a combination of coarse-grained features and fine-grained features, namely, string features, function name features and function content. Then, the function features are output, namely, the extracted string features, function name features and function content are output, so as to embed the coarse-grained features and fine-grained features into the low-dimensional representation to improve the accuracy of component dependency detection. Finally, the above-mentioned verification device can receive the function features output by the above-mentioned neural network model, namely, string features, function name features and function content, and build a function content database based on the string features, function name features and function content.

[0141] Step S2042'': Acquire a basic feature database, and perform similarity matching between the function content database and the basic feature database.

[0142] It should be noted that the basic feature database can be constructed based on the extracted function features by extracting features from the open source database using a neural network model. Each function feature in the basic feature database is a function feature of each binary file in the open source database.

[0143] In a specific implementation, the verification device may obtain the basic feature database, and then perform similarity matching between the function content in the function content database and the function features in the basic feature database to determine the function features of the function content in the function content database.

[0144] Furthermore, in this embodiment, before step S2042'', the following steps are further included:

[0145] Step S20421'': extracting initial coarse-grained features and initial fine-grained features from a preset open source database.

[0146] It should be noted that the preset open source database may be a pre-set database using an open source protocol. The initial coarse-grained features may be coarse-grained features of binary files in the preset open source database, and correspondingly, the initial fine-grained features may be fine-grained features of binary files in the preset open source database.

[0147] In a specific implementation, the verification device may extract the initial coarse-grained features and the initial fine-grained features in the preset open source database through the neural network model to determine the functional features of each binary file in the preset open source database.

[0148] Step S20421'': construct a basic feature database according to the initial coarse-grained features and the initial fine-grained features.

[0149] In a specific implementation, the above-mentioned verification device can combine the extracted initial coarse-grained features and the initial fine-grained features to extract the initial string features, initial function name features and initial function content to determine the function features of each binary file in a preset open source database, and then build a basic feature database based on the determined function features.

[0150] Step S2043'': Determine the component dependency information of the product to be processed according to the matching result.

[0151] In a specific implementation, the verification device can determine the function features of the function content in the function content database based on the similarity comparison results between the function content database and the basic feature database, and the component dependency information of the product to be policy-based can be determined through the function features.

[0152] Furthermore, in this embodiment, the step S2043'' includes:

[0153] Step S20431'': when there is a target function content in the function content database whose similarity with the preset function content in the basic database is greater than the preset similarity, it is determined that the function content database matches the basic feature database successfully.

[0154] It should be noted that the above-mentioned preset function content may be the function content stored in the above-mentioned basic database, and the function content may be the function feature.

[0155] It is understandable that the above-mentioned preset similarity may be a similarity for determining whether the similarity between the contents of two functions reaches a close degree.

[0156] In a specific implementation, when the matching result is that there is a target function content in the function content database whose similarity with the preset function content in the basic database is greater than the preset similarity, the verification device can determine that the function characteristics of the target function content are close to the function characteristics represented by the preset function content, and the function characteristics of the target function content can be determined to be the preset function content, that is, it is determined that the function content database matches the basic feature database successfully.

[0157] Step S20432'': taking the successfully matched preset function content as the target function feature of the product to be processed, and determining the component dependency information of the product to be processed according to the target function feature.

[0158] In a specific implementation, after a successful match, the verification device can use the successfully matched preset function content as the target function feature of the target function content. The function feature of the target function content can be the function feature of the product to be processed, thereby determining the function feature of the product to be processed. Based on the function feature of the product to be processed, the component dependency information of the product to be processed can be determined, that is, the component name, component version and component type of the dependent component.

[0159] This embodiment determines whether the product type of the product to be processed is a mirror product type or a non-mirror product type. If it is a mirror product type, a parsing strategy for parsing the mirror product is obtained; if it is a non-mirror product type and a compressed file type, a product parsing strategy for parsing the compressed file product is obtained; if it is a non-parsable product type and a binary file type, a product parsing strategy for parsing the binary file product is obtained. When the product parsing strategy for the binary file product fails to be parsed, a neural network product parsing strategy for product parsing based on a neural network model is obtained, thereby adopting corresponding product parsing strategies based on different product types, thereby improving the accuracy of component dependency information extraction, and further improving the accuracy of product library security.

[0160] refer to Figure 7 , Figure 7 It is a flowchart of the third embodiment of the product library security verification method of the present invention.

[0161] Based on the above embodiments, in this embodiment, the step of obtaining different types of products to be processed from the product library includes:

[0162] Step S11: Pull different types of initial products from the Kafka message queue by consuming Kafka messages. The initial products are produced by the product library and transmitted through the real-time Kafka message queue.

[0163] Correspondingly, in this embodiment, the artifact library includes: an image artifact warehouse, a deployment file artifact warehouse and a component dependency artifact warehouse; the image artifact warehouse produces initial artifacts of the image artifact type; the deployment file artifact warehouse or the component dependency artifact warehouse produces non-image artifact types to be processed, and the non-image artifact types include compressed file types and binary file types.

[0164] It should be noted that the above verification device can support remote pulling of image files in the image product warehouse, can support container images such as Docker and PodMan, and also supports image archive files, and automatically pulls the image layer cache to the local open source; it can also support pulling the deployment file products in the deployment file warehouse and the component dependency products in the component dependency product warehouse. Among them, the above verification device supports one type of product at a time for security verification.

[0165] It is understandable that the initial product may be a product stored in the product library that has not been verified. The initial product may be an initial image product in the image product warehouse that has not been verified, a deployment file product in the deployment file product warehouse that has not been verified, or a component dependency product in the component dependency product warehouse that has not been verified.

[0166] It should be noted that the above-mentioned deployment file artifact warehouse can be a warehouse for storing common files built through the pipeline and files pulled during deployment. The file type of the deployment file artifact in the above-mentioned deployment file artifact warehouse is a compressed file type or a binary file type.

[0167] It is understandable that the above-mentioned component dependency artifact warehouse can be a dependency package management warehouse of various development languages, and can pull open source dependency packages or third-party packages from various central warehouses through agents, or second-party packages provided by the enterprise public component development team or partner companies. The file type of the component dependency artifact in the above-mentioned component dependency artifact warehouse is also a compressed file type or a binary file type.

[0168] In the specific implementation, the server where the above-mentioned verification device is located adopts the consumer mode, the above-mentioned product library produces the initial product, and transmits the initial product through the real-time Kafka message queue. The above-mentioned verification device pulls the initial product from the Kafka message queue by consuming Kafka messages.

[0169] Step S12: Verify the initial product to obtain the product to be processed.

[0170] In a specific implementation, the verification device can verify the initial product to ensure the correctness of the file, and the product to be processed can be the verified initial product.

[0171] For ease of understanding, refer to Figure 8 This is for illustration only, but does not limit this solution. Figure 8 This is a schematic diagram of the overall process of security verification of different product libraries in the third embodiment of the product library security verification method of the present invention. Figure 8In the process, the artifact library includes the image artifact warehouse, the deployment file artifact warehouse and the component dependency artifact warehouse. The artifacts in one of the warehouses, the image artifact warehouse, the deployment file artifact warehouse and the component dependency artifact warehouse, are pulled at a time, and then the component dependency is parsed. According to the different artifact types, the operations of directly parsing the configuration file, directly parsing the binary file and parsing the binary file by neural network are performed respectively. After the parsing is completed, the obtained component dependency information is reviewed, that is, the component details in the component dependency information are scanned for vulnerability risks, the component details are matched with the vulnerability risk library, and then the risk review report, that is, the above-mentioned security verification result, is reviewed.

[0172] Furthermore, in this embodiment, step S12 includes:

[0173] Step S121: When the product type of the initial product is a mirror product type, perform sha256 or md5 verification on the initial product to obtain a product to be processed.

[0174] In a specific implementation, if the verification device pulls the image product from the image product warehouse, that is, the product type of the initial product obtained is a mirror product type, then the initial product of the mirror product type can be verified by sha256 or md5 to ensure the correctness of the mirror product.

[0175] Furthermore, in this embodiment, step S12 further includes:

[0176] Step S121: When the product type of the initial product is a non-mirror type, perform an md5 check on the initial product to obtain a product to be processed.

[0177] In a specific implementation, if the verification device pulls a deployment file artifact from a deployment file artifact repository, that is, the artifact type of the initial artifact obtained is a deployment file artifact type, then the initial artifact of the deployment file artifact type can be subjected to an md5 verification to ensure the correctness of the deployment file artifact. Correspondingly, if a component dependency artifact is pulled from a component dependency artifact repository, that is, the artifact type of the initial artifact obtained is a component dependency artifact type, then the initial artifact of the component dependency artifact type can be subjected to an md5 verification to ensure the correctness of the component dependency artifact.

[0178] For ease of understanding, refer to Fig. 9 This is for illustration only, but does not limit this solution. Fig. 9 This is a schematic diagram of verification of different product types in the third embodiment of the product library security verification method of the present invention. Fig. 9In the process, the product library produces a product to be scanned, which can be the initial product to be scanned, and adds the produced product to be scanned to the Kafka message queue. The verification device pulls the product file from the Kafka message queue by consuming the Kafka message. If the product pulled is an image product, the image product is verified by sha256 or md5. If the product pulled is a deployment file product, the deployment file product is verified by md5. If the product pulled is a component-dependent product, the component-dependent product is verified by md5.

[0179] This embodiment pulls different types of initial products from the Kafka message queue by consuming Kafka messages, and then performs a sha256 or md5 check on the initial product when the product type of the initial product is a mirror product type, and performs an md5 check on the initial product when the product type of the initial product is a non-mirror type, so as to obtain the product to be processed, ensure the correctness of the product file to be processed, and thus improve the accuracy of the product library security check.

[0180] In addition, an embodiment of the present invention further proposes a storage medium, on which a product library security verification program is stored. When the product library security verification program is executed by a processor, the steps of the product library security verification method described above are implemented.

[0181] Reference Fig.10 , Fig.10 This is a structural block diagram of the first embodiment of the product library security verification device of the present invention.

[0182] like Fig.10 As shown, the product library security verification device proposed in the embodiment of the present invention includes:

[0183] The parsing strategy module 501 is used to obtain different types of products to be processed from the product library and determine the product parsing strategy according to the product type of the product to be processed.

[0184] The artifact parsing module 502 is used to parse the artifact to be processed through the artifact parsing strategy to obtain component dependency information of the artifact to be processed.

[0185] The risk matching module 503 is used to perform risk matching between the component dependency information and a preset vulnerability risk library to obtain a security verification result.

[0186] This embodiment obtains different types of products to be processed from the product library, and determines the product parsing strategy according to the product type of the product to be processed; then the product to be processed is parsed through the product parsing strategy to obtain the component dependency information of the product to be processed; finally, the component dependency information is risk matched with the preset vulnerability risk library to obtain the security verification result. This embodiment parses the component dependency information of the product to be processed, and risk matches the component dependency information with the preset vulnerability risk library. Compared with the prior art, which lacks the detection of whether the dependent components in the product library have vulnerabilities, and it is unclear whether vulnerabilities are introduced into the product library, the above-mentioned product library security verification method of this embodiment performs security verification on the component dependency information of the product through the preset vulnerability risk library. The security verification result obtained through the security verification can accurately know whether vulnerabilities are introduced into the product library, which effectively reduces security risks.

[0187] As an implementation mode, the preset vulnerability risk library includes a preset component vulnerability library and a preset vulnerability blacklist, and the risk matching module 503 is further used to match each component dependency in the component dependency information with each preset component vulnerability in the preset component vulnerability library once to obtain a primary matching result; to match the component dependencies in the component dependency information that fail to match the preset component vulnerabilities once with the preset vulnerability blacklist twice to obtain a secondary matching result; and to use the primary matching result and the secondary matching result as security verification results.

[0188] As an implementation method, the component dependency information includes: the component name, component version and component type of each component, and the risk matching module 503 is also used to match the component name, the component version and the component type with the component version, component type and component type of each preset component vulnerability in the preset component vulnerability library to obtain a matching result.

[0189] Correspondingly, the risk matching module 503 is also used to perform a secondary match between the component name, component version and component type in the component dependency information that failed to match the preset component vulnerability library once and the preset vulnerability blacklist to obtain a secondary matching result.

[0190] Based on the above-mentioned first embodiment of the product library security verification device of the present invention, a second embodiment of the product library security verification device of the present invention is proposed.

[0191] In this embodiment, the parsing strategy module 501 is also used to determine whether the product type of the product to be processed is a mirror product type or a non-mirror product type; if it is the mirror product type, then obtain the product parsing strategy for parsing the mirror product.

[0192] As an implementation mode, the product parsing module 502 is also used to parse the image configuration file of the product to be processed to determine the image parameter information of the product to be processed; parse each image layer of the product to be processed according to the image parameter information to obtain the component dependency information of the product to be processed.

[0193] As an implementation mode, the parsing strategy module 501 is also used to determine whether the product type is a compressed file type or a binary file type if it is the non-image product type; if it is the compressed file type, obtain the product parsing strategy for parsing the compressed file product.

[0194] As an implementation mode, the product parsing module 502 is also used to determine a decompression tool according to the compressed file suffix of the product to be processed; decompress the product to be processed by the decompression tool to obtain the decompressed file; and parse the configuration file of the decompressed file to obtain component dependency information of the product to be processed.

[0195] As an implementation manner, the parsing strategy module 501 is further used to obtain a product parsing strategy for parsing a binary file product if the file type is the binary file.

[0196] As an implementation mode, the product parsing module 502 is also used to obtain a binary parsing tool, and extract binary file information in the product to be processed through the binary parsing tool; parse the binary file information; if the parsing is successful, use the parsing result as component dependency information of the product to be processed.

[0197] As an embodiment, the product parsing module 502 is also used to obtain a neural network product parsing strategy for product parsing based on a neural network model if the parsing fails; the product to be processed is parsed through the neural network product parsing strategy to obtain component dependency information of the product to be processed.

[0198] As an implementation mode, the product parsing module 502 is also used to extract the coarse-grained features and fine-grained features of the product to be processed, and to construct a function content database based on the coarse-grained features and the fine-grained features; to obtain a basic feature database, and to perform similarity matching between the function content database and the basic feature database; and to determine the component dependency information of the product to be processed based on the matching results.

[0199] As an embodiment, the product parsing module 502 is also used to determine that the function content database and the basic feature database are successfully matched when there is a target function content in the function content database whose similarity with the preset function content in the basic database is greater than the preset similarity; use the successfully matched preset function content as the target function feature of the product to be processed, and determine the component dependency information of the product to be processed based on the target function feature.

[0200] As an implementation manner, the product parsing module 502 is also used to extract initial coarse-grained features and initial fine-grained features from a preset open source database; and construct a basic feature database based on the initial coarse-grained features and the initial fine-grained features.

[0201] As an implementation mode, the parsing strategy module 501 is also used to pull different types of initial products from the Kafka message queue by consuming Kafka messages. The initial products are produced by the product library and transmitted through the real-time Kafka message queue; the initial products are verified to obtain products to be processed.

[0202] As an implementation mode, the artifact library includes: an image artifact repository, a deployment file artifact repository, and a component dependency artifact repository; the image artifact repository produces initial artifacts of the image artifact type; the deployment file artifact repository or the component dependency artifact repository produces unprocessed artifacts of the non-image artifact type, and the non-image artifact type includes a compressed file type and a binary file type.

[0203] As an implementation manner, the parsing strategy module 501 is further used to perform a sha256 or md5 check on the initial product to obtain a product to be processed when the product type of the initial product is a mirror product type.

[0204] As an implementation manner, the parsing strategy module 501 is further used to perform an md5 check on the initial product to obtain a product to be processed when the product type of the initial product is a non-mirror type.

[0205] Other embodiments or specific implementations of the product library security verification device of the present invention can refer to the above-mentioned method embodiments and will not be described in detail here.

[0206] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0207] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0208] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0209] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A product library security verification method, characterized in that: The method comprises the following steps: Acquire different types of products to be processed from the product library, and determine a product parsing strategy according to the product types of the products to be processed; Parsing the product to be processed by using the product parsing strategy to obtain component dependency information of the product to be processed; The component dependency information is risk matched with a preset vulnerability risk library to obtain a security verification result.

2. The product library security verification method according to claim 1, characterized in that: The step of determining the product parsing strategy according to the product type of the product to be processed includes: Determining whether the product type of the product to be processed is a mirror image product type or a non-mirror image product type; If it is the image product type, then obtain the product resolution strategy for resolving the image product. Correspondingly, the step of parsing the artifact to be processed by using the artifact parsing strategy to obtain component dependency information of the artifact to be processed includes: Parsing the image configuration file of the product to be processed to determine the image parameter information of the product to be processed; Each image layer of the product to be processed is parsed according to the image parameter information to obtain component dependency information of the product to be processed.

3. The product library security verification method according to claim 2, characterized in that: After the step of determining whether the product type of the product to be processed is a mirror image product type or a non-mirror image product type, the method further includes: If it is the non-image product type, determining whether the product type is a compressed file type or a binary file type; If it is the compressed file type, then obtain the artifact parsing strategy for parsing the compressed file artifact. Correspondingly, the step of parsing each image layer of the product to be processed by the product parsing strategy to obtain component dependency information of the product to be processed includes: Determine a decompression tool according to the compressed file suffix of the product to be processed; Decompressing the product to be processed by the decompression tool to obtain a decompressed file; The configuration file of the decompressed file is parsed to obtain component dependency information of the product to be processed.

4. The product library security verification method according to claim 3, characterized in that: If the product type is the non-image product type, after the step of determining whether the product type is a compressed file type or a binary file type, the method further includes: If it is the binary file type, then obtain an artifact parsing strategy for parsing the binary file artifact. Correspondingly, the step of parsing the artifact to be processed by using the artifact parsing strategy to obtain component dependency information of the artifact to be processed includes: Obtain a binary parsing tool, and extract binary file information in the product to be processed through the binary parsing tool; Parsing the binary file information; If the analysis is successful, the analysis result is used as the component dependency information of the product to be processed.

5. The product library security verification method according to claim 4, characterized in that: After the step of parsing the binary file information, the method further includes: If the parsing fails, a neural network product parsing strategy for performing product parsing based on a neural network model is obtained; The product to be processed is parsed using the neural network product parsing strategy to obtain component dependency information of the product to be processed. Accordingly, the step of parsing the artifact to be processed by the neural network artifact parsing strategy to obtain component dependency information of the artifact to be processed includes: Extracting coarse-grained features and fine-grained features of the product to be processed, and constructing a function content database according to the coarse-grained features and the fine-grained features; Acquire a basic feature database, and perform similarity matching between the function content database and the basic feature database; The component dependency information of the product to be processed is determined according to the matching result.

6. The product library security verification method according to claim 5, characterized in that: The step of determining the component dependency information of the product to be processed according to the matching result includes: When there is a target function content in the function content database whose similarity with the preset function content in the basic database is greater than the preset similarity, it is determined that the function content database is successfully matched with the basic feature database; The successfully matched preset function content is used as the target function feature of the product to be processed, and the component dependency information of the product to be processed is determined according to the target function feature.

7. The product library security verification method according to any one of claims 1 to 6, characterized in that: The step of obtaining different types of products to be processed from the product library includes: Pull different types of initial products from the Kafka message queue by consuming Kafka messages. The initial products are produced by the product library and transmitted through the real-time Kafka message queue; The initial product is verified to obtain a product to be processed.

8. A product warehouse safety verification device, characterized in that: The device comprises: A parsing strategy module, used to obtain different types of products to be processed from the product library, and determine a product parsing strategy according to the product type of the product to be processed; A product parsing module, used to parse the product to be processed by using the product parsing strategy to obtain component dependency information of the product to be processed; The risk matching module is used to perform risk matching between the component dependency information and a preset vulnerability risk library to obtain a security verification result.

9. A product warehouse safety verification device, characterized in that: The device includes: a memory, a processor, and a product library security verification program stored in the memory and executable on the processor, wherein the product library security verification program is configured to implement the steps of the product library security verification method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a product library security verification program, and when the product library security verification program is executed by the processor, the steps of the product library security verification method as described in any one of claims 1 to 7 are implemented.

Images