A method, system, medium and program product for detecting sensitive information of installation package
Through the multi-process parallel decompilation and decompilation optimization mechanism, combined with cloud and local detection, the problem of low installation package detection efficiency in the existing technology is solved, and efficient multiple installation package detection is achieved, supporting the growth of large-scale business volume.
Patent Information
- Application Number
- CN202411238699.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-09-05
AI Technical Summary
The prior art has low detection efficiency due to the functional limitations of the decompilation tool during detection of multiple installation packages, and the detection process is long and cannot effectively support large-scale business volume.
Adopt multi-process parallel decompilation mechanism to copy and allocate the initial installation package to multiple target folders, and establish an independent decompilation process for each folder. Combined with the decompilation optimization mechanism, optimize the decompilation process using the similarity of historical installation packages. , dynamically select cloud or local detection to improve efficiency.
It significantly improves the speed and efficiency of installation package inspection, can support dozens of times the growth of business scale, shortens the average inspection time by nearly 2/3, and improves the performance and resource utilization of the detection system.
Smart Images

Figure CN118760611B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data detection, and in particular to a method, system, medium and program product for detecting sensitive information of an installation package. Background Art
[0002] With the rapid development of mobile Internet, the number of smartphone applications continues to increase. When developing apps (mobile applications) or before releasing them, many companies need to conduct strict security checks on installation packages to protect the developer's user privacy and the company's trade secrets, and prevent the leakage of sensitive information such as user privacy data. For example, before a financial company releases its mobile payment app, it must ensure that all files containing user accounts and financial information are not leaked. Therefore, sensitive information detection of installation packages has become a key step before App release.
[0003] In the related art, sensitive information detection usually includes keyword matching and regular expression search. The related art usually first uses a decompilation tool to decompile, and then uses the Apkleaks tool to directly scan all source code files in the installation package file to find specific sensitive data.
[0004] However, when multiple installation packages need to be detected in the related technology, due to the limited decompilation function of the decompilation tool, there will be a long queue waiting for the installation package detection, resulting in a long overall detection process and low detection efficiency. Summary of the invention
[0005] The present application provides a method, system, medium and program product for detecting sensitive information of an installation package, which are used to improve the efficiency of detecting sensitive information when multiple installation packages need to be detected.
[0006] In the first aspect, the present application provides a method for detecting sensitive information of an installation package, which is applied to a detection system, and the method includes: in response to a detection start instruction input by a user, obtaining a detection path and a type to be detected set by the user; identifying multiple initial installation packages in an initial folder corresponding to the detection path; copying and distributing the initial installation packages to multiple target folders as installation packages to be detected according to a preset allocation rule; establishing a preset number of decompilation processes, and decompiling the installation packages to be detected in the multiple target folders respectively to obtain source code packages to be detected; the decompilation processes correspond to the target folders one by one; screening multiple source code files in the source code package to be detected to obtain target files to be detected belonging to the type to be detected; scanning the target files to be detected respectively, and generating a sensitive scanning report corresponding to the initial installation package based on the scanning results.
[0007] In the above embodiment, the detection system distributes multiple installation packages and uses multiple decompilation processes to perform parallel decompilation processing. The initial installation package is copied and distributed to multiple target folders according to the preset distribution rules, and multiple decompilation processes corresponding to the target folders are established. This can make full use of the detection system resources and improve the parallelism of decompilation; at the same time, by screening out the target files to be detected that belong to the detection type, meaningless file detection is avoided. The efficiency of decompilation and information detection of multiple installation package files is effectively improved, the detection time is shortened, and the detection efficiency of sensitive information of the installation package is improved.
[0008] In combination with some embodiments of the first aspect, in some embodiments, multiple target folders include multiple decompilation and optimization folders; multiple decompilation and optimization folders respectively cache historical installation packages that have completed detection in different project fields; the step of copying and distributing the initial installation package to multiple target folders as the installation package to be inspected according to the preset allocation rule specifically includes: obtaining metadata information of the initial installation package; based on the metadata information, extracting the initial installation package whose similarity with the historical installation package is higher than the preset similarity threshold as the installation package to be optimized; determining that the installation package to be optimized corresponds to the same code segment as the historical installation package, and removing the non-compiled code segment in the installation package to be optimized to obtain the code segment to be compiled; copying the code segment to be compiled into the decompilation and optimization folder as the installation package to be inspected; after the steps of scanning the target files to be inspected respectively and generating a sensitive scanning report corresponding to the initial installation package based on the scanning results, the method also includes: determining the historical scanning data corresponding to the non-compiled code segment in the historical installation package according to the detection record of the historical installation package; and integrating the historical scanning data into the sensitive scanning report.
[0009] In the above embodiment, the detection system caches the historical installation packages that have completed the detection by setting a decompilation optimization folder, and optimizes the decompilation process by using the similarity between the historical installation packages and the current installation package to be detected. The detection system will compare the similarity between the initial installation package and the historical installation package. For installation packages with high similarity, its historical scanning results can be directly obtained to avoid repeatedly compiling the same code segment, thereby effectively reducing the workload of decompilation. By integrating historical scanning data into the final sensitive scanning report, not only the decompilation efficiency is improved, but also the accuracy of the detection results is ensured.
[0010] In combination with some embodiments of the first aspect, in some embodiments, the target folder includes a cloud folder set on a cloud server and a local folder set on a local computer; the step of copying and distributing the initial installation package to multiple target folders as the installation package to be inspected according to a preset distribution rule specifically includes: obtaining the file size of the initial installation package; calculating the local detection time of the initial installation package on the local computer and the cloud detection time of the initial installation package on the cloud server according to the file size; the local detection time is not less than the cloud detection time; calculating the absolute value of the difference between the cloud detection time and the local detection time as the cloud advantage time; obtaining the current network speed range, and determining the cloud upload time of the initial installation package according to the file size and the network speed range; when the cloud upload time is less than the cloud advantage time, copying and distributing the initial installation package to the cloud folder as the installation package to be inspected; when the cloud upload time is not less than the cloud advantage time, copying and distributing the initial installation package to the local folder as the installation package to be inspected.
[0011] In the above embodiment, the detection system will dynamically determine the detection settings based on factors such as the size of the initial installation package, the estimated detection time locally and on the cloud, and the current network conditions. When the advantageous detection time on the cloud is greater than the upload time, cloud detection will be given priority. Otherwise, local detection will be selected, which can save system resources while improving detection efficiency.
[0012] In combination with some embodiments of the first aspect, in some embodiments, the step of obtaining the file size of the initial installation package specifically includes: determining whether the initial installation package is set with cloud upload permissions; if so, obtaining the file size of the initial installation package; if not, copying and allocating the initial installation package to a local folder as the installation package to be checked.
[0013] In the above embodiment, the detection system introduces a judgment mechanism for cloud upload permissions, which can effectively protect the privacy and security of users. Only installation packages authorized by users can be uploaded to the cloud for detection, and installation packages without cloud permissions will be directly allocated to the local folder, eliminating unauthorized data upload and reducing the risk of privacy data leakage.
[0014] In combination with some embodiments of the first aspect, in some embodiments, a preset number of decompilation processes are established, and the installation packages to be checked in multiple target folders are decompiled respectively to obtain the steps of the source code packages to be checked, specifically including: establishing a preset number of decompilation processes; based on the decompilation processes, decompiling the installation packages to be checked in the target folders according to a preset decompilation order to obtain the source code packages to be checked; in the target folder, replacing the corresponding installation packages to be checked with the source code packages to be checked; during the decompilation process, if a priority adjustment instruction input by the user is received, adjusting the preset decompilation order based on the priority adjustment instruction to obtain a decompilation guidance order; according to the decompilation guidance order, after the current decompilation thread is completed, continuing to decompile the remaining installation packages to be checked in the target folder; the decompilation thread is a sub-process of the decompilation process, which is used to decompile a certain installation package to be checked.
[0015] In the above embodiment, the detection system establishes multiple decompilation processes, and each process contains multiple decompilation threads. During the specific decompilation, the detection system processes the installation packages in sequence according to the preset or dynamically adjusted decompilation guidance order. After completing the decompilation of a package, the original installation package will be replaced with the source code package, which saves storage space and can also use the change of file type to characterize the processing progress. The detection system provides users with the function of adjusting the processing priority in real time during the decompilation process, and flexibly guides the execution order of decompilation to cope with the dynamic changes in the actual detection process.
[0016] In combination with some embodiments of the first aspect, in some embodiments, after the step of establishing a preset number of decompilation processes, the method also includes: during the decompilation process, if a new installation package is detected in the initial folder, the estimated remaining time of multiple decompilation processes is queried; the target folder corresponding to the decompilation process with the shortest estimated remaining time is used as the folder to be updated, and the new installation package is allocated to the folder to be updated.
[0017] In the above embodiment, the detection system adds a real-time monitoring function for the decompilation process. If a new installation package to be detected is found during operation, it will be automatically included in the detection scope without the user having to reselect for detection. In order to efficiently process the newly added package, the detection system will evaluate the remaining task volume of all current decompilation processes, and then assign the newly added package to the process with the least task, thereby achieving a dynamic balance of tasks and avoiding the accumulation of tasks for a specific process.
[0018] In combination with some embodiments of the first aspect, in some embodiments, the steps of scanning the target files to be inspected separately and generating a sensitive scanning report corresponding to the initial installation package based on the scanning results specifically include: determining the file type of the target file to be inspected; if the target file to be inspected is a text file, then based on the memory mapping technology, mapping the text file to the memory for scanning to obtain the file scanning result; if the target file to be inspected is a binary file, extracting the string sequence of the binary file as the part to be scanned to obtain the file scanning result; generating a sensitive scanning report corresponding to the initial installation package based on the scanning result, and displaying the sensitive scanning report.
[0019] In the above embodiment, the detection system uses different scanning strategies to process different types of target files, which can effectively improve the scanning efficiency and accuracy. For text files, the system uses memory mapping technology to load them directly into memory for scanning, avoiding frequent disk I / O operations and speeding up the scanning speed; for binary files, the system will extract the string sequence therein for targeted scanning, reducing invalid byte comparisons and improving the scanning accuracy.
[0020] In a second aspect, an embodiment of the present application provides a detection system, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the detection system to perform the method described in the first aspect and any possible implementation method of the first aspect.
[0021] In a third aspect, an embodiment of the present application provides a computer program product comprising instructions, and when the above-mentioned computer program product runs on a detection system, the above-mentioned detection system executes the method described in the first aspect and any possible implementation method of the first aspect.
[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a detection system, enable the detection system to execute the method described in the first aspect and any possible implementation of the first aspect.
[0023] It is understandable that the detection system provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the method provided in the embodiment of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here.
[0024] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:
[0025] 1. Due to the use of a multi-process parallel decompilation mechanism, the initial installation package is copied to different target folders according to the preset distribution rules, and an independent decompilation process is established for each folder for processing, which fully utilizes system resources and effectively solves the problem of low efficiency of existing single-process decompilation, thereby achieving an improvement in the installation package detection speed.
[0026] 2. Due to the adoption of the decompilation optimization mechanism, the historical installation packages that have completed the detection are cached by setting the decompilation optimization folder, and the similarity of the installation packages is used to optimize the decompilation process. For installation packages with high similarity, the historical scanning results can be used directly, which effectively avoids repeated compilation and thus realizes the improvement of detection efficiency.
[0027] 3. Due to the adoption of a hybrid detection solution that combines cloud and local, the time advantage and data upload cost of cloud detection are evaluated in real time, and the optimal detection settings are dynamically selected. While giving full play to the powerful computing power of the cloud, the flexibility of local detection is taken into account, effectively balancing detection efficiency and system overhead, thereby achieving improved detection performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 It is a flowchart of a method for detecting sensitive information of an installation package in an embodiment of the present application;
[0029] Figure 2 It is another flowchart of the method for detecting sensitive information of an installation package in an embodiment of the present application;
[0030] Figure 3 It is another flowchart of the method for detecting sensitive information of an installation package in an embodiment of the present application;
[0031] Figure 4 It is a schematic diagram of the structure of a physical device of the detection system in the embodiment of the present application. DETAILED DESCRIPTION
[0032] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification of the present application, the singular expressions "one", "a kind of", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more of the listed items.
[0033] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.
[0034] For ease of understanding, the application scenarios of the embodiments of the present application are introduced below.
[0035] A mobile application development company A plans to integrate an installation package detection function in the new version of the application management platform to scan the application installation packages uploaded by developers for sensitive information to ensure the compliance and security of application distribution. However, as the company grows in size and the number of developers increases, the number of installation packages that need to be processed every day increases. If the traditional detection method is used, not only will the scanning take a long time, but it is also easy to cause delays in the feedback of the detection results, affecting the developer's experience.
[0036] In the related art, the installation package detection function of mobile applications can be implemented by adopting a simple serial processing architecture. Although this architecture is simple and easy to implement, it not only has a slow overall detection speed, but also has difficulty supporting large-scale business volumes. The following introduces the scenario of using the installation package sensitive information detection method in the related art.
[0037] Developers at Company A tried to use an open source installation package detection tool to build a prototype system for the detection platform. The basic processing logic of this prototype system is: copy the installation package file to be detected to a certain agreed input folder, and then execute the serial detection process file by file, including calling the decompilation tool to unpack, calling the scanning tool to search the full text for sensitive information, and other steps. After the processing is completed, the detection report is written to another agreed output folder for developers to review. Although this simple architectural design realizes the installation package detection function, it can only process one installation package detection task at a time, and the overall detection speed is limited by the execution time of a single task.
[0038] By using the method for detecting sensitive information of the installation package in the embodiment of the present application, the detection efficiency is significantly improved by automatically allocating the installation package task to multiple detection engine processes for concurrent execution. The following describes the scenario in which the method for detecting sensitive information of the installation package in the present application is used.
[0039] The developers of Company A decided to adopt a method for detecting sensitive information in installation packages proposed in this application to comprehensively improve and optimize the detection platform. The improved system introduced a multi-process parallel detection architecture, and the installation package tasks were automatically distributed to multiple different detection engine processes, allowing multiple tasks to be executed simultaneously. After more than half a year of trial operation, the improved detection platform successfully supported a business scale growth of dozens of times, and the average detection time was shortened by nearly 2 / 3.
[0040] It can be seen that the method for detecting sensitive information of installation packages in the embodiment of the present application optimizes the detection efficiency of a large number of installation packages.
[0041] For ease of understanding, the following describes the process of the method provided by this implementation in combination with the above scenario. Figure 1 , which is a flow chart of a method for detecting sensitive information of an installation package in an embodiment of the present application.
[0042] S101 . In response to a detection start instruction input by a user, obtain a detection path and a type to be detected set by the user.
[0043] The detection start instruction indicates the operation instruction of the user to start the sensitive information detection of the installation package. The detection path refers to the folder path where the installation package to be detected is located. The type to be detected is used to indicate the file type where the sensitive information to be detected is located, such as text files, log files, etc.
[0044] Specifically, when the user decides to perform sensitive information detection on a batch of installation packages, the initial parameters such as the detection path and detection type will be set through a special detection interface or configuration file. After receiving the user's detection start instruction, the detection system will read the detection path and detection type set by the user to provide the necessary input conditions for the subsequent detection process.
[0045] In some embodiments, the detection start instruction input by the user can be responded to and the detection settings can be obtained in the following ways: Optionally, the detection system provides a graphical detection configuration interface, and the user can select the folder path to be detected through the file browsing control, and check the sensitive information type to be detected through check boxes or drop-down menus, and click the "Start Detection" button to trigger the detection process; Optionally, the detection system requires the user to set detection parameters in a configuration file in a specified format, and start the detection program through a command line, parsing the configuration file to obtain the detection path and type information.
[0046] S102: Identify multiple initial installation packages in an initial folder corresponding to the detection path.
[0047] The initial folder refers to the folder corresponding to the detection path, and the initial installation package refers to the installation package file to be detected in the initial folder.
[0048] Specifically, when the detection system obtains the detection path set by the user, it will scan the folder corresponding to the path and identify the installation package files therein. Since there may be multiple different installation packages in a folder in actual scenarios, the detection system needs to identify and record them one by one to perform targeted detection operations.
[0049] In some embodiments, multiple initial installation packages in an initial folder can be identified in the following ways: Optionally, the detection system establishes an installation package file type feature library, which contains feature information such as file headers and extensions of common installation package formats, and determines whether it is an installation package by comparing the features of each file in the folder; Optionally, the detection system requires that the installation package to be detected must be stored in a specified name format, such as "APK name + version number + suffix name", and identifies the installation package by matching file name rules.
[0050] S103: Copy and distribute the initial installation package to multiple target folders according to a preset distribution rule as the installation packages to be checked.
[0051] The preset allocation rule refers to the rule pre-set by the detection system to determine the installation package allocation scheme, which can be formulated based on different properties of the installation package (such as size, type, priority, etc.). The target folder is used to store the allocated installation packages to be inspected, and each target folder corresponds to a decompilation process. The installation package to be inspected refers to the installation package waiting for decompilation and subsequent scanning.
[0052] Specifically, after the detection system identifies all the installation packages in the initial folder, it will copy them to different target folders according to the preset allocation rules. The allocation rules are based on factors such as load balancing and priority protection to achieve a balance between detection efficiency and order. For example, the installation packages can be evenly distributed to various folders according to their size to avoid the situation where individual processes are overloaded; or a higher priority can be set for important or urgent installation packages to give priority to resource allocation.
[0053] In some embodiments, the initial installation package can be copied and distributed to multiple target folders in the following ways: Optionally, the detection system copies the high-risk installation package to the target folder for high-priority processing and the low-risk installation package to the target folder for regular processing based on the risk score of the installation package; Optionally, the detection system performs cluster analysis on the file attributes of the installation package, sets dedicated target folders for different types of installation packages (such as system software packages, application software packages, game software packages, etc.), and copies and distributes them according to the clustering results. It is understandable that the design of the allocation rules can also consider other feature dimensions of the installation package, such as file format, development language, internal component composition, etc., to achieve more refined task diversion, which is not limited here.
[0054] S104: Establish a preset number of decompilation processes, and decompile the installation packages to be checked in multiple target folders respectively to obtain source code packages to be checked.
[0055] The decompilation process is used to represent an independent processing unit that restores the mobile application installation package to source code. The preset number of processes refers to the number of concurrent decompilation processes determined based on factors such as system resources and the number of tasks. The source code package to be checked refers to the collection of application source code files obtained after decompilation, usually including Java, Kotlin, XML and other file types.
[0056] Specifically, this step is performed after the installation package is copied and distributed. The detection system will create an equal number of decompilation processes based on the pre-configured process number parameters. Each process is responsible for taking out the installation package to be tested from the specified target folder, calling decompilation tools such as apktool, dex2jar, etc. to unpack and restore the code, and finally output the decompiled source code file package, that is, the source code package to be tested. The number of decompilation processes is set based on factors such as the number of system CPU cores, memory size, and the amount of tasks to be tested, and the overall efficiency of decompilation is improved through concurrent execution.
[0057] In some embodiments, the number of decompilation processes can be dynamically adjusted in the following ways: Optionally, the detection system monitors the real-time usage of the current CPU and memory. When the idle rate is higher than a preset threshold, the number of decompilation processes is appropriately increased; when it is lower than a preset threshold, the number of processes is appropriately reduced; Optionally, the detection system counts the average decompilation time of each installation package. When the time is higher than the expected value, the number of decompilation processes is appropriately increased; when the time is lower than the expected value, the number of processes is appropriately reduced. It is understandable that the adjustment strategy can also be combined with other factors such as task priority and decompilation error rate, which are not limited here.
[0058] S105 , screening multiple source code files in the source code package to be inspected to obtain target files to be inspected that belong to the type to be inspected.
[0059] Among them, screening refers to the process of identifying and extracting target files from a large number of source code files according to preset rules. The type to be detected is used to indicate the file type that needs to be scanned for security, which is usually set by the user or has a default setting. The target file to be checked refers to the subset of source code files obtained after screening that need to be scanned for sensitive content first.
[0060] Specifically, since a complete application source code package usually contains a large number of code and resource files, and only some core files involve sensitive data processing logic, the monitoring system will first classify and filter the source code files. The detection system will traverse all files in the source code package, read their file type, path and other attribute information, match them with the preset file type definition to be detected, identify the corresponding target file set, and eliminate the remaining non-target files. This screening process can avoid wasting time on irrelevant files in the subsequent scanning stage, thereby improving detection efficiency.
[0061] In some embodiments, the definition of the type to be detected can be generated and updated in the following ways: Optionally, security experts manually define a set of default source code types to be detected based on common components and interface specifications for mobile development, such as database operation classes, network communication classes, encryption and decryption algorithm classes, etc. Optionally, the detection system automatically counts the frequency of occurrence of sensitive information in different types of source code during the scanning process, and dynamically includes types whose frequency exceeds a preset threshold into the scope of detection; Optionally, the detection system receives external security intelligence input and dynamically updates the definition of the type to be detected based on newly discovered high-risk vulnerabilities and sensitive interface information. It is understandable that the screening granularity can also be expanded from file types to more detailed code elements such as specific class names and method names to support more accurate filtering, which is not limited here.
[0062] S106: Scan the target files to be inspected respectively, and generate a sensitivity scanning report corresponding to the initial installation package based on the scanning results.
[0063] Sensitive information refers to user privacy data that may be maliciously collected and abused in mobile applications, such as ID number, mobile phone number, location information, etc. The scan result represents the collection of all sensitive information detected in the source code file, usually including metadata such as the content summary of the information and the code location. The sensitive scan report is used to represent the complete scan results of sensitive information detected in an installation package, which can be used as a basis for evaluating the privacy and security risks of the application.
[0064] Specifically, this step is performed after the target source code file screening is completed. The detection system will traverse each target file to be inspected and search for suspicious sensitive information points in it using predefined regular expressions, feature strings and other rules. After finding the sensitive point, the system will extract its context information, such as the code file, class name, method name, line number, etc., and calculate the MD5 and other summary values of the sensitive content. After the scan is completed, the system will summarize the information of all sensitive points to generate a report and associate it with the original installation package. This report can be used to evaluate the privacy compliance of the application or assist developers in making targeted code security improvements.
[0065] In some embodiments, scanning efficiency and accuracy can be improved in the following ways: Optionally, the detection system preprocesses the target code file before scanning, such as removing comments, standardizing naming, etc., to reduce interference factors; Optionally, the detection system uses a multi-level filtering mechanism, first using lightweight string matching to quickly filter out obviously harmless code, and then using more accurate methods such as regular expressions and syntax analysis to detect remaining suspicious points; Optionally, the detection system clusters and subtracts the scan results, merges content with high similarity, and removes repeated known sensitive points. It is understandable that the scanning strategy can also be customized according to different compliance requirements, such as privacy regulations in different regions and industries, which are not limited here.
[0066] The following is a more detailed description of the process of the method provided by this implementation. Figure 2 , is another flow chart of the method for detecting sensitive information of an installation package in an embodiment of the present application.
[0067] S201 . In response to a detection start instruction input by a user, obtain a detection path and a type to be detected set by the user.
[0068] Referring to step S101, the detection system obtains the detection path and the type to be detected set by the user.
[0069] S202: Identify multiple initial installation packages in an initial folder corresponding to the detection path.
[0070] Referring to step S102, the detection system identifies multiple initial installation packages.
[0071] S203: Obtain metadata information of the initial installation package.
[0072] Metadata information refers to data that describes the basic properties of the installation package, which is usually contained in the configuration file or header of the installation package. Package name information represents the unique identifier of the installation package, which is used to distinguish different applications. Version information refers to the version number of the installation package, which reflects the update and iteration status of the application.
[0073] Specifically, after identifying the installation packages in the initial folder, the detection system will read the metadata information of the installation packages one by one. The way to obtain the metadata depends on the specific format of the installation package. For example, for an APK file, the AndroidManifest.xml configuration file can be parsed to extract the package name and version number.
[0074] In some embodiments, the metadata information of the initial installation package can be obtained in the following ways: Optionally, the detection system maintains a mapping table between the installation package format and the metadata extraction script, automatically selects the corresponding parsing script according to the type of the installation package, and reads the package name and version and other information from the specified location; Optionally, the detection system uses the installation package decompression tool to extract metadata, for example, using a general decompression program to read the ZIP format installation package and obtain the file attributes therefrom. It is understandable that the acquisition of metadata can also be achieved by using technical means such as the package management interface provided by the operating system or a custom metadata extraction algorithm, which is not limited here.
[0075] S204: Extracting, based on the metadata information, initial installation packages whose similarity to historical installation packages is higher than a preset similarity threshold as installation packages to be optimized.
[0076] Among them, historical installation packages refer to installation packages that have completed detection and have a cache in the decompilation optimization folder. The decompilation optimization folder is a type of target folder. Similarity refers to the consistency of the package name and version number of two installation packages, which can be quantified by a string matching algorithm. The preset similarity threshold is used to determine whether two installation packages are similar enough to determine whether historical detection results can be reused. The installation package to be optimized refers to the installation package that can be decompiled and optimized after similarity comparison.
[0077] Specifically, after obtaining the metadata information of the installation package to be inspected, the detection system will compare it with the metadata of the historical installation package and calculate the similarity. The calculation of similarity usually needs to consider the weight distribution of the package name and version number, as well as factors such as whether fuzzy matching is allowed. For installation packages whose similarity exceeds the preset threshold, the detection system determines that the difference with the historical version is small, and most of the code segments in the binary file are the same. Therefore, the decompilation results of the historical version can be directly used without repeated processing. These installation packages are marked as installation packages to be optimized. By setting a reasonable similarity threshold by the user, the detection system can make full use of the historical cache while ensuring that the detection of new code segments is not missed, achieving a balance between detection efficiency and accuracy.
[0078] In some embodiments, the installation package to be optimized can be extracted in the following manner: Optionally, the detection system sets different matching rules and weights for the package name and version number, respectively. For example, the package name must be exactly the same, and the version number allows the last digit to be different. Then, a weighted average is taken of the similarity scores of the two parts, and those exceeding 0.8 are considered to be installation packages to be optimized. Optionally, when comparing metadata, the detection system not only considers the literal similarity, but also combines other attributes such as file size and modification time to make a more comprehensive judgment on the overall similarity of the installation package.
[0079] S205: Determine the same code segments corresponding to the installation package to be optimized and the historical installation package as non-compiled code segments, and remove the non-compiled code segments in the installation package to be optimized to obtain the code segments to be compiled.
[0080] The code segment refers to an independent functional module or class definition in the source code file obtained after the installation package is decompiled. The non-compiled code segment refers to the code segment that is exactly the same in the installation package to be optimized and the historical installation package and does not need to be recompiled. The code segment to be compiled refers to the code segment that remains after removing the non-compiled code segment and needs to be recompiled.
[0081] Specifically, after identifying the installation package to be optimized, the detection system will further compare it with the historical installation package of the corresponding version, and determine the identical code segments between the two through differential analysis. Through code comparison technology, find out the code fragments with completely consistent functions and implementations. After identifying the non-compiled code segments, the detection system can directly remove them from the installation package to be optimized, and only retain the parts that are different and need to be recompiled, thereby obtaining the code segments to be compiled. Through this incremental decompilation process, the detection system can minimize the actual amount of compilation that needs to be performed, make full use of the existing historical cache, and greatly shorten the detection time of the installation package to be optimized.
[0082] In some embodiments, the non-compiled code segments can be determined and the code segments to be compiled can be obtained in the following ways: Optionally, after obtaining the decompilation results of the two versions, the detection system performs text difference comparison on each file, finds out the source code files that are exactly the same, and regards them as non-compiled code segments; then considers the dependencies between files, and recursively determines the affected compiled code segments; Optionally, the detection system performs incremental processing in the decompilation stage, locates the difference intervals between the two versions of the installation packages at the bytecode level by means of grammatical analysis, and only compiles the difference parts, and regards the non-difference parts as non-compiled code segments. It is understandable that the difference comparison function interface based on version management tools such as Git can also be called, or technical means such as code similarity analysis tools can be introduced to achieve the distinction between compiled code segments, which is not limited here.
[0083] S206, copy the code segment to be compiled into the decompilation optimization folder as the installation package to be checked. The decompilation optimization folder is a type of target folder.
[0084] The decompilation optimization folder refers to a target folder used to store the decompilation results of historical installation packages and accelerate the detection of installation packages to be optimized. The installation package to be detected refers to the installation package that has been decompiled and optimized and is waiting for the subsequent detection step.
[0085] Specifically, after completing the preprocessing of the installation package to be optimized, the detection system will copy the screened code segments to be compiled to the decompilation optimization folder and use it as a new installation package to be tested. Since the non-compiled part has been removed, this new package to be tested will be smaller and easier to handle than the original installation package. The subsequent decompilation process can also be performed directly in the optimization folder without occupying the storage space of the initial folder. At the same time, the corresponding historical installation package decompilation results will also be retained in place. When a new installation package with a higher similarity appears later, it can be directly reused, eliminating the overhead of repeated compilation, thereby achieving continuous optimization of detection performance.
[0086] S207 , establishing a preset number of decompilation processes, and respectively decompiling the installation packages to be checked in the multiple target folders to obtain source code packages to be checked.
[0087] Referring to step S104, the detection system decompiles the installation package to be detected in the target folder including the decompilation optimization folder to obtain the source code package to be detected.
[0088] In some embodiments, the detection system will establish a preset number of decompilation processes; then, based on the decompilation processes, the detection system will decompile the installation packages to be checked in the target folder in a preset decompilation order to obtain the source code packages to be checked; in the target folder, the detection system will replace the corresponding installation packages to be checked with the source code packages to be checked; during the decompilation process, if a priority adjustment instruction input by the user is received, the detection system will adjust the preset decompilation order based on the priority adjustment instruction to obtain a decompilation guidance order; and in accordance with the decompilation guidance order, after the current decompilation thread is completed, continue to decompile the remaining installation packages to be checked in the target folder; the decompilation thread is a sub-process of the decompilation process, which is used to decompile a certain installation package to be checked.
[0089] Among them, the decompilation thread refers to a relatively independent execution flow used to handle the decompilation task of a single installation package within the decompilation process. The preset decompilation order refers to the rules pre-defined by the detection system for sorting a batch of installation packages to be decompiled. The decompilation guidance order refers to the new processing order obtained by dynamically adjusting the preset decompilation order according to the user's instructions during the decompilation process.
[0090] Specifically, after starting multiple decompilation processes, the detection system will establish multiple decompilation threads within each process to perform specific installation package unpacking and source code generation operations. At the same time, the detection system will sort all installation packages in the target folder according to the pre-set decompilation priority rules to obtain an orderly decompilation task queue. The decompilation thread will receive and process the installation packages one by one in this preset order. When the decompilation of an installation package is completed, the decompilation thread will write the obtained source code package back to the target folder, overwriting the original installation package file, so as to provide a unified input format for the scanning program. During the decompilation process, if the detection system receives an instruction from the user requesting to adjust the processing priority of certain installation packages, the original decompilation order will be rearranged, a new decompilation guidance order will be generated, and all decompilation threads will continue to perform subsequent processing in accordance with the new order, thereby realizing dynamic optimization of the detection task.
[0091] In some embodiments, after establishing a preset number of decompilation processes, the detection system will query the estimated remaining time of multiple decompilation processes if it detects that a new installation package appears in the initial folder during the decompilation process; the target folder corresponding to the decompilation process with the shortest estimated remaining time is the folder to be updated, and the new installation package is allocated to the folder to be updated.
[0092] Among them, the newly added installation package refers to the installation package file that the detection system finds to be newly added to the initial folder by the user after the start of this detection task and has not yet been included in the detection scope. The estimated remaining time means the estimated time when the process ends after completing the existing tasks, obtained by the detection system through the evaluation of the current task queue, processing speed and other factors of the decompilation process. The folder to be updated refers to the target folder with the smallest process task volume that is selected as the copy target of the newly added installation package after time evaluation.
[0093] Specifically, in order to dynamically adapt to the user's new detection needs during the detection process, the detection system will continue to monitor the changes in the file list of the initial folder after the decompilation process is started. Once a new installation package is detected to be added to the initial folder, and the new installation package has not been assigned to any target folder before, the detection system will immediately start a fast allocation process. First, the detection system will send an estimated remaining time query request to all running decompilation processes, requiring them to estimate and return the remaining time to complete all currently assigned tasks. After collecting this feedback information, the detection system will compare the estimated duration of different processes, select the process with the least remaining tasks and the earliest to be completed, and mark the target folder corresponding to the process as the folder to be updated. Finally, the detection system will copy the newly identified new installation package to this folder to be updated, so that it will be automatically incorporated into the task queue of the decompilation process, and will be decompiled in time after all the original installation packages are processed.
[0094] S208 , screening multiple source code files in the source code package to be inspected to obtain target files to be inspected that belong to the type to be inspected.
[0095] Referring to step S105 , the detection system will screen the target files to be inspected.
[0096] S209: Scan the target files to be inspected respectively, and generate a sensitive scanning report corresponding to the initial installation package based on the scanning results.
[0097] Referring to step S106, the detection system scans the target file to be inspected and generates a sensitive scanning report.
[0098] In some embodiments, the monitoring system determines the file type of the target file to be inspected; if the target file to be inspected is a text file, then based on memory mapping technology, the text file is mapped into the memory for scanning to obtain the file scanning result; if the target file to be inspected is a binary file, the string sequence of the binary file is extracted and scanned as the part to be scanned to obtain the file scanning result; based on the scanning result, a sensitive scanning report corresponding to the initial installation package is generated, and the sensitive scanning report is displayed.
[0099] The file type refers to the data storage format of the target file to be inspected, which can usually be divided into two categories: text files and binary files. The string sequence represents the ordered arrangement of printable characters extracted from the binary file.
[0100] Specifically, after obtaining the source code package to be inspected, the detection system will further analyze and filter each source code file to find the target files that may contain sensitive information. Then, the detection system will conduct in-depth scans on these target files one by one. Among them, the detection system will adopt different scanning strategies for different types of files. If the target file is in text format, such as .java, .xml, etc., the detection system will use memory mapping technology to load the entire file into memory at one time, and then use fast algorithms such as string search and regular expression matching to locate and identify sensitive information in the memory data; if the target file is in binary format, such as .so, .dll, etc., the detection system will first extract printable characters from the file content to obtain an intermediate result of character serialization, and then perform search and matching in this character sequence to locate sensitive information. The result of the scan is a series of sensitive information description records, including the type of information, the file where it is located, the line number of code, etc. Finally, the detection system will summarize and format all the scan results to generate a sensitive information scan report, including the content summary, code location, risk level assessment, etc. of various sensitive information. This report can help developers quickly locate and fix security risks in their applications, and can also serve as an important basis for application security audits.
[0101] S210: Determine historical scanning data corresponding to the non-compiled code segment in the historical installation package according to the detection record of the historical installation package.
[0102] The detection record refers to the data generated by the historical installation package in the past detection process, which describes the scanning results of sensitive information in the code segment. The historical scanning data represents the partial detection results corresponding to the non-compiled code segment that can be directly reused.
[0103] Specifically, since the non-compiled code segments in the installation package to be optimized are exactly the same as the historical version, the distribution of sensitive information inside them will not change. Therefore, when the detection system is scanning the installation package to be checked, for those unmodified non-compiled parts, the corresponding scanning results can be directly extracted from the historical detection records without repeating the scanning operation. By maintaining a mapping relationship between a historical installation package and its detection records, the detection system can accurately locate the scanning data of different code segments and reuse them at the right time, thereby minimizing the detection overhead.
[0104] S211. Integrate historical scan data into sensitive scan reports.
[0105] Specifically, after completing the extraction of historical scan data and the real-time scanning of the code segments to be compiled, the detection system needs to integrate the two parts of the detection results together to generate a complete report reflecting the real sensitive information risks of the current installation package. The integration process needs to deal with the format compatibility of the scan data, align the file paths of the code segments and the installation packages, as well as deduplication and sorting, and finally form a scan report with a clear structure and complete content.
[0106] In some embodiments, historical scan data can be integrated into sensitive scan reports in the following ways: Optionally, the detection system inserts historical scan data and newly added detection results into corresponding positions of the scan report in the order in which the code segments appear in the installation package to ensure the continuity and readability of the report content; for repeated scan results, the historical data shall prevail and no longer be recorded repeatedly; Optionally, the detection system defines a unified scan result mode, which is converted to this standard format when extracting historical data and generating real-time results; in the integration stage, the two types of results are directly connected, and data normalization is performed again to ensure the consistency of the scan report. It is understandable that other technical means such as data synchronization solutions or report generation tools can also be used to achieve seamless integration of historical data and incremental results, which are not limited here.
[0107] In the above embodiment, the traditional serial detection architecture is significantly improved by introducing key technologies such as multi-process parallelism and task hierarchical scheduling. In practical applications, the method of this application can also be combined with the cloud computing architecture to organically integrate the local detection platform with the cloud elastic cluster through the hybrid detection mode of cloud-edge collaboration, further expanding the detection scale and service capabilities. The following is a supplement to the scenario of this embodiment.
[0108] After successfully launching the new version of the installation package detection platform and operating it stably for a period of time, Company A further considered whether it could integrate the concept of cloud computing and migrate the installation package detection processing to the high-computing power server on the cloud, thereby greatly expanding the system's service capabilities. The system adopts a hybrid detection mode of cloud-edge collaboration, where the local detection server works with the cloud elastic cluster: the local server is responsible for collecting and aggregating installation package files, uploading the installation package to the cloud for detection, and the detection engines on both ends share the detection file task, which can achieve dual-end cooperation and significantly improve detection processing efficiency.
[0109] The following is another more specific description of the process of the method provided by this implementation. Figure 3 , which is another flow chart of the method for detecting sensitive information of an installation package in an embodiment of the present application.
[0110] S301 : In response to a detection start instruction input by a user, obtain a detection path and a type to be detected set by the user.
[0111] Referring to step S101, the detection system obtains the detection path and the type to be detected set by the user.
[0112] S302: Identify multiple initial installation packages in an initial folder corresponding to the detection path.
[0113] Referring to step S102, the detection system identifies multiple initial installation packages.
[0114] S303: Determine whether the initial installation package is set with cloud upload permission.
[0115] The cloud upload permission refers to the authorization to upload the installation package to be tested to the cloud server for testing. The initial installation package refers to the original installation package to be tested identified from the initial folder.
[0116] Specifically, after identifying the initial installation package, the detection system will check each installation package one by one to see if it has the permission to upload and detect in the cloud. This permission information may come from the internal configuration of the installation package, or it may be uniformly set by the user in the detection policy. The detection system will read the predefined permission acquisition method, such as parsing specific tags in AndroidManifest.xml to determine the upload permission status of the installation package. Installation packages with cloud permissions can be assigned to the cloud resource pool and submitted to the cloud server via network transmission; installation packages without cloud permissions can only be detected in the local environment and assigned to a local folder. By distinguishing the cloud permissions of different installation packages, the detection system can flexibly choose to use local or cloud computing resources while taking into account data security to optimize detection efficiency.
[0117] In some embodiments, the cloud upload permission of the initial installation package can be determined in the following ways: Optionally, the detection system requires that the installation package project to be detected must explicitly declare its cloud detection permission when submitting, embed the permission information into the metadata area of each installation package, and directly read the permission identifier in the metadata when confirming the right; Optionally, the detection system generates a unique authorization code for each project to be inspected, and the user chooses whether to enable cloud detection in the project configuration interface, and matches the project configuration according to the authorization code when identifying the installation package to obtain the cloud permission status. It is understandable that the determination of the cloud upload permission of the installation package can also be achieved by using third-party cloud security authentication services, or by connecting to the organization's internal permission management system and other technical means, which are not limited here.
[0118] S304: If yes, obtain the file size of the initial installation package.
[0119] The file size indicates the storage space size of the installation package file, usually in bytes.
[0120] Specifically, for those installation packages that have cloud upload permissions, the detection system will first obtain their file size information before copying and allocating them to the cloud folder. File size is a key factor affecting cloud detection time and network transmission costs, so it needs to be known in advance. The detection system can quickly obtain the exact size of an installation package file by calling the file system API or executing a specific shell command. After obtaining the file size, the detection system will record it for subsequent detection time estimation and upload time prediction to determine the final installation package allocation plan. Through the size attribute data of the installation package, the detection system can make a more accurate assessment of the time and performance of cloud detection, thereby realizing the rule allocation of detection tasks locally and in the cloud.
[0121] S305: If not, copy and distribute the initial installation package to a local folder as the installation package to be checked.
[0122] The local folder refers to a folder set on the local computer where the detection system is located and used to store the installation package to be checked. The installation package to be checked refers to the installation package file that needs to be decompiled and scanned in the local environment.
[0123] Specifically, when the detection system finds that a certain initial installation package does not have cloud upload permissions, it will copy it directly to the local folder and include it in the scope of the local detection task. This ensures that all installation packages to be detected can be properly handled and will not be missed due to cloud permission restrictions. At the same time, since the local folder is set in the detection system's own storage space, copying the installation package without cloud permissions here can also avoid additional network transmission overhead and security risks. After completing the copying of the installation package in the local folder, the detection system will update the task allocation record and add these local exclusive packages to be inspected to the subsequent decompilation and scanning process.
[0124] In some embodiments, before distribution, the detection system obtains the package name information and version number information of the initial installation package; according to the package name information and the version number information, the initial installation package is renamed so that the initial installation package is arranged according to a preset priority; the preset priority is determined by the user's preset weight, the file size weight in the package name information, and the version number weight in the version number information.
[0125] The package name information refers to the unique identifier of the installation package, which is usually defined by the provider of the software package and is used to distinguish different applications. The version number information refers to the version number of the installation package and is used to identify different release versions of the application.
[0126] Specifically, before distributing the initial installation package to the target folder, the detection system will first extract two key attributes of the installation package: package name information and version number information. Then, the detection system will use a preset priority evaluation function to calculate the priority value of each installation package based on the package name information, version number information and other relevant factors. Among them, multiple weight parameters are set in the priority evaluation function, including: fixed weights pre-configured by the user, dynamic weights of the installation package file size, dynamic weights of the installation package version number, etc. The detection system substitutes the information extracted from multiple dimensions into the evaluation function, and obtains a value that reflects the comprehensive priority of the installation package after weighted calculation. Finally, the detection system will rename all the installation packages to be distributed based on these priority values, so that the file names contain the priority information encoding, and they are arranged in order from high to low priority.
[0127] S306: Predicting the local detection time of the initial installation package on the local computer and the cloud detection time on the cloud server respectively according to the file size.
[0128] The local detection time refers to the estimated processing time required to decompile and scan the installation package on the local computer. The cloud detection time refers to the estimated processing time required to decompile and scan the installation package on the cloud server. The predicted detection time refers to the theoretical time required to complete the installation package detection in a specific environment based on factors such as the size of the installation package and environmental parameters.
[0129] Specifically, after obtaining the file size attributes of the initial installation package, the detection system will predict the detection time in both local and cloud environments. The calculation of the predicted time is usually based on a complex mathematical model, which needs to consider factors such as the size and complexity of the installation package itself, as well as the hardware configuration and resource load of the detection environment. The detection system will use the file size as the main input parameter of the model, and combine it with the preset local and cloud environment parameters to evaluate the theoretical time required to decompile and scan the installation package under the two deployment methods. By comparing the local detection time and the cloud detection time, the detection system can preliminarily determine the efficiency of different detection methods for the installation package.
[0130] When predicting cloud detection time, in addition to considering the size of the installation package file, you can also refer to the historical processing time of the cloud server. Specifically, the detection system can record the actual time taken to upload the installation package to the cloud for processing each time, and save it in association with factors such as the size of the installation package and the cloud hardware configuration. When you encounter an installation package of similar size in the future, you can directly use historical data to make time predictions without recalculating.
[0131] For example, each time the detection system uploads the installation package to the cloud for processing, it will record the actual time T of this detection task and related influencing factors, such as the size S of the installation package, the CPU model C of the cloud server, the memory capacity M, etc., forming a historical record.<T,S,C,M> When a new installation package to be checked is encountered later (size S'), the system first searches the historical records for a set of records with the same configuration as the current cloud server (i.e., C and M are the same) and whose installation package size is close to S', and then takes the weighted average of the execution time T in this set of records according to S' to obtain T'. If the confidence of T' exceeds 95%, T' is directly used as the estimated detection time of this installation package in the cloud, thus avoiding repeated estimation overhead.
[0132] In some embodiments, the local and cloud detection time of the initial installation package can be predicted in the following ways: Optionally, the detection system builds a detection time prediction model based on linear regression and uses massive historical detection data for training; each time a prediction is made, the installation package file size and standardized environmental parameters are input into the model to obtain an estimate of the local and cloud detection time; Optionally, the detection system regularly performs benchmark tests in local and cloud environments to obtain real detection time samples of installation packages of different sizes; when predicting, the theoretical detection time for local and cloud is calculated based on the file size by interpolation or extrapolation. It is understandable that other machine learning algorithms or empirical formulas can also be used to achieve the prediction of detection time, which is not limited here.
[0133] Specifically, the detection time prediction model takes the metadata information of the installation package to be tested (such as file size, format, architecture, etc.) as input, and outputs the estimated time required to complete the decompilation task of the installation package under the specified software and hardware environment. The model is trained using a machine learning algorithm based on a gradient boosting regression tree, using a large number of real decompilation time data samples as a training set, and optimizing the model parameters by minimizing the mean square error between the predicted time and the actual time. The detection time prediction model can help the detection system to rationally plan and schedule decompilation tasks and improve resource utilization efficiency.
[0134] S307: Calculate the absolute value of the difference between the cloud detection time and the local detection time as the cloud advantage time.
[0135] The cloud advantage time refers to the amount of time saved by using cloud detection compared to local detection, which is represented by the absolute value of the difference between the cloud detection time and the local detection time. The cloud advantage time is calculated to evaluate the time benefit of cloud detection and provide a reference for task allocation decisions.
[0136] Specifically, after the detection system completes the prediction of the local detection time and cloud detection time of a certain installation package, it will compare the two time values to obtain the time difference that can be saved by using cloud detection. By calculating the absolute value of the difference between the two, the detection system can quantify the time advantage of cloud detection, that is, how much time can be saved by using cloud detection compared to local detection. The cloud advantage time is an important evaluation indicator that directly reflects the room for improvement in cloud detection efficiency. The detection system will weigh the cloud advantage time with other factors (such as network transmission time, etc.) and comprehensively consider whether it is worth uploading the installation package to the cloud for detection.
[0137] When choosing whether to upload the installation package to the cloud for processing, in addition to considering the transmission time and cloud acceleration effect, the real-time load status of the cloud server can also be taken into account. The detection system can regularly query the utilization rate of the cloud server's CPU, memory, network and other resources. When these indicators are lower than a certain preset threshold, the installation package will be considered for uploading to avoid increasing the processing pressure on the cloud.
[0138] For example, the detection system sends a status query request to the cloud server every 10 minutes to obtain key indicators such as the current CPU usage C, memory usage M, upstream and downstream network bandwidth BU and BD of the cloud server; after receiving the request, the cloud server packages the real-time indicators into a status message and returns it to the detection system. After receiving the status message, the detection system determines whether each indicator is lower than the preset idle threshold (such as C <50% and M <60% and BU>1MB / s and BD>2MB / s). If so, the installation package to be tested is added to the upload queue, otherwise polling continues until the conditions are met. By monitoring the cloud status in real time, the detection task can be scheduled in time when the cloud load is low, making full use of system resources.
[0139] S308: Obtain the current network speed status, and determine the cloud upload time of the initial installation package according to the file size and the network speed status.
[0140] The network speed refers to the speed of the network connection between the current detection system and the cloud server, usually in bytes or bits per second. The cloud upload time refers to the estimated time required to upload the installation package from the local detection environment to the cloud server, which is determined by the file size and the network speed.
[0141] Specifically, after obtaining the time advantage of cloud detection, the detection system also needs to evaluate the time cost of uploading the installation package to the cloud. Only when the cloud advantage time is greater than the cloud upload time, using cloud detection is an expected choice. In order to predict the upload time, the detection system will obtain the current network speed status in real time, and obtain the theoretical upload time through simple calculations with the file size. The network speed status can be obtained by calling the network diagnostic API or executing a third-party speed test tool. After determining the cloud upload time, the detection system will compare it with the previously obtained cloud advantage time. If the upload time is less than the advantage time, it tends to assign the installation package to cloud detection; otherwise, local detection is given priority. By evaluating the network speed status and upload time in real time, the detection system can dynamically adjust task allocation according to the current network conditions and improve the actual benefits of cloud detection.
[0142] S309: When the cloud upload time is less than the cloud advantage time, copy and distribute the initial installation package to the cloud folder as the installation package to be checked.
[0143] The cloud folder refers to a folder created on the cloud server for receiving the installation packages to be inspected uploaded from the local inspection system. Copying the initial installation package to the cloud folder means copying it from the local environment to the cloud environment through network transmission and incorporating it into the cloud inspection scope.
[0144] Specifically, when the detection system determines that the cloud upload time of a certain installation package is less than the cloud advantage time, it considers that the installation package is suitable for cloud detection and will copy it to the cloud folder for subsequent processing. The detection system uploads the installation package from the local initial folder to a dedicated folder in the cloud environment through the file transfer channel between the detection system and the cloud server. The upload process is essentially a cross-network file copying behavior, which will be affected by factors such as network speed.
[0145] Considering that the configuration of cloud servers in actual scenarios may vary greatly, the hardware configuration of the cloud can be divided into several gradients, and different detection strategies can be set for different gradients. For example, for low-configuration cloud servers, only a small number of installation packages can be uploaded each time; for high-configuration cloud servers, batches of installation packages can be uploaded at one time. The configuration level of the cloud can be evaluated through regular scoring test results.
[0146] For example, for high-performance cloud servers, up to 50 installation packages can be uploaded in batches for each detection task; for medium-performance cloud servers, only 10 installation packages can be uploaded for each detection task; and for low-performance cloud servers, the frequency of use should be reduced, and only a single small installation package should be assigned to it for processing. Finally, in the actual detection process, the scheduler will give priority to matching newly received installation packages with high-performance cloud servers; if the high-performance server is busy, it will consider sending the installation package to the second-highest performance server for execution, and so on. By building a multi-level detection strategy, detection tasks can be allocated in grades according to the actual performance of the cloud server, which can not only give full play to the computing advantages of high-performance servers, but also avoid paralysis of low-performance servers. At the same time, this hierarchical scheduling also helps to quickly transfer tasks to other nodes when a local node fails, thereby improving the adaptability of the system.
[0147] In some embodiments, after the installation package is copied to the cloud folder, the detection system can also perform the following steps: the detection system compresses and packages the installation package files in the cloud folder to generate a compressed package file in a unified format; the detection system calls a built-in or third-party network disk synchronization tool to synchronize the compressed package files to the cloud network disk system; the detection system packages the compressed package file address in the cloud network disk together with the metadata information of the installation package into a task message and sends it to the message queue corresponding to the cloud detection cluster; the cloud detection cluster monitors the message queue, obtains new detection tasks, and schedules idle detection engine nodes to perform remote decompression and analysis. This method of centrally packaging and uploading installation packages and driving cloud detection through task messages can further improve the efficiency of network transmission and computing resource utilization.
[0148] It should be noted that, in theory, due to the high computing power of cloud servers, cloud detection time is generally shorter than local detection time, so the above-mentioned planning of uploading to the cloud for processing will occur. However, in some embodiments, due to the optimization and upgrade of the user's local computer, the local detection time may be shorter than the cloud detection time. In this case, the detection system will assign most of the detection tasks to the local computer, and for file detection tasks that exceed the computing power processing range, they will be uploaded to the cloud server for processing.
[0149] S310: When the cloud upload time is not less than the cloud advantage time, copy and distribute the initial installation package to a local folder as the installation package to be checked.
[0150] The local folder refers to a folder set up on the local computer where the detection system is located, which is used to store the assigned and screened installation packages to be detected. Copying the initial installation package to the local folder means copying it from the initial folder to another specific folder within the local environment and including it in the scope of local detection.
[0151] Specifically, when the detection system determines that the cloud upload time of a certain installation package is greater than or equal to the cloud advantage time, it considers that the installation package is not suitable for cloud detection and will copy it to a local folder for subsequent processing. Unlike cloud folders, local folders are set in the detection system's own storage space, so copying the installation package locally is just a simple file copy operation and will hardly incur additional time overhead. Local folders, like cloud folders, are dedicated folders used by the detection system to organize and manage installation packages to be inspected, but the two belong to different physical environments. Once the installation package is copied to the local folder, the subsequent decompilation and scanning processes will be carried out entirely in the local environment, without being transmitted over the network.
[0152] S311 , establishing a preset number of decompilation processes, and decompiling the installation packages to be checked in multiple target folders respectively to obtain source code packages to be checked.
[0153] Referring to step S104, the detection system decompiles the installation package to be detected to obtain a source code package to be detected.
[0154] In some embodiments, the detection system will first determine whether there is a corresponding decompilation tool based on the installation package format information of the installation package to be checked; if so, it will call the execution path of the decompilation tool; if not, based on the user's tool option selection, it will obtain the external decompilation tool corresponding to the installation package to be checked from the local or network, and obtain the execution path of the external decompilation tool.
[0155] Among them, the installation package format information refers to the data organization and packaging specifications followed by the installation package file. Different operating systems and software platforms have different corresponding installation package formats, such as Android's APK format, iOS's IPA format, Windows' MSI format, etc. A decompilation tool refers to a special software that can unpack the installation package file and extract the program code and resource files therein. The tool option indicates the method of obtaining the alternative decompilation tool specified by the user when the built-in decompilation tool is not found. It can be selected from the local file system or downloaded from the Internet. An external decompilation tool refers to a third-party decompilation tool software that is independent of the detection system. The execution path indicates the program entry for starting a software tool, which usually includes the name of the executable program file and the path to the folder where it is located.
[0156] Specifically, before preparing to perform a decompilation operation on the installation package in the target folder, the detection system first needs to determine a suitable decompilation tool. Usually, the detection system will have some built-in decompilation tools for common installation package formats for direct calling. However, considering that the formats of installation packages in actual scenarios may be diverse, the built-in decompilation tools of the detection system may not necessarily cover all types. Therefore, before starting the decompilation task, the detection system will read the format information of the installation package to be inspected, and then search for a matching decompilation tool in the internal tool configuration list. If a match is found, the execution entry path of the built-in tool is directly extracted for subsequent calls; if no built-in tool with a matching format is found, the detection system will prompt the user whether to use an external third-party decompilation tool. The detection system will display a tool selection dialog box, listing two alternatives: selecting tool software from the local file system, or downloading tool software from the network. After the user makes a choice, the detection system will obtain the executable program path of the external tool and prepare for the call. By flexibly using both built-in and external decompilation tools, the detection system can adapt to various installation package format scenarios to meet the needs of detection tasks.
[0157] S312: Screen multiple source code files in the source code package to be inspected to obtain target files to be inspected that belong to the type to be inspected.
[0158] Referring to step S105 , the detection system will screen the target files to be inspected.
[0159] S313: Scan the target files to be inspected respectively, and generate a sensitive scanning report corresponding to the initial installation package based on the scanning results.
[0160] Referring to step S106, the detection system scans the target file to be inspected and generates a sensitive scanning report.
[0161] In the embodiment of the present application, due to the use of multi-process parallel decompilation and multi-level dynamic scheduling mechanism, the installation package to be detected is copied and distributed to different detection processes according to preset rules, and dynamic adjustment of priority is supported, so that the system resource advantages can be maximized, and the problems of low detection efficiency and inability to support elastic expansion existing in the traditional stand-alone serial detection architecture are effectively solved, thereby realizing the rapid and safe detection of massive mobile application installation packages. In addition, due to the use of detection acceleration technologies such as memory scanning and cloud collaboration, and the optimization strategies are set for different file types and risk levels, while fully covering the content of the installation package, the accuracy of sensitive information discovery is also improved, and flexible detection strategy customization is supported. In summary, the installation package detection method of the present application can comprehensively improve the overall effectiveness of mobile application security detection from multiple angles such as speed and accuracy while reducing system overhead as much as possible.
[0162] The following describes the detection system in the embodiment of the present invention from the perspective of hardware processing. Figure 4 , is a schematic diagram of a physical device structure of a detection system in an embodiment of the present application.
[0163] It should be noted that Figure 4 The structure of the detection system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0164] like Figure 4 As shown, the detection system includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage part 408 to the random access memory (RAM) 403, such as executing the method described in the above embodiment. In the RAM 403, various programs and data required for system operation are also stored. The CPU 401, the ROM 402 and the RAM 403 are connected to each other through the bus 404. The input / output (I / O) interface 405 is also connected to the bus 404.
[0165] The following components are connected to the I / O interface 405: an input section 406 including an audio input device, a button switch, etc.; an output section 407 including a liquid crystal display (LCD) and an audio output device, an indicator light, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed so that a computer program read therefrom is installed into the storage section 408 as needed.
[0166] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 409, and / or installed from a removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, various functions defined in the present invention are performed.
[0167] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.
[0168] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings.
[0169] Specifically, the detection system of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, the method for detecting sensitive information of the installation package provided in the above embodiment is implemented.
[0170] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the detection system described in the above embodiment; or may exist independently without being assembled into the detection system. The above storage medium carries one or more computer programs, and when the above one or more computer programs are executed by a processor of the detection system, the detection system implements the method for detecting sensitive information of the installation package provided in the above embodiment.
[0171] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
[0172] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.
[0173] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.
Claims
1. A method for detecting sensitive information of an installation package, characterized in that: Applied to a detection system, the method comprises: In response to a detection start instruction input by a user, obtaining a detection path and a type to be detected set by the user; Identify multiple initial installation packages in the initial folder corresponding to the detection path; The initial installation package is copied and distributed to multiple target folders as installation packages to be inspected according to a preset distribution rule; the multiple target folders include multiple decompilation optimization folders; the multiple decompilation optimization folders respectively cache historical installation packages that have completed inspection in different project fields; The step of copying and distributing the initial installation package to multiple target folders as the installation package to be checked according to the preset distribution rule specifically includes: obtaining metadata information of the initial installation package; extracting, according to the metadata information, the initial installation package whose similarity with the historical installation package is higher than a preset similarity threshold as the installation package to be optimized; determining the same code segment corresponding to the installation package to be optimized and the historical installation package as the non-compiled code segment, and removing the non-compiled code segment in the installation package to be optimized to obtain the code segment to be compiled; copying the code segment to be compiled to the decompiled optimization folder as the installation package to be checked; Establishing a preset number of decompilation processes, respectively decompiling the installation packages to be checked in the plurality of target folders to obtain source code packages to be checked; the decompilation processes correspond to the target folders one by one; Screening multiple source code files in the source code package to be inspected to obtain target files to be inspected that belong to the type to be inspected; The target files to be inspected are scanned respectively, and a sensitivity scanning report corresponding to the initial installation package is generated based on the scanning results.
2. The method according to claim 1, characterized in that After the steps of respectively scanning the target files to be inspected and generating a sensitivity scanning report corresponding to the initial installation package based on the scanning results, the method further includes: Determining, according to the detection record of the historical installation package, historical scanning data corresponding to the non-compiled code segment in the historical installation package; The historical scan data is integrated into the sensitive scan report.
3. The method according to claim 1, characterized in that: The target folder includes a cloud folder set on the cloud server and a local folder set on the local computer; The step of copying and distributing the initial installation package to multiple target folders as the installation packages to be checked according to the preset distribution rule specifically includes: Obtaining the file size of the initial installation package; According to the file size, respectively predicting the local detection time of the initial installation package on the local computer and the cloud detection time on the cloud server; the local detection time is not less than the cloud detection time; Calculate the absolute value of the difference between the cloud detection time and the local detection time as the cloud advantage time; Obtaining the current network speed status, and determining the cloud upload time of the initial installation package according to the file size and the network speed status; When the cloud upload time is less than the cloud advantage time, copy and distribute the initial installation package to the cloud folder as the installation package to be checked; When the cloud upload time is not less than the cloud advantage time, the initial installation package is copied and distributed to the local folder as the installation package to be checked.
4. The method according to claim 3, characterized in that The step of obtaining the file size of the initial installation package specifically includes: Determine whether the initial installation package is set with cloud upload permission; If yes, obtaining the file size of the initial installation package; If not, the initial installation package is copied and distributed to the local folder as the installation package to be checked.
5. The method according to claim 1, characterized in that The step of establishing a preset number of decompilation processes, respectively decompiling the installation packages to be checked in the plurality of target folders to obtain the source code packages to be checked specifically includes: Establish a preset number of decompilation processes; Based on the decompilation process, decompile the installation package to be checked in the target folder in accordance with a preset decompilation order to obtain a source code package to be checked; In the target folder, replacing the corresponding installation package to be checked with the source package to be checked; During the decompilation process, if a priority adjustment instruction input by a user is received, the preset decompilation order is adjusted based on the priority adjustment instruction to obtain a decompilation guidance order; According to the decompilation guidance sequence, after the current decompilation thread is completed, the remaining installation packages to be checked in the target folder continue to be decompiled; the decompilation thread is a sub-process of the decompilation process, which is used to decompile a certain installation package to be checked.
6. The method according to claim 5, characterized in that After the step of establishing the decompilation process of the preset process number, the method further includes: During the decompilation process, if it is detected that a new installation package appears in the initial folder, the estimated remaining time of a plurality of the decompilation processes is queried; The target folder corresponding to the decompilation process with the shortest estimated remaining time is used as the folder to be updated, and the newly added installation package is allocated to the folder to be updated.
7. The method according to claim 1, characterized in that The step of scanning the target files to be inspected respectively and generating a sensitive scanning report corresponding to the initial installation package based on the scanning results specifically includes: Determine the file type of the target file to be inspected; If the target file to be inspected is a text file, based on the memory mapping technology, the text file is mapped into the memory for scanning to obtain the file scanning result; If the target file to be inspected is a binary file, extract the string sequence of the binary file as the part to be scanned to obtain the file scanning result; A sensitive scanning report corresponding to the initial installation package is generated based on the scanning result, and the sensitive scanning report is displayed.
8. A detection system, characterized in that: The detection system comprises: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors call the computer instructions so that the detection system executes the method as described in any one of claims 1-7.
9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a detection system, the detection system is caused to execute the method according to any one of claims 1 to 7.
10. A computer program product, characterized in that When the computer program product is run on a detection system, the detection system is caused to perform the method according to any one of claims 1 to 7.
